Keeping OEM Honest: Automating OEM Target Maintenance After Oracle Out-of-Place Patching of Grid and DBMS
As Oracle DBAs, we spend a significant amount of time patching Grid Infrastructure, database homes, and supporting components across large server estates. One challenge I’ve encountered repeatedly is that Oracle Enterprise Manager (OEM) does not always reflect the actual Oracle Home locations after out-of-place patching activities. While the software components on the server may be fully patched and operational, OEM target properties such as OracleHome, ASM Home, listener home, and related metadata can still reference older installations. This leads to inventory inconsistencies, confusing reports, and unnecessary troubleshooting effort when teams rely on OEM as the source of truth.
To address this problem, I developed a secure OEM Post-Patching Target Synchronization Utility. The script connects to target hosts, discovers the actual Grid Infrastructure, ASM, Listener, and Database homes directly from the server, compares them against the values currently stored in OEM, and updates only the targets that are out of sync. The utility supports both production and non-production OEM environments, allows targeted or bulk operations, provides a safe reporting mode for validation, and incorporates encrypted credential management so OEM passwords are never stored in clear text.
Over time, the script evolved well beyond a simple synchronization tool. It now supports multiple hosts in a single execution, full estate processing across all OEM hosts, environment-specific EMCLI clients, encrypted OpenSSL-based credential storage, reporting and summary modes, and safety controls designed to minimize operational risk. The goal was to create a utility that can be safely used during patching cycles while reducing manual effort and ensuring OEM accurately reflects the real server configuration.
Key Features
OEM Synchronization
- Automatically discovers the active Grid Infrastructure home from the target server.
- Retrieves actual Listener homes using srvctl.
- Retrieves actual Database homes using srvctl.
- Compares discovered values against OEM target properties.
- Updates OEM only when mismatches are detected.
Target Selection
- Process all supported OEM targets.
- Process only Database targets.
- Process only Listener targets.
- Process only ASM targets.
- Process only HAS targets.
- Supports host-level and target-level filtering.
Host Processing Options
- Process a single host.
- Process multiple hosts using a comma-separated host list.
- Process all OEM hosts using –all-hosts.
Reporting & Validation
- –report mode performs a complete validation without modifying OEM.
- –summary mode provides a concise per-host compliance report.
- Summary mode automatically enforces read-only behavior to prevent accidental updates.
- Displays environment information before execution to ensure the correct OEM environment is being used.
Security
- Uses encrypted credential storage based on OpenSSL AES-256-CBC.
- Stores encryption keys separately from credential files.
- Supports credential rotation through –init-wallet.
- Does not use emcli setup.
- Does not modify EMCLI configuration directories.
- Uses environment-specific EMCLI clients for PROD and DEV isolation.
Operational Enhancements
- Supports production and non-production OEM environments.
- Provides host-by-host processing and reporting.
- Includes dry-run capabilities for safe validation.
- Designed for large-scale post-patching operations across many servers.
Implementation & Source Code
The complete source code is available in my GitHub repository. If you’re managing Oracle environments with OEM and looking for a reliable way to keep target metadata synchronized after patching, feel free to explore the project and adapt it to your own environment.
However, before using the OEM Post-Patching Synchronization Utility, there are a few prerequisites that must be in place. The script communicates with Oracle Enterprise Manager using EMCLI, connects to Oracle database servers over SSH, and uses OpenSSL for encrypted credential management. The good news is that once the initial setup is complete, onboarding new users and rotating credentials becomes a simple and repeatable process.
Prerequisites
1. EMCLI Client
The utility leverages Oracle Enterprise Manager Command Line Interface (EMCLI) to query and update OEM target properties. Oracle provides EMCLI as a downloadable client that can be installed independently of the OMS server. The EMCLI client can be downloaded from the OEM console under Setup => Command Line Interface => Download EMCLI Kit.
Oracle documents that EMCLI can be installed on any managed system and configured to communicate with a specific OMS. A typical Linux installation looks like:
$ export EMCLI_HOME=/u01/app/oem_mgmt/emcli_prd
$ export OEM_URL=https://oemprd.parsabahrami.com:7803/em
$ mkdir -p $EMCLI_HOME
$ wget --no-check-certificate ${OEM_URL}/public_lib_download/emcli/kit/emcliadvancedkit.jar
$ java -jar emcliadvancedkit.jar client -install_dir=${EMCLI_HOME}
$ cd $EMCLI_HOME
$ export EMCLI_STATE_DIR=${EMCLI_HOME}
$ ./emcli setup -url=${OEM_URL} -username=sysman -trustall -autologin -dir=${EMCLI_HOME}
$ ./emcli syncRepeat the process for each OEM environment you intend to manage. In my scenario I needed to have one for OEM Prod and one for OEM non-prod, so they ended up as:
/u01/app/oem_mgmt/emcli_dev
/u01/app/oem_mgmt/emcli_prdThis will then allow you to use the –env parameter to connect to the correct OEM environment.
2.SSH Connectivity
The server running the script must be able to connect to all target database hosts using SSH. The SSH user and key are configurable at the top of the script.
Initial Credential Setup
The first time you use the script, initialize the encrypted credential wallet. For example, to setup the wallet for the DEV OEM environment you will need to run:
$ ./update_oem_targets.sh \
--env dev \
--init-walletYou will be prompted for:
- OEM username
- OEM password
- Confirm password
You would have to repeat the same thing for the PRD environment if you need to. After initialization, the script creates the following files:
.oem_wallet/
├── oem_wallet.key
├── dev.cred.enc
└── prd.cred.encoem_wallet.key: Contains the encryption key used to decrypt OEM credentials.
dev.cred.enc: Encrypted DEV OEM credentials.
prd.cred.enc: Encrypted PROD OEM credentials.
The actual OEM password is never stored in clear text.
Examples
After completing the above steps, you’re ready to use the utility in either DEV or PROD and begin keeping OEM target metadata synchronized with the actual Oracle infrastructure configuration. To see the full array of parameters that can be passed to the script, please use the --help, however, here are some examples:
Single Host Report Mode
$ ./update_oem_targets.sh \
--env dev \
--hosts dbsrv1,dbsrv2,dbsrv3 \
--reportMultiple Hosts Summary Mode
$ ./update_oem_targets.sh \
--env dev \
--hosts dbsrv1,dbsrv2,dbsrv3 \
--summarySynchronizing Only Listener and ASM Target Type for Multiple Hosts
$ ./update_oem_targets.sh \
--env dev \
--hosts dbsrv1,dbsrv2,dbsrv3 \
--listener --asmSynchronizing all Target Types for Multiple Hosts
$ ./update_oem_targets.sh \
--env dev \
--hosts dbsrv1,dbsrv2,dbsrv3 \
--all-targetsSynchronizing all Target Types for All Hosts Registered with OEM
$ ./update_oem_targets.sh \
--env dev \
--all-hosts \
--all-targetsFinal Thoughts
This utility started as a simple solution to a recurring post-patching problem and gradually evolved into a secure OEM synchronization framework. While every environment is slightly different, the overall goal remains the same: ensuring OEM accurately reflects the reality of the Oracle software stack running on each server.
The complete source code is available in my GitHub repository. If you decide to try it out, I’d love to hear your feedback, suggestions, and enhancement ideas.