This note details the procedure used to upgrade an OEM Grid Control 10.2.0.5 environment on Linux to Oracle 12c Enterprise Manager Cloud Control.

Broadly speaking at a high level, these are the main steps involved in the 12c upgrade.

• Decide on what upgrade you are going to follow. I have used the 2-System Upgrade approach which although requires an additional server for the 12g OMS, is non-intrusive and does not affect the existing 10g OEM environment.
• Apply the pre-upgrade console patch on existing 10g OEM – this will provide us the interface in the existing 10g OEM Grid Control for performing many of the upgrade tasks. Note the 10g OMS needs to down for this.
• Provide information about the host where the 12c upgrade is going to take place.
• Download the 12c Grid Control software and plug-ins and stage them on the server which is going to host the 12c OMS
• Register the software and plug-ins with Grid Control
• Analyze the existing management agents and determine which agents are having an upgradeable status – note that currently there are no Windows 12c Management Agents available. On Metalink it is mentioned that Windows 12c agents will be available by June 2012.
• Backup the existing 10g Repository Database and restore it on the same host or another host. In our case, I have restored the 10.2.0.5 OEM repository database on the server which will host the 12c OMS. So both repository database and OMS are on the same host. We can perform an upgrade of the database here if we so desire, but 10.2.0.5 repository database is supported with the 12c Grid Control.
• Install 12c Enterprise Manager Cloud Control and upgrade the Management Repository in the database you backed up and restored.
• Link the earlier release of the Management Repository with the upgraded Management Repository.
• Upgrade the management agents by deploying the 12c agent software on the hosts where we are upgrading the existing 10g agent. All these tasks are done from the Upgrade Console in the 10g OEM Grid Control. We can do this in a piecemeal manner and do not have to upgrade all the agents to 12c in one go. So we will have both the 10g and 12c Grid Control environments running side by side until we have migrated all the 10g agents to 12c. Note that the 12c OMS can only communicate with the 12c agent and not any other agent version.
• Generate a health report and check the readiness of the redeployed Management Agents. Verify and sign off the health report.
• Switch over the old Management Agent to the newly deployed 12c agent
• Note – Ensure that you have the Enterprise Manager Cloud Control software released in February 2012 – Enterprise Manager Cloud Control 12c Release 1 (12.1.0.1) (With Bundle Patch 1).

Read the note: Upgrading Grid Control 10.2.0.5 to OEM 12c Cloud Control

 Here are some of my notes  on Installing 12c Cloud Control on a Linux x86_64 platform.

In a previous note I described the process of  upgrading 10.2.0.5 Grid Control to 12c Cloud Control.

If you have installed 11g Grid Contro;l, then the process is quite similar. The real difference is the screen related to selecting Plug-ins which is something new in 12c. Plug-ins and Connectors are extensions to Grid Control to enable management of non-Oracle products and the complete ‘cloud’ stack which includes databases, middleware and even virtual servers.

We have plug-ins for Oracle like Oracle Database, Oracle Exadata, My Oracle Support, Oracle Fusion Middleware, Oracle Virtualisation and also a number of no-Oracle plug-ins like IBM DB2, Sybase etc

Packages required:

• make-3.81
• binutils-2.17.50.0.6
• gcc -4.1.1
• libaio-0.3.106
• glibc-common-2.3.4
• libstdc++ -4.1.1
• setarch-1.6
• sysstat-5.0.5
• rng-utils-2.0
• libXtst-1.0.1-3.1(x86_64)
• glibc-2.5.12

In addition, install the 32-bit version as well as the 64-bit version of glibc-devel-2.5-49

Port Requirements

•Admin Server HTTP SSL Port = 7101 – 7200

•Enterprise Manager Upload HTTP Port = 4889 – 4898

•Enterprise Manager Upload HTTP SSL Port = 1159, 4899 – 4908

•Enterprise Manager Central Console HTTP Port = 7788 – 7798

•Enterprise Manager Central Console HTTP SSL Port = 7799 – 7809

•Oracle Management Agent Port = 3872, 1830 – 1849

Repository Database requirements

10.2.0.5 database can be used as repository for cloud control

If you use Oracle Database 11g Release 2 (11.2.0.1), then ensure that you apply the patches for bugs 10014178 and 8799099

For fresh installs, database should not have SYSMAN schema. If Database Control exists in the dtaabase chosen for the OMS repository, then we need to use the emca -deconfig dbcontrol db -repos drop command to deconfigure Database Control first.

open_cursors  needs  to be 300 to 400

processes  should be greater than  300

session_cached_cursors between 200 and 500

job_queue_interval needs to be at least 20

shared pool size minimum 600 MB

Redo log file sizes minimum 300 MB

UNDO tablespace has a minimum space of 200 MB

 stop the Gather Statistics job

For 10g use :

execute dbms_scheduler.disable(‘GATHER_STATS_JOB’,TRUE);

execute dbms_scheduler.stop_job(‘GATHER_STATS_JOB’,TRUE);

For 11g use :

execute dbms_auto_task_admin.disable(‘auto optimizer stats collection’,null,null);

Use the Enterprise Manager Prerequisite Kit utility (emprereqkit)

The emprereqkit utlity is available  in the 12c software under the directory location install/requisites/bin/emprereqkit. After installation also, it can be run if say we want to upgrade to the next higher version and now need to check if the repository meets the minimum requirements.

It will be located under the OMS_HOME at /install/requisites/bin

It is a command line utility which we can run to test if our repoitory database for the 12c OMS meets all the requirements before we start the installation.

An example of running this is shown here:

$ ./emprereqkit -executionType install -prerequisiteXMLLoc $ORACLE_HOME/install/requisites/list -dbHost oem-prod -dbPort 1521 -dbSid oem12c -dbUser SYS -dbPassword syspwd -dbRole sysdba -showPrereqs

Also …

Ensure the directory selected for the Middleware Home Location does not contain any files or subdirectories

If not already present, Oracle WebLogic Server 11g Release 1 (10.3.5) and JDK 1.6 v24 are installed in the Middleware home

A WebLogic domain called GCDomain is created

Outlined here is the process used to deploy the 12c management agent on a target Linux x86_64 server using the silent installation method with an agent response file.

In this case the 12c  OMS host server is called linux-oem-prod and the target host where we are deploying the agent is .linux-orasql-001-dev.

In the next post we will see how we can deploy or push the 12c management agents to target hosts from 12c Cloud Control itself.

On the OMS host launch the EMCLI client

[oracle@linux-oem-prod Middleware]$ cd oms

 [oracle@linux-oem-prod oms]$ cd bin

 [oracle@linux-oem-prod bin]$ ./emcli login -username=sysman -password=xxx

Login successful

Synchronize EMCLI

[oracle@linux-oem-prod bin]$ ./emcli sync

Synchronized successfully

Check the platforms for which the Management Agent software is available on the OMS host

[oracle@linux-oem-prod bin]$ ./emcli get_supported_platforms

Getting list of platforms …

Check the logs at /u01/app/oracle/Middleware/oms/bin/agent.log

About to access self-update code path to retrieve the platforms list..

Getting Platforms list  …

———————————————–

Version = 12.1.0.1.0

 Platform = Linux x86-64

———————————————–

Platforms list displayed successfully.

Download the Management Agent software from Oracle Software Library to a temporary directory on the OMS host

[oracle@linux-oem-prod bin]$ ./emcli get_agentimage -destination=/u01/app/oracle/agent_software -platform=”Linux x86-64″ -version=”12.1.0.1.0″

Platform:Linux x86-64

Destination:/u01/app/oracle/agent_software

 === Partition Detail ===

Space free : 14 GB

Space required : 1 GB

Check the logs at /u01/app/oracle/agent_software/get_agentimage_2012-04-15_22-50-00-PM.log

Setting property ORACLE_HOME to:/u01/app/oracle/Middleware/oms

calling pulloneoffs with arguments:/u01/app/oracle/Middleware/oms/u01/app/oracle/Middleware/oms/sysman/agent/12.1.0.1.0_AgentCore_226.zip12.1.0.1.0linux_x64

Check this logs for more information: /u01/app/oracle/Middleware/oms/sysman/prov/agentpush/logs

[oracle@linux-oem-prod bin]$ cd /u01/app/oracle/agent_software

[oracle@linux-oem-prod agent_software]$ ls -l

total 230400

-rw-r—– 1 oracle oinstall 235679029 Apr 15 22:50 12.1.0.1.0_AgentCore_226.zip

-rw-r–r– 1 oracle oinstall      1805 Apr 15 22:50 get_agentimage_2012-04-15_22-50-00-PM.log

Transfer the downloaded zip file to the host where we would like to install the 12c management agent

[oracle@linux-oem-prod agent_software]$ scp -rp 12.1.0.1.0_AgentCore_226.zip oracle@linux-orasql-001-dev.dev.domain:/u01/app/oracle/agent_software

Unzip the file on the target host

Check the directory contents

[oracle@linux-orasql-001-dev agent_software]$ ls -l

total 460420

-rw-r—– 1 oracle oinstall 235679029 Apr 16 10:50 12.1.0.1.0_AgentCore_226.zip

-rwxr-xr-x 1 oracle oinstall   4979329 Apr 16 10:50 12.1.0.1.0_PluginsOneoffs_226.zip

-rw-rw-r– 1 oracle oinstall 230621327 Sep 23  2011 agentcoreimage.zip

-rwxrwxr-x 1 oracle oinstall     17150 Sep 23  2011 agentDeploy.sh

-rw-rw-r– 1 oracle oinstall        91 Sep 23  2011 agentimage.properties

-rw-rw-r– 1 oracle oinstall      3856 Sep 23  2011 agent.rsp

drwxr-xr-x 7 oracle oinstall      4096 Apr 16 10:50 plugins

-rwxr-xr-x 1 oracle oinstall       223 Apr 16 10:50 plugins.txt

-rwxr-xr-x 1 oracle oinstall    145976 Sep 23  2011 unzip

Edit the response file agent.rsp

OMS_HOST=”linux-oem-prod.dev.domain”

EM_UPLOAD_PORT=”4902″

AGENT_REGISTRATION_PASSWORD=”xxx”

AGENT_INSTANCE_HOME=”/u01/app/oracle/agent12g/agent_inst”

AGENT_PORT=”3872″

ORACLE_HOSTNAME=”linux-orasql-001-dev.dev.domain”

Run the agent deployment script

[oracle@linux-orasql-001-dev agent_software]$ ./agentDeploy.sh AGENT_BASE_DIR=/u01/app/oracle/agent12g RESPONSE_FILE=/u01/app/oracle/agent_software/agent.rsp 

 Validating the OMS_HOST & EM_UPLOAD_PORT

Executing command : /u01/app/oracle/agent12g/core/12.1.0.1.0/jdk/bin/java -classpath /u01/app/oracle/agent12g/core/12.1.0.1.0/jlib/agentInstaller.jar:/u01/app/oracle/agent12g/core/12.1.0.1.0/oui/jlib/OraInstaller.jar oracle.sysman.agent.installer.AgentInstaller /u01/app/oracle/agent12g/core/12.1.0.1.0 /u01/app/oracle/agent_software /u01/app/oracle/agent12g -prereq

 Validating oms host & port with url: http://linux-oem-prod.dev.domain:4902/empbs/genwallet

Validating oms host & port with url: https://linux-oem-prod.dev.domain:4902/empbs/genwallet

Return status:3

Unzipping the agentcoreimage.zip to /u01/app/oracle/agent12g ….

12.1.0.1.0_PluginsOneoffs_226.zip

Executing command : /u01/app/oracle/agent_software/unzip -o /u01/app/oracle/agent_software/12.1.0.1.0_PluginsOneoffs_226.zip -d /u01/app/oracle/agent12g

 Checking the ownership of agent base directory:/u01/app/oracle/agent12g

Checking for proper ownership on the agent base directory.

Checks whether the agent base directory is owned by the agent user oracle  and that its parent directory is owned by either the agent user or root.

….

Login name is : oracle & file owner is : oracle

…………

Agent Base directory verification completed Successfully.

 Ownership check completed.

Executing command : /u01/app/oracle/agent12g/core/12.1.0.1.0/jdk/bin/java -classpath /u01/app/oracle/agent12g/core/12.1.0.1.0/oui/jlib/OraInstaller.jar:/u01/app/oracle/agent12g/core/12.1.0.1.0/oui/jlib/xmlparserv2.jar:/u01/app/oracle/agent12g/core/12.1.0.1.0/oui/jlib/srvm.jar:/u01/app/oracle/agent12g/core/12.1.0.1.0/oui/jlib/emCfg.jar:/u01/app/oracle/agent12g/core/12.1.0.1.0/jlib/agentInstaller.jar:/u01/app/oracle/agent12g/core/12.1.0.1.0/oui/jlib/share.jar oracle.sysman.agent.installer.AgentInstaller /u01/app/oracle/agent12g/core/12.1.0.1.0 /u01/app/oracle/agent_software /u01/app/oracle/agent12g AGENT_BASE_DIR=/u01/app/oracle/agent12g AGENT_BASE_DIR=/u01/app/oracle/agent12g RESPONSE_FILE=/u01/app/oracle/agent_software/agent.rsp

 Executing agent install prereqs…

Executing command: /u01/app/oracle/agent12g/core/12.1.0.1.0/oui/bin/runInstaller -ignoreSysPrereqs -prereqchecker -silent -ignoreSysPrereqs -waitForCompletion  -prereqlogloc /u01/app/oracle/agent12g/core/12.1.0.1.0/cfgtoollogs/agentDeploy -entryPoint oracle.sysman.top.agent_Complete -detailedExitCodes PREREQ_CONFIG_LOCATION=/u01/app/oracle/agent12g/core/12.1.0.1.0/prereqs  -J-DORACLE_HOSTNAME=linux-orasql-001-dev.dev.domain

Prereq Logs Location:/u01/app/oracle/agent12g/core/12.1.0.1.0/cfgtoollogs/agentDeploy/prereq<timestamp>.log

Agent install prereqs completed successfully

Cloning the agent home…

Executing command: /u01/app/oracle/agent12g/core/12.1.0.1.0/oui/bin/runInstaller -ignoreSysPrereqs -clone -forceClone -silent -waitForCompletion -nowait ORACLE_HOME=/u01/app/oracle/agent12g/core/12.1.0.1.0 -responseFile /u01/app/oracle/agent_software/agent.rsp  AGENT_BASE_DIR=/u01/app/oracle/agent12g AGENT_BASE_DIR=/u01/app/oracle/agent12g RESPONSE_FILE=/u01/app/oracle/agent_software/agent.rsp -noconfig  ORACLE_HOME_NAME=agent12g1 -force

Clone Action Logs Location:/u01/app/oraInventory/logs/cloneActions<timestamp>.log

Cloning of agent home completed successfully

Attaching sbin home…

Executing command: /u01/app/oracle/agent12g/core/12.1.0.1.0/oui/bin/runInstaller -ignoreSysPrereqs -attachHome -waitForCompletion -nowait ORACLE_HOME=/u01/app/oracle/agent12g/sbin ORACLE_HOME_NAME=sbin12g1 -force

Attach Home Logs Location:/u01/app/oracle/agent12g/core/12.1.0.1.0/cfgtoollogs/agentDeploy/AttachHome<timestamp>.log

Attach home for sbin home completed successfully.

Updating home dependencies…

Executing command: /u01/app/oracle/agent12g/core/12.1.0.1.0/oui/bin/runInstaller -ignoreSysPrereqs -updateHomeDeps -waitForCompletion HOME_DEPENDENCY_LIST=”/u01/app/oracle/agent12g/sbin:/u01/app/oracle/agent12g/core/12.1.0.1.0″ -invPtrLoc /u01/app/oracle/agent12g/core/12.1.0.1.0/oraInst.loc -force

Update Home Dependencies Location:/u01/app/oracle/agent12g/core/12.1.0.1.0/cfgtoollogs/agentDeploy/UpdateHomeDeps<timestamp>.log

Update home dependency completed successfully.

Performing the agent configuration…

Executing command: /u01/app/oracle/agent12g/core/12.1.0.1.0/oui/bin/runConfig.sh ORACLE_HOME=/u01/app/oracle/agent12g/core/12.1.0.1.0 RESPONSE_FILE=/u01/app/oracle/agent12g/core/12.1.0.1.0/agent.rsp ACTION=configure MODE=perform COMPONENT_XML={oracle.sysman.top.agent.11_1_0_1_0.xml} RERUN=true

Configuration Log Location:/u01/app/oracle/agent12g/core/12.1.0.1.0/cfgtoollogs/cfgfw/CfmLogger<timestamp>.log

Agent Configuration completed successfully

The following configuration scripts need to be executed as the “root” user.

#!/bin/sh

#Root script to run

 /u01/app/oracle/agent12g/core/12.1.0.1.0/root.sh

To execute the configuration scripts:

1. Open a terminal window

2. Log in as “root”

3. Run the scripts

Agent Deployment Successful.

Agent deployment log location:

/u01/app/oracle/agent12g/core/12.1.0.1.0/cfgtoollogs/agentDeploy/agentDeploy_<timestamp>.log

Agent deployment completed successfully.

Run the root.sh script

[root@linux-orasql-001-dev ~]#  /u01/app/oracle/agent12g/core/12.1.0.1.0/root.sh

Finished product-specific root actions.

/etc exist

Finished product-specific root actions.

Check the status of the agent on the target host where it has been deployed

[oracle@linux-orasql-001-dev oracle]$ pwd

/u01/app/oracle

[oracle@linux-orasql-001-dev oracle]$ cd agent12g/

[oracle@linux-orasql-001-dev agent12g]$ cd core

[oracle@linux-orasql-001-dev core]$ cd 12.1.0.1.0/bin

[oracle@linux-orasql-001-dev bin]$ ./emctl status agent

Oracle Enterprise Manager 12c Cloud Control 12.1.0.1.0

Copyright (c) 1996, 2011 Oracle Corporation.  All rights reserved.

—————————————————————

Agent Version     : 12.1.0.1.0

OMS Version       : 12.1.0.1.0

Protocol Version  : 12.1.0.1.0

Agent Home        : /u01/app/oracle/agent12g/agent_inst

Agent Binaries    : /u01/app/oracle/agent12g/core/12.1.0.1.0

Agent Process ID  : 2678

Parent Process ID : 2631

Agent URL         : https://linux-orasql-001-dev.dev.domain:3872/emd/main/

Repository URL    : https://linux-oem-prod.dev.domain:4902/empbs/upload

Started at        : 2012-04-16 11:20:59

Started by user   : oracle

Last Reload       : (none)

Last successful upload                       : 2012-04-16 11:31:33

Last attempted upload                        : 2012-04-16 11:31:33

Total Megabytes of XML files uploaded so far : 0.02

Number of XML files pending upload           : 0

Size of XML files pending upload(MB)         : 0

Available disk space on upload filesystem    : 45.31%

Collection Status                            : Collections enabled

Last attempted heartbeat to OMS              : 2012-04-16 11:32:06

Last successful heartbeat to OMS             : 2012-04-16 11:32:06

—————————————————————

Agent is Running and Ready

In an earlier post, I had mentioned the procedure for doing a silent install on the 12c management agent from the command line.

http://gavinsoorma.com/2012/05/12c-management-agent-silent-installation/

We will see here how we can install as well as deployment the management agent on single or multiple target hosts from within 12c Cloud Control itself.

 From the Cloud Control Setup menu, click Add Target and then Add Targets Manually

Select Add Host Targets and click Add Host

We can provide a name for the agent deployment session activity or leave it at the default value.

 Click Add and enter the host name as well as select the appropriate platform from the dropdown list. It is recommended to use a fully qualified host name which includes the domain name as well.

 Note – to deploy on multiple host targets, do not enter multiple hosts on the same line separated by commas. Instead each host must be entered on a separate row

On the Installation Details page,  we have entered the following:

 For Installation Base Directory we have entered the absolute path to the agent base directory under which the 12c management agent software will be installed. Ensure that this directory is empty.

 The Instance Directory is where all the management agent configuration files will be stored.

 For Named Credential, either use an existing profile or create a new one. These credentials are used for setting up SSH connectivity between the OMS and the managed hosts as well as installing the Management Agent on the target hosts.

 If we leave the Privileged Delegation Setting blank, then the root scripts will not be run and we have to manually run the same after the agent installation

 We can review the agent deployment details and then click on the Deploy Agent button

We can review the progress made on each of the phases of the agent deployment operation — Initialization, Remote Prerequisite Check, and Agent Deployment.

After the agent deployment has been completed we need to do the following

1. Run the agent root.sh script as root which is located in the Management Agent home.
2. Discover and Add Targets for monitoring via Cloud Control

From the Setup menu, select Configure Auto Discovery

Select Multiple Target-Type Discovery on Single Host and then click on Configure

Select the relevant Agent Host Name and click on Configure

Under the Discovery Module, we are checking the Enabled box for Oracle Database,Listener …. and Oracle Home Discovery

Click on the Run Discovery Now button

We see that for the host where we have just run the target discovery, out of the 5 Discovered Targets, so far 3 are Managed Targets. We still need to discover the Database Instance and the Listener.

Unlike the earlier OEM versions, in 12c we cannot download and install the Management Agent software by downloading the software from OTN.

 In 12c, we need to use the Cloud Control Self Update feature to install as well as patch the Management Agent.

 In this case, the 12c Cloud Control OMS is running on a Linux X86-64 platform and we are deploying the agent on a Windows 64 bit box.

 There are a couple of things we need to do first.

 a)     Setup and configure the 12c Software Library in Cloud Control

b)    Install the Bundle Patch 1 (13242773 )and patch 13707704.

c)     Follow the procedure described in the 12c Cloud Control Basic Installation Guide for installing Cygwin and Starting the SSH Daemon

http://docs.oracle.com/cd/E24628_01/install.121/e22624/preinstall_req_cygwin_ssh.htm#CBHIAFGI

The Software Repository is a file system based repository which stores software entities like software and patches, VM Images, referenced gold images, plug-ins etc.

 The Software Library facilitates provisioning and patching via Cloud Control in both Offline as well as Online mode.

 In the Online mode we need to configure Cloud Control with My Oracle Support details and obviously internet connectivity is required.

 In the example shown here, we will be using the Software Library in the Offline Patching mode.

 If we have a multiple OMS environment, then the file system location needs to be one which is shared or mounted across all the Oracle Management Server hosts.

 In a single OMS environment, the file system for the Software Library needs to reside on the host where the 12c OMS is running.

 From the Setup menu, select Provisioning and Patching, then select Software Library.

Read the document -Deploying the 12c Management Agent on Windows

 In an earlier post I had described how to use the Self Update feature of 12c Cloud Control to deploy the 12c management agent on a Windows 64 bit server.

Let us now see how to can deploy the 12c Plug-in on the same Windows server to enable us to monitor a Microsoft SQL Server 2008 environment running on the same Windows server.

 A plug-in is an additional component which can be plugged into an existing 12c Cloud Control installation in order to extend the default out-of-the-box management and monitoring capabilities.

 The plug-ins as well as updates to the plug-ins are released from time to time and these plug-ins enable us to monitor both Oracle as well as non-Oracle database and applications.

 Deploying a plug-in essentially consists of the following:

 a)     Download the plug-in archive and store it in the Software Library configured on the OMS host

b)    Deploy the plug-in first to the OMS host which manages and monitors the target pertaining to the particular plug-in type. In this example the plug-in is Microsoft SQL Server.

c)     Update the OMS repository with metadata information about the plug-in

d)    Deploy the plug-in on the Management Agent

 Note that in this example we will be using the Offline Patching mode to update the Software Library.

From the Setup menu, choose Extensibility, then select Plug-ins.

From the Actions drop-down menu, select Check Updates 

This will take up to the Self Update page. Note our Connection Mode is Offline

Click on Plug-In

The Plug-in Updates page will show us all the updates available for the various plug-ins. We can see that there is a newer version of the plug-in which is available for Microsoft SQL Server Database.

Select that line and click on Download

Since we using the Offline Patching mode, we need to download the file using the URL shown and then copy that file to the OMS host.

We will then use the EMCLI to run the command to import the plug-in metadata into the OMS rtepository.

From the OMS $ORACLE_HOME/bin run the following emcli commands:

[oracle@kens-oem-prod bin]$ ./emcli login -username=”sysman” -password=”xxx”
Login successful
[oracle@kens-oem-prod bin]$ ./emcli import_update -omslocal -file=”/u01/stage/p14047236_112000_Generic.zip”

Processing update: Plug-in – Microsoft SQL Server Plugin for monitoring SQL Server database from Enterprise Manager
Operation completed successfully. Update has been uploaded to Enterprise Manager. Please use the Self Update Home to manage this update.

We will now see that  the plug-in status has now changed to Downloaded from Available for Microsoft SQL Server Database

Click on Plug-in

Now click on Deploy On and select Management Servers…

The Microsoft SQL Server Database updated plug-in will now be deployed on the Management Server.

We can monitor the progress of the plug-in deployment on the Management Server – click on the Show Status button.

Once the deployment job is completed, we can see that the column “On Management Server” has been populated with the Microsoft SQL Server Database plug-in details.

We now have to deploy the plug-in on the Management Agent.

Click on Deploy On drop-down menu and select Management Agent …

Select the Management Agent where this plug-in needs to be deployed.

Click on the Add and select the Management Agent and its associated Operating System.

Click on Continue

Click on Next

Click on Deploy button

Click on Setup, Add Targets, Add Targets Manually

Select the option Add Non-Host Targets by Specifying Target Monitoring Properties

In the Target Type field, select Microsoft SQL Server.

In the Monitoring Agent field , select the agent deployed on the Windows server hosting the SQL Server instance to be monitored.

Here we will need to now enter the SQL Server credentials for the sa database user account as well as the TCP/IP Port number for SQL Server

Click on the Test Connection button

From the Targets menu, Select All Targets. We can now see the SQL Server target we configured earlier.

I recently upgraded EM 12c version 12.1.0.1 to EM 12c Release 2 (12.1.0.2) on a Linux 64 bit platform. Here are few of the things to keep in mind and some notes I made which may be helpful to others planning the similar upgrade.

The 12c Release 2 software is made up of three zip files. Ensure we have between 10-15 GB free in the stage area where we are going to unzip the files.

em12cr2_linux64_disk1.zip
em12cr2_linux64_disk2.zip
em12cr2_linux64_disk3.zip

The 12.1.0.1 to 12.1.0.2 upgrade is an out of place or what is called a 1 system upgrade.

The 1-system upgrade approach upgrades Enterprise Manager 12c Cloud Control on the same host—upgrades Oracle Management Service (OMS) on the same host and Oracle Management Repository (Management Repository) in the existing database. Since the upgrade happens on the same host, there will be downtime involved in the upgrade

Ensure we have at least 10 GB free on the OMS host for the 12c Release 2 Middleware Home. When we upgrade the 12c management agent we also need to ensure that there is a minimum of 1 GB of free disk space in the 12c Release 1 Agent Home.

Before the upgrade

Ensure that we have take a backup up the 12c Release 1 OMS (the middleware home and the inventory), the Management Repository, and the Software Library.

Ensure that the tables in the Management Repository do not have any snapshots created.

Log in to the Management Repository and run the following SQL query as SYSMAN user:

SQL> select master, log_table from all_mview_logs where log_owner=’SYSMAN’;

no rows selected

If there are snapshots, then drop them by running the following command as SYSMAN user:

SQL> Drop snapshot log on <master> ;

Copy the emkey from the existing OMS to the existing Management Repository

[oracle@kens-oem-prod bin]$ ./emctl config emkey -copy_to_repos
Oracle Enterprise Manager Cloud Control 12c Release 12.1.0.1.0
Copyright (c) 1996, 2012 Oracle Corporation. All rights reserved.
Enter Enterprise Manager Root (SYSMAN) Password :
The EMKey has been copied to the Management Repository. This operation will cause the EMKey to become unsecure.
After the required operation has been completed, secure the EMKey by running “emctl config emkey -remove_from_repos”.
[oracle@kens-oem-prod bin]$

[oracle@kens-oem-prod bin]$ ./emctl status emkey
Oracle Enterprise Manager Cloud Control 12c Release 12.1.0.1.0
Copyright (c) 1996, 2012 Oracle Corporation. All rights reserved.
Enter Enterprise Manager Root (SYSMAN) Password :
The EMKey is configured properly, but is not secure. Secure the EMKey by running “emctl config emkey -remove_from_repos”.
[oracle@kens-oem-prod bin]$

Shut down the Management Agent that monitors the Management Services and Repository target

Launch the 12c R2 installer from the location where we have unzipped the 12.1.0.2 software.

Enter the location of the 12c R2 Middleware and OMS home – so we have to enter a new location since this is a One-System approach

At this point we need to stop the OMS.

$ <OMS_HOME>/bin/emctl stop oms

Make a note of the current value of the parameter job_queue_processes as the upgrade resets it to 0. After the upgrade check if the value has changed from 0 to the original pre-upgrade value

The deployed plug-ins are upgraded if newer versions are available in the Enterprise Manager Cloud Control 12c Release 2 (12.1.0.2.0) software.

If newer versions are not available in the Enterprise Manager Cloud Control 12c Release 2 (12.1.0.2.0) software, then existing deployed plug-ins are carried over without upgrading them.

Java Development Kit (JDK) 1.6 v24 and Oracle WebLogic Server 11g Release 1 (10.3.5) are installed by the upgrade process if they are not already available in the Middleware home we specify.

A new Oracle WebLogic domain is created using the existing Administration Server configuration details .

It creates a new Oracle Management Service Instance Base directory (gc_inst) for storing all configuration details related to Oracle Management Service 12c R2.

After the Upgrade

The installer does NOT upgrade the existing 12.1.0.1 Management Agent that was installed with the OMS. You must upgrade it (and all other Management Agents) using the Upgrade Agents Console.

The ports used by the earlier release of the Management Agents are carried over to the upgraded Management Agents so we do not need to make any changes to our firewall settings as such.

The 12.1.0.1 agent is compatible with the EM 12c Release 2 OMS. So we can still continue using the 12.1.0.1 management agents with the 12.1.0.2 OMS, but a preferred option would be to upgrade the existing 12c Release 1 agents.

After the upgrade we can see that an additional item has been added to the SetUp menu which is Manage Cloud Control.

From the Manage Cloud Control menu option we select Upgrade Agents.

We can now see a list of all our existng 12.1.0.1 agents.

Select the agent we would like to upgrade. In this case Privilege Delefation setting has not been setup for this particular host so we need to run the root.sh manually.

Note that after the upgrade a another directory called 12.1.0.2.0 is created under the AGENT_HOME base/core.

To start and stop the 12c R2 agent we need to ensure that we are now using the emctl executable located under for example:

/u01/app/agent12c/core/12.1.0.2.0/bin

[root@kens-oem-prod 12.1.0.2.0]# pwd
/u01/app/em12c/Middleware/agent/core/12.1.0.2.0

[root@kens-oem-prod 12.1.0.2.0]# ./root.sh
Finished product-specific root actions.
/etc exist
Finished product-specific root actions.
[root@kens-oem-prod 12.1.0.2.0]#

[root@kens-orasql-001-dev 12.1.0.2.0]# ./replacebins.sh
Replaced sbin executables

[oracle@kens-oem-prod bin]$ ./emctl status agent

Oracle Enterprise Manager Cloud Control 12c Release 2
Copyright (c) 1996, 2012 Oracle Corporation. All rights reserved.
—————————————————————
Agent Version : 12.1.0.2.0
OMS Version : 12.1.0.2.0
Protocol Version : 12.1.0.1.0
Agent Home : /u01/app/em12c/Middleware/agent/agent_inst
Agent Binaries : /u01/app/em12c/Middleware/agent/core/12.1.0.2.0
Agent Process ID : 6725
Parent Process ID : 6673
Agent URL : https://kens-oem-prod.corporateict.domain:3872/emd/main/
Repository URL : https://kens-oem-prod.corporateict.domain:4900/empbs/upload
Started at : 2012-11-13 22:33:30
Started by user : oracle
Last Reload : 2012-11-13 22:33:53
Last successful upload : 2012-11-13 22:36:06
Last attempted upload : 2012-11-13 22:36:06
Total Megabytes of XML files uploaded so far : 0.05
Number of XML files pending upload : 0
Size of XML files pending upload(MB) : 0
Available disk space on upload filesystem : 13.80%
Collection Status : Collections enabled
Heartbeat Status : Ok
Last attempted heartbeat to OMS : 2012-11-13 22:35:54
Last successful heartbeat to OMS : 2012-11-13 22:35:54
Next scheduled heartbeat to OMS : 2012-11-13 22:36:54

—————————————————————
Agent is Running and Ready

This note describes the procedure of implementing the GoldenGate plug-in for Oracle Cloud Control 12c.

After deploying the GoldenGate Plug-in we can see a new Target Type “GoldenGate” appearing in the OEM 12c Targets menu and can now monitor the status and progress of the Extract, Manager and Replicat processes running in a particular GoldenGate environment.

Download the note: Deploying the Oracle GoldenGate Plug-in on OEM 12c Cloud Control

Let us have a look at the Provisioning and Patching feature in OEM 12c.

We have a ‘gold image’ of a 11.2.0.3 Oracle Database Software where the Jan 2013 PSU has been installed.

We now want to deploy this particular 11.2.0.3 version of the database software on a number of target hosts.

Note – the deployment procedure outlined below assumes that  the 12c management agent has been installed on the target hosts and the Software Library has been setup and configured in OEM 12c.

An Oracle white paper on this subject has recently been released.

Provisioning & Patching Oracle Database using Enterprise Manager 12c
Here is an overview of the steps involved.

From the menu Enterprise- Provisioning and Patching – Software Library

Create folder to hold our 11.2.0.3 database software gold image

We will now add a component to this folder

Select Oracle Database Software Clone for the component type

Enter some details to describe the Oracle Database software being cloned

Select the Oracle Home Location you will use to create the gold image from. These are existing Oracle Homes hosted on servers where 12c agents are currently running

In the Create Component from list box, select Reference Oracle Home because we are going to use this particular Oracle Home as the basis for the software clone or gold image which we are then going to store in the Software Library.

Review the information related to the Create Oracle Database Software Clone.

Click the Save and Upload button.

We can see that the component has been added to the Software Library but the status is showing Incomplete because the clone job is still not completed.

From the Job Activity menu we can monitor the progress of the software clone job.

Once the job has completed successfully, we can see that the status of the component 11.2.0.3 PSU Jan 2013 has changed to Ready.

Let us now deploy the software clone on a target server.

From the Enterprise menu, select Provisioning and Patching, then Database Provisioning.

Highlight the Procedure Name Provision Oracle Database and click Launch.

Select the options as shown below. We are only deploying the Oracle Software and not creating a database and this is not a Grid Infrastructure environment.

Click on the Add button to select the host where we are going to deploy the software clone.

Let us briefly discuss the significance of the icon in the shape of a lock which can be seen in several of the deployment screens.

OEM 12c has introduced a new concept of Designer and Operator roles.

An OEM user with Designer role privileges can ensure standardization in deployment procedures by locking down deployment procedure inputs. Once these input values are locked in, when another user who has now Operator role privileges runs the same deployment procedure, they cannot edit the procedure and have to run the deployment with the values which were input and locked in by the OEM user who had Designer role privileges.

Select the credentials for the target host. The deployment procedure also executes the root.sh so we have also chosen a privileged user account with sudo privileges.

We see that the tasks Setup hosts is now having the status Completed.

We now need to complete the task Deploy Software. Click Next.

Select the deployment type as Component and we see the component which we earlier saved to the Software Library displaying.

Select the location for the Oracle Base and Oracle Home as well as a temporary working directory or software staging location. This location is cleaned up after the deployment job completes.

We now see that the Deploy Software task has the status Completed.

We can schedule the job to either run immediately or at a later time. Provide a name for the deployment job.

Review the job and click Submit button.

We can monitor the job while it is running and can see the different stages the clone software deployment job goes through.

In one of my earlier posts I had described the 12c agent silent method installation

This note explains another silent procedure used to install a 12c management agent but this time using a .rpm file.

Currently the Agent RPM is only available on Linux 32-bit and 64-bit platforms.

At a high level these are the steps involved:

• Download the .rpm file on the OMS host server using EMCLI
• Copy the .rpm file from the OMS host to the destination host where we want to install the agent.
• Install the .rpm file.

It should be noted that using the RPM method, the agent gets installed under a directory /usr/lib/oracle and this cannot be amended manually at this time.

It is documented in the MOS note:

EM 12c: Enterprise Manager 12.1.0.2 Cloud Control Agent Deployment with the RPM Method Does not Allow Passing a Custom Agent Base Directory Location [ID 1531693.1]

Installation Procedure

On the OMS host login to the EMCLI client.

After logging in, synchronize the EMCLI and check the platforms for which we currently have the agent software available in our Software Library.

[oracle@kens-oem-prod bin]$ ./emcli login -username=sysman
Enter password :

Login successful

[oracle@kens-oem-prod bin]$ ./emcli sync
Synchronized successfully

[oracle@kens-oem-prod bin]$ ./emcli get_supported_platforms
Getting list of platforms ...
Check the logs at /u01/app/Middleware/gc_inst/em/EMGC_OMS1/sysman/emcli/setup/.emcli/agent.log
About to access self-update code path to retrieve the platforms list..
Getting Platforms list  ...
-----------------------------------------------
Version = 12.1.0.2.0
 Platform = Linux x86-64
-----------------------------------------------
Version = 12.1.0.2.0
 Platform = Microsoft Windows x64 (64-bit)
-----------------------------------------------
Platforms list displayed successfully.

Download the .rpm file from the Software Library to a temporary location on the OMS host.

Note – we had to create the directory /usr/lib/oracle on the OMS host as well otherwise the get_agentimage_rpm command failed and this was the error which was logged:

“Directory /usr/lib/oracle doesnt exist. Please create the directory with write permissions and then retry the emcli command.”

[oracle@kens-oem-prod bin]$ ./emcli get_agentimage_rpm -destination=/u01/stage -platform="Linux x86-64" -version="12.1.0.2.0"
Platform:Linux x86-64
Destination:/u01/stage
Exalogic:false
 Checking for disk space requirements...
 === Partition Detail ===
Space free : 9 GB
Space required : 1 GB
RPM creation in progress ...
Check the logs at /u01/app/Middleware/gc_inst/em/EMGC_OMS1/sysman/emcli/setup/.emcli/get_agentimage_rpm_2013-06-19_09-53-32-AM.log
Copying agent image from software library to /u01/stage
Setting property ORACLE_HOME to:/u01/app/Middleware/oms
calling pulloneoffs with arguments:/u01/app/Middleware/oms/u01/app/Middleware/oms/sysman/agent/12.1.0.2.0_AgentCore_226.zip12.1.0.2.0Linux x86-64/u01/stagetrue
Agent Image copied successfully...
Creation of RPM started...
Agent image to rpm conversion failed

After creating the directory on the OMS host, we ran the command again and it still failed. This time with a different error message.

“RPM creation failed…
/u01/app/Middleware/oms/install/rpm/rpm.sh: line 61: rpmbuild: command not found”

So we need to install the rpmbuild package as it was not present on our OMS host.

We installed it using the yum command ‘yum install rpm-build‘

Confirmed the package was now present.

[root@kens-oem-prod bin]# which rpmbuild
/usr/bin/rpmbuild

Now the command ran fine and the .rpm file was successfully created.

[oracle@kens-oem-prod bin]$ ./emcli get_agentimage_rpm -destination=/u01/stage -platform="Linux x86-64" -version="12.1.0.2.0"
Platform:Linux x86-64
Destination:/u01/stage
Exalogic:false
 Checking for disk space requirements...
 === Partition Detail ===
Space free : 9 GB
Space required : 1 GB
RPM creation in progress ...
Check the logs at /u01/app/Middleware/gc_inst/em/EMGC_OMS1/sysman/emcli/setup/.emcli/get_agentimage_rpm_2013-06-19_10-08-55-AM.log
Copying agent image from software library to /u01/stage
Setting property ORACLE_HOME to:/u01/app/Middleware/oms
calling pulloneoffs with arguments:/u01/app/Middleware/oms/u01/app/Middleware/oms/sysman/agent/12.1.0.2.0_AgentCore_226.zip12.1.0.2.0Linux x86-64/u01/stagetrue
Agent Image copied successfully...
Creation of RPM started...
RPM creation successful.
Agent image to rpm conversion completed successfully

[oracle@kens-oem-prod bin]$ cd /u01/stage

[oracle@kens-oem-prod stage]$ ls -lrt
total 480384
-rw-r----- 1 oracle oinstall 255521471 May  8 11:34 12.1.0.2.0_AgentCore_233.zip
-rw-r----- 1 oracle oinstall 235885551 Jun 19 10:13 oracle-agt-12.1.0.2.0-1.0.x86_64.rpm

Next copy the .rpm file to the target server where we wish to install the 12c agent.

On the target host, as root, install the .rpm file. Create the directory /usr/lib/oracle as well and give permissions.

[root@kens-orawebl-001-dev ~]# mkdir /usr/lib/oracle
[root@kens-orawebl-001-dev ~]# chown oracle:dba /usr/lib/oracle

[root@kens-orawebl-001-dev stage]# rpm -ivh oracle-agt-12.1.0.2.0-1.0.x86_64.rpm
Preparing...                ########################################### [100%]
Running the prereq
   1:oracle-agt             ########################################### [100%]
Agent RPM installation is completed successfully. Now to configure the agent follow the below steps:
1. Edit the properties file: /usr/lib/oracle/agent/agent.properties with the correct values
2. Execute the script /etc/init.d/oracle-agt RESPONSE_FILE=/usr/lib/oracle/agent/agent.properties

Edit the agent.properties file and add the following values as shown in this example.

[root@kens-orawebl-001-dev stage]# vi /usr/lib/oracle/agent/agent.properties
#-------------------------------------------------------------------------------
#OMS_HOST: OMS host info required to connect to OMS
#OMS_PORT: OMS port info required to connect to OMS
#AGENT_REGISTRATION_PASSWORD: Agent Registration Password needed to
#     establish a secure connection to the OMS.
#-------------------------------------------------------------------------------
OMS_HOST="kens-oem-prod.domain"
OMS_PORT="4909"
AGENT_REGISTRATION_PASSWORD='password"
#-------------------------------------------------------------------------------
#AGENT_USERNAME: User name with which the agent should be installed.
#AGENT_GROUP: Group to which the agent user belogs.
#AGENT_PORT: Port in which the agent process will come up.
#-------------------------------------------------------------------------------
AGENT_USERNAME="oracle"
AGENT_GROUP="dba"
AGENT_PORT='3876"
#-------------------------------------------------------------------------------
#ORACLE_HOSTNAME: Virtual hostname where the agent is deployed.
#Example: ORACLE_HOSTNAME=hostname.domain
#-------------------------------------------------------------------------------
ORACLE_HOSTNAME="kens-orawebl-001-dev.corporateict.domain"

Run the following command (as root) to complete the agent installation

[root@kens-orawebl-001-dev oracle]# /etc/init.d/oracle-agt RESPONSE_FILE=/usr/lib/oracle/agent/agent.properties
Response File:/usr/lib/oracle/agent/agent.properties

The OEM 12c Cloud Control Compliance Management framework provides the ability to evaluate the compliance of targets as compared to industry-wide and business defined best practices and standards related to configuration, storage and security.

It enables us to automatically determine from a Risk and Security Management viewpoint as well as IT Auditing if any of our enterprise targets are exposed to any security vulnerabilities as well as if any any industry standards or regulations (Payment Card Industry PCI for example) are being violated.

The Compliance Library contains the Compliance Frameworks which are comprised of Compliance Standards which in turn a collection of one or more Compliance Standard Rules.

Oracle provides a number of such frameworks, standards and rules out of the box with OEM 12c, but we can also create a user-defined standard or rule to satisfy the requirements of our organization as we will see in this example described below.

OEM 12c comes bundled with a number of in-built Compliance Standards which are each made up of a number of rules which evaluates if a target is meeting or violating these compliance standards which are based on industry and business accepted best practices related to different IT aspects like configuration, security, and storage.

For example there are a number of out-of-the-box standards related to storage as shown below.

Each of those standards is comprised of a number of rules.

Click on the Standard - Storage Best Practices for Oracle Database.

We see that this standard is made up of rules like no user should have default tablespace or temporary tablespace pointing to a SYSTEM tablespace or all tablespaces should be configured with ASSM or there should be at least 3 redo log groups and so on.

If any of the managed target databases to which this Compliance Standard has been applied violates any of these rules, then we can see that showing up and displayed on the Compliance Dashboard.

We now want to add another standard rule related to storage which ensures that all the datafiles of tablespaces belonging to production databases are configured with AUTOEXTEND turned on. This rule is not part of the out-of-the-box Storage Best Practices for Oracle Database standard rule.

On the Compliance Standard Rules tab, click on the Create button.

The rule type selected in Repository Rule which indicates that the rule is based on evaluating data collected and stored in the OMS repository.

We will create a rule called AUTOEXTEND Not Turned On. At this point in time the rule is still being defined and tested so we set the lifecycle state of the rule to development.After we promote a rule to production, we cannot change it back to development.

Provide a description and rationale why the rule is being implemented and also we can provide a reference link to the official Oracle documentation which provided more information on the AUTOEXTEND functionality and its usage.

We need to provide a SQL query that will execute against the Cloud Control Management Repository along with a message which will be displayed either when the rule is complied with or violated.

From a SQL*PLUS session connected as SYSMAN, we create a custom view and grant appropriate privileges on the view to the different EM users as shown below.

SQL> create view MGMT$CS_DB_DATAFILES
  2  as
  3  select a.TABLESPACE_NAME, a.FILE_NAME,a.AUTOEXTENSIBLE,b.target_guid
  4  from MGMT_DB_DATAFILES_ECM a, MGMT$ECM_CURRENT_SNAPSHOTS b
  5  where a.ecm_snapshot_id =b.ecm_snapshot_id;

View created.

SQL> desc MGMT$CS_DB_DATAFILES
 Name                                      Null?    Type
 ----------------------------------------- -------- ----------------------------
 TABLESPACE_NAME                           NOT NULL VARCHAR2(30)
 FILE_NAME                                 NOT NULL VARCHAR2(512)
 AUTOEXTENSIBLE                                     VARCHAR2(3)
 TARGET_GUID                                        RAW(16)

SQL> grant select on MGMT$CS_DB_DATAFILES to SYSMAN_RO;

Grant succeeded.

SQL> grant select on MGMT$CS_DB_DATAFILES to MGMT_VIEW;

Grant succeeded.

SQL> grant select on MGMT$CS_DB_DATAFILES to MGMT_USER;

Grant succeeded.

SQL> create synonym mgmt_view.MGMT$CS_DB_DATAFILES for sysman.MGMT$CS_DB_DATAFILES;

Synonym created.

SQL> create synonym  SYSMAN_RO.MGMT$CS_DB_DATAFILES for sysman.MGMT$CS_DB_DATAFILES;

Synonym created.

We can see the columns which are returned by our SQL query and we can select the columns which will be displayed in alert messages when violations occur.

We also can set the condition we are checking against the returned query results to look for a violation.

In this example the rule checks if any rows are returned by the SQL query. If any rows are returned, it means then that the rule has been violated.

We can test our new rule by selecting a target and we see that for this particular database there are two datafiles which do not have autoextend turned on.

After the rule has been tested, we review the rule

The AUTOEXTEND Not Turned On rule has now been added to the Compliance Library.

The existing compliance standards cannot be edited to add additional rules so we will create a new custom compliance standard based on an existing standard and add a new custom rule to that custom standard.

Click on the Create Like buttton.

Provide a name for our custom standard and click Continue

This custom standard is applicable to all targets of the type Database Instance and the standard type is Repository. This means that the standard uses rules which are This means that the standard uses rules which are based on evaluating data collected and stored in the OMS repository.

Right-click the Standard we just added and select Add Rules

Search for the AUTOEXTEND rule we newly created

The rule is now added to the custom Compliance Standard

We now have to associate a target to which the compliance standard will apply. From the Compliance Standards tab, click on Associate Targets

Click on Enable and we should see the Evaluation Status for the chosen target display Enabled.

We see this message now displayed.

Now that the rule has been applied to a target (and we know that the particular target does have 2 tablespaces with datafiles not having autoextend enabled), we can view the Compliance Dashboard via the Enterprise – Compliance – Dashboard menu.

The Compliance Standard Custom Storage Best Practices for Oracle Databases displays 2 violations.

Unlike previous releases OEM 12c can now detect and discover targets even before the management agents are deployed on managed hosts.

This agentless technology is based on using nmap which is an IP scanning utility.

Once targets are discovered, we can use the promotion process to convert these unmanaged hosts into managed hosts by deploying the management agents on these hosts.

We can schedule regular jobs using an existing management agent to continually perform scans so that when new Oracle components are added to our infrastructure they are automatically discovered and brought under OEM12c management.

Since the entire network will be scanned, the Sudo Privilege Delegation must be set on the Management Agent host that will perform the scan.

To set up  Privilege Delegation, we need to add the following lines to the /etc/sudoers file as shown below.

oracle ALL=(root) /u01/app/oracle/agent12c/sbin/nmosudo *

Note – in versions prior to Enterprise Manager Cloud Control 12c Release 2 (12.1.0.2), nmosudo was located in the agent instance directory and not in the sbin directory. For example, /u01/oracle/agent/agent_inst/bin/nmosudo

Fron the Setup - Security – Privilege Delegation menu

Add the line in the Sudo Command field (location of sudo executable)(

/usr/bin/sudo -u %RUNAS% %COMMAND%

Click on Update

Click on preferred credentials

Select the host and then click on Set icon

Click on the Test icon.

Select Custom in the Test Type list of values

In the Command  enter ‘id’

We can see that the id command has been executed not by the oracle user but by the root user – so our Privelege Delegation setting is configured now in OEM 12c

From the Setup – Add Target – Configure Auto Discovery menu

Click on Host and Oracle VM Manager using IP Scan

Click on Create and the click on Add

Here we will add the host and management agent which is going to perform the nmap scan for us – note that this is the target where we have configured Privilege Delegation in a previous step

We can provide a name for the IP scan job and enter either the IP address or range of IP addresses to scan and we can also enter just a hostname or group of hostnames to scan.

For the given host or IP address or range of IP addresses we can specify which ports we want to scan.

In this case for example we have added the listener port 1523 to the list of default ports for scanning which are supplied out of the box.

Once the IP scanning job has been completed, we can check the status from the Setup – Add Target – Auto Discovery Results

We can see that on the provided IP address to scan  a host running on the Linux platform has been discovered. We can then click on the Promote button which will bring us to the Add Host Targets wizard from where we can  automatically deploy the 12c management agent.

In this example we will see how to mask sensitive data in a table using the Data Masking Pack which is included (as a separate licensed option) in Oracle 12c Cloud Control.

We create an Application Data Model first where we define which columns are considered sensitive and are candidates for data masking and then we create data masking policies or rules which instructs Oracle how to mask or scrub the data .

We can also use masking formats which are already supplied and ready to use out-of-the-box or we can create our own masking formats which can be then stored in a masking format library for future use.

Let us take the EMP table as an example.

We have cloned the table from the production database and in our test or development database we want to mask or hide any data which we consider to be confidential or sensitive from the development team or the user testing team for example.

Our data masking requirements are this:

1)      Shuffle data in the EMP table and group it on the JOB column. So when someone selects a record for a particular employee belonging to the job category say SALESMAN, the data is masked and rows belonging to some other random employee but belonging to the same job category SALESMAN is returned instead

2)      Hide the day and month the employee joined the company but retain the year value as the application requires the original year value and not some fictitious value

3)      The salary for the job category PRESIDENT should not be revealed

Note that data masking will replace data unlike the Data Redaction feature in the 12c database where the data which is displayed or returned by a query is changed on the fly.

So we create for this exercise a table called EMP_MASK which is a copy of the EMP table owned by SCOTT.

This is the data in the table before the data masking:

SQL> select * from emp_mask;

     EMPNO ENAME      JOB              MGR HIREDATE         SAL       COMM     DEPTNO
---------- ---------- --------- ---------- --------- ---------- ---------- ----------
      7369 SMITH      CLERK           7902 17-DEC-80        800                    20
      7499 ALLEN      SALESMAN        7698 20-FEB-81       1600        300         30
      7521 WARD       SALESMAN        7698 22-FEB-81       1250        500         30
      7566 JONES      MANAGER         7839 02-APR-81       2975                    20
      7654 MARTIN     SALESMAN        7698 28-SEP-81       1250       1400         30
      7698 BLAKE      MANAGER         7839 01-MAY-81       2850                    30
      7782 CLARK      MANAGER         7839 09-JUN-81       2450                    10
      7788 SCOTT      ANALYST         7566 19-APR-87       3000                    20
      7839 KING       PRESIDENT            17-NOV-81       5000                    10
      7844 TURNER     SALESMAN        7698 08-SEP-81       1500          0         30
      7876 ADAMS      CLERK           7788 23-MAY-87       1100                    20
      7900 JAMES      CLERK           7698 03-DEC-81        950                    30
      7902 FORD       ANALYST         7566 03-DEC-81       3000                    20
      7934 MILLER     CLERK           7782 23-JAN-82       1300                    10

14 rows selected.

After the data masking job has been run, we can see that the table data has changed according to the data masking policies which we had defined.

SQL> select * from emp_mask;

     EMPNO ENAME      JOB              MGR HIREDATE         SAL       COMM     DEPTNO
---------- ---------- --------- ---------- --------- ---------- ---------- ----------
      7844 WARD       SALESMAN        7698 02-AUG-81       1250        500         30
      7369 MILLER     CLERK           7782 29-MAY-82       1300                    10
      7934 JAMES      CLERK           7698 27-JAN-81        950                    30
      7788 FORD       ANALYST         7566 18-DEC-81       3000                    20
      7521 ALLEN      SALESMAN        7698 01-APR-81       1600        300         30
      7654 TURNER     SALESMAN        7698 25-NOV-81       1500          0         30
      7839 KING       PRESIDENT            10-MAY-81                               10
      7698 BLAKE      MANAGER         7839 02-AUG-81       2850                    30
      7499 MARTIN     SALESMAN        7698 29-MAY-81       1250       1400         30
      7902 SCOTT      ANALYST         7566 27-JAN-87       3000                    20
      7876 SMITH      CLERK           7902 01-AUG-80        800                    20
      7566 JONES      MANAGER         7839 29-MAY-81       2975                    20
      7782 CLARK      MANAGER         7839 27-JAN-81       2450                    10
      7900 ADAMS      CLERK           7788 02-AUG-87       1100                    20

14 rows selected.

The SAL column for KING who is the PRESIDENT has a null value.

The day and month for the HIREDATE column has been changed to a random value while retaining the year.

In the pre-masked table, EMPNO 7844 had these values:

     EMPNO ENAME      JOB              MGR HIREDATE         SAL       COMM     DEPTNO
---------- ---------- --------- ---------- --------- ---------- ---------- ----------
      7844 TURNER     SALESMAN        7698 08-SEP-81       1500          0         30

In the post-masked table we see that the data for the row with 7844 EMPNO has been shuffled with the original row which had the EMPNO 7521 as both these rows belonged to the job category SALESMAN

     EMPNO ENAME      JOB              MGR HIREDATE         SAL       COMM     DEPTNO
---------- ---------- --------- ---------- --------- ---------- ---------- ----------
      7844 WARD       SALESMAN        7698 02-AUG-81       1250        500         30

Note:

The following permissions are required for Data Masking.

• EM_ALL_OPERATOR for Enterprise Manager Cloud Control users
• SELECT_CATALOG_ROLE for database users
• SELECT ANY DICTIONARY privilege for database users
• EXECUTE privileges for the DBMS_CRYPTO package

Let us take a look at the steps involved.

Download the note on 12c Cloud Control Data Masking …

Let us have a look at some of the security features in 12c Cloud Control and we will look at Roles and Dynamic Groups.

Let us say for example we have a team of DBA’s supporting both MS SQL Server as well as Oracle databases and the 12c agents have now been deployed to all the Oracle as well as SQL Server hosts.

The requirement is that when the SQL Server DBA’s connect via 12c Cloud Control they should only see the SQL Server target hosts and databases and when the Oracle DBA’s connect likewise they see all the target servers where the Oracle databases are hosted.

We first create a Dynamic Group. From the Setup >> Add Target >> Dynamic Group menu

Enter the group name MSSQL_DBA_GROUP and click the Privilege Propagation box

Click on the Define Membership Criteria button

In the Target Types field select Microsoft SQL Server from the list of target values

Create a similar group called ORA_DBA_GROUP

We can see that depending on the targets we have already discovered on the hosts where the agents have been deployed the Oracle and SQL Server database target members are automatically added to their respective groups.

Next we create a couple of roles – MSSQL_DBA_ROLE and ORA_DBA_ROLE.

From the Setup >> Security >> Roles menu click on Create button.

Create a role called MSSQL_DBA

Click on Next and in the Target Privileges section click on Add
In the Target Type select Group and select the MSSQL_DBA_GROUP

Click on the pencil icon in the Manage Target Privilege Grants and change that from View to Full

Click on Review and then Finish

Similarly create another role called ORA_DBA_ROLE and ensure that we select the ORA_DBA_GROUP this time.

Grant the roles we have created to the administators based on the type of databases they support and wish to view as targets in Cloud Control.

From the Setup >> Security >> Administrators menu select the administrator account and click on Edit.

Click Next and in the Roles screen select the appropriate role which we had created earler

If connect as this admin user, we only see the Oracle database targets displayed.

But if we connect as the Sysman user, we can see all the targets – both Oracle as well as SQL Server

Note that in Oracle 12c Cloud Control, we cannot download the agent software from the online Oracle download software web site – we have to provision the same via the 12c OMS framework.

In this example we will download the 12c agent software for Windows 32-bit – our OMS is hosted on a Linux platform.

From the Setup >> Extensibility menu select Self Update

We see that we are not connected online to MOS and are using the offline method for the self update.

Note the software library in OEM has not been refreshed recently.

Click on Check Updates

On OMS server – execute the following commands

[oracle@vmnapp01 bin]$ ./emcli login -username="sysman" -password="xxx"
Login successful

[oracle@vmnapp01 bin]$ ./emcli import_update_catalog -file=/app/oracle/Middleware/oms/p9348486_112000_Generic.zip –omslocal

Processing catalog for Middleware Profiles and Gold Images
Processing update: Middleware Profiles and Gold Images - Three Fusion Middleware Provisioning Profiles with different heap size configuration
Processing catalog for Agent Software
Processing update: Agent Software - Agent Software (12.1.0.4.0) for Microsoft Windows (32-bit)
Processing update: Agent Software - Agent Software (12.1.0.3.0) for Microsoft Windows (32-bit)
Processing update: Agent Software - Agent Software (12.1.0.2.0) for Microsoft Windows (32-bit)
Processing update: Agent Software - Agent Software (12.1.0.1.0) for Microsoft Windows (32-bit)
Processing update: Agent Software - Agent Software (12.1.0.4.0) for HP-UX PA-RISC (64-bit)

….
……

Successfully uploaded the Self Update catalog to Enterprise Manager. Use the Self Update Console to view and manage updates.

We now see that the refresh has happened successfully and the refresh time has been updated

Select the Microsoft Windows (32-bit) row and click on Download

Enter URL in web browser and download the file

https://updates.oracle.com/Orion/Services/download/p18797166_112000_Generic.zip?aru=17700668&patch_file=p18797166_112000_Generic.zip

Copy the downloaded file to the OMS host

On OMS server run

[oracle@csmsdc-vmnapp01 bin]$ ./emcli import_update -omslocal -file=/var/tmp/p18797166_112000_Generic.zip
Processing update: Agent Software - Agent Software (12.1.0.4.0) for Microsoft Windows (32-bit)

Successfully uploaded the update to Enterprise Manager. Use the Self Update Console to manage this update.

We see that the status has changed from Available to Downloaded

Highlight the Microsoft Windows (32-bit) row and click Apply

Note status has now changed to Applied

We now see that Windows (32-bit) now shows up in the list of supported agent12c downloadable platforms

[oracle@vmnapp01 bin]$ ./emcli get_supported_platforms
-----------------------------------------------
Version = 12.1.0.4.0
Platform = Microsoft Windows (32-bit)
-----------------------------------------------
Version = 12.1.0.4.0
Platform = Linux x86-64
-----------------------------------------------
Version = 12.1.0.1.0
Platform = Microsoft Windows (32-bit)
-----------------------------------------------
Platforms list displayed successfully.

Download from OMS to directory on the Linux server. From here we can copy it to the Windows 32-bir machines where we wish to deploy the agent.

[oracle@vmnapp01 bin]$ which zip
/usr/bin/zip
[oracle@vmnapp01 bin]$ export ZIP_LOC=/usr/bin/zip



[oracle@vmnapp01 bin]$ ./emcli get_agentimage -destination=/app/oracle/Middleware/oms/agent_software -platform="Microsoft Windows (32-bit)" -version="12.1.0.4.0"

=== Partition Detail ===
Space free : 206 GB
Space required : 1 GB
Check the logs at /app/oracle/Middleware/gc_inst/em/EMGC_OMS1/sysman/emcli/setup/.emcli/get_agentimage_2014-08-27_15-16-20-PM.log
Downloading /app/oracle/Middleware/oms/agent_software/12.1.0.4.0_AgentCore_912.zip
File saved as /app/oracle/Middleware/oms/agent_software/12.1.0.4.0_AgentCore_912.zip
Downloading /app/oracle/Middleware/oms/agent_software/12.1.0.4.0_PluginsOneoffs_912.zip
File saved as /app/oracle/Middleware/oms/agent_software/12.1.0.4.0_PluginsOneoffs_912.zip
Downloading /app/oracle/Middleware/oms/agent_software/unzip
File saved as /app/oracle/Middleware/oms/agent_software/unzip
Agent Image Download completed successfully.

So what are Metric Extensions?

As the name implies metric extensions enables us to extend the out-of-the-box monitoring capabilities of Enterprise Manager and customize it according to the requirements of our specific environment.

Very often we may have some monitoring and alerting requirements for which metrics are not available in OEM and we are having  to run external scripts outside OEM.

For example we would like to be alerted and notified via OEM in the following circumstances:

• Standby database lags behind the Primary database by more than 5 archive log files
• There are failed DBMS jobs
• In case some key program units become INVALID
• In case a very important index becomes unusable or is dropped ….. and so on.

We can achieve all of the above by creating metric extensions in EM 12c.

The  EM 12c Metric Extensions note shows us how to create a metric extension to raise an incident and be notified if one of the critical DBMS_SCHEDULER_JOBS fails for any reason. The metric extension checks for the job status of ‘BROKEN’ to raise a critical event which will then send an email notification to the administrator.

We first create a metric extension as a deployable draft, test it, deploy it to required targets and also publish it. Metric extensions can also be part of a monitoring template.

The note shows the various screen shots of creating and deploying a metric extension in EM 12c.

Note – we can set up email notification for failed DBMS Scheduler Jobs outside of OEM as well using the DBMS_SCHEDULER package.

BEGIN
  DBMS_SCHEDULER.set_scheduler_attribute('email_server', 'mail.soorma.com:25');
  DBMS_SCHEDULER.set_scheduler_attribute('email_sender', 'gavin.soorma@soorma.com');
END;
/

BEGIN
DBMS_SCHEDULER.ADD_JOB_EMAIL_NOTIFICATION (
job_name => ‘TEST_JOB’,
recipients => ‘gavin.soorma@soorma.com’,
sender => ‘gavin.soorma@soorma.com’,
subject => ‘Scheduler Job Notification-%job_owner%.%job_name%-%event_type%’,
body => ‘%event_type% occurred at %event_timestamp%. %error_message%’,
events => ‘JOB_FAILED, JOB_BROKEN’);
END;
/

This is an example of the email alert we will be sent:

JOB_FAILED occurred at 05-JAN-15 01.32.22.808171 PM +08:00. ORA-01653: unable to extend table SYSTEM.MYOBJECTS by 128 in tablespace GAVIN
ORA-06512: at line 3

If we have a large number of databases to upgrade, we can use the Database Provisioning feature of 12c Cloud Control to automate and streamline the process and take away much of the manual tasks from the DBA.

Not is this time saving but also reduces the risk of failure in the upgrade process due to human errors.

The note provides an example of how two Oracle 11g databases residing on separate Linux servers are automatically upgraded to Oracle 12c via 12c Cloud Control.

Read the note on upgrading databases using 12c Cloud Control ….

One of the challenges faced today in the database provisioning process is time to deliver.

Consider a normal workflow in this case.

A developer requests a database. It then goes to his manager for approval.Once that stage is passed the DBA comes into the picture and the DBA in turn will contact the Storage and System Admin to request the hardware and storage. There could be OS and Network setup involved as well and then finally the DBA will create the database and then provide access to the developer.

Typically this can take days and sometimes weeks as well – using DBaaS with EM12c we reduce the time to deliver to minutes!

Not only is this time consuming and inefficient but this very process has caused the problems we face today around database and server sprawl, multiple versions and patch levels, high cost of deployment and operation, poor resource utilization – the current state of most database deployments is largely siloed and complex.

In this example we see a use case of DBaaS where a Developer is able to create an Oracle 12c pluggable database without any intervention of the DBA by submitting a Service Request using a Service Catalog- but the DBA still controls what the developers can and cannot do.

We will introduce DBaaS concepts like Platform as a Service (PaaS) Infrastructure Zones, Database Resource Pools, Service Templates, Quotas, Service Catalogs and finally an example of Chargeback and Metering as well.

We have created two users for the example – CLOUD_ADMIN and DEV_SSA and a CLOUD_SSA Role as well.

Connect as the CLOUD_ADMIN user
 
From the Setup -Cloud -PaaS Infrastructure Zones menu
 

 
A PaaS Infrastructure Zone is a collection of Host targets or an OVM zone. Hosts are targets identified in Enterprise Manager and a single host can be a member of only one PaaS Zone.

We also associate the SSA role we created earlier so as to regulate access to the PaaS Infrastructure Zone.

We also have to provide the host credentials of the members the PaaS Zone is comprised of.

Placement policy constraints can be used to specify a ceiling or limit on the amount of resources any host in the zone can use

 
The next step is to create a Database Resource Pool – this will be based on databases and Oracle Homes existing on the hosts that make up the PaaS Infrastructure Zone.

A Database Pool is a collection of resources (homogenous databases and Oracle Homes) to provision a database instance within a PaaS Infrastructure Zone – so basically the same platform, type and version.

We can create a Database Pool for Database as a Service, Schema as a Service or PDB as a service.

Also at the Database Resource Pool level we can specify some placement constraints to control resource utilization.

Through the use of Quotas we can also control and limit resource utilization at the SSA user or role level.

From the Setup – Cloud – Databases menu
 

The services that are offered to SSA Cloud users are defined via Service Templates. A Service Template can be also based on a database provisioning Profile.

In this example we are creating a service template for provisioning an EMPTY PLUGGABLE DATABASE.

The PaaS Infrastructure Zone earlier created is associated with the Service Template and we also assign the Database Resource Pool as well. In this way we are regulating which hosts or OVM zones can be used by a particular SSA user as well as the type of database that can be created via the Self Service Portal.

The Service Template can also define other things like the number of tablespaces which will be created in the pluggable database, the total size of the PDB, init.ora parameters and other resource limits like number of database sessions etc.

Next connect as the DEV_SSA Cloud SSA user

From the Cloud menu select Self Service Portal

The DEV_SSA user will request the creation of a 12c Pluggable Database from the Service Catalog available via the Self Service Portal.

Note that the DEV_SSA user has been provided in this example a limit of just one single service request and a quota of 1 GB of RAM and 1 GB of disk space and this quota is exhausted when he submits the service request for the creation of the pluggable database.

After submitting the service request the user can track the progress of the request.

After the request completes a Pluggable Database has been created and is now available for use by the developer.

So in this case a developer has created a 12c Pluggable Database with no involvement of the DBA or the System Administrator or Storage Administrator in a few minutes. The DBA or Cloud Administrator still controls which physical host the database is created on ,which Container Database the pluggable database is a tenant of, how much resources can be allocated in terms of disk space, RAM and CPU.

This is EM12c enabling Database as a Service!

Let us take a brief look at how Chargeback and Metering work in a DBaaS model.

Simply put,based on resource usage charges are calculated and then allocated or metered to individual cost centers who are consumers of the Database as a Service offering.

In this example we create a charge plan for the Pluggable Database entity and the metric being used to calculate the charges is Uptime of the database.

The user is being charged a rate of 10$ per hour of database uptime.
 

 
 
The SSA user can also view a number of reports of the Metering and Charges and there is an Enterprise Manager job which runs once in a 24 hour period collecting such data which is displayed in the reports.

In this case we are forcing the manual collection of data and once the job completes we can see a number of graphical reports on Chargeback data showing the usage or consumption and amount which is payable for usage of the service and the same is displayed on a daily basis as well in case the user wishes to get a day by day breakdown of the Chargeback.
 

Let us look at a common requirement the DBA faces on a regular basis related to performing a database clone.

A developer needs a copy or clone of the production database to test some urgent fixes and requires this database for a short period of time to perform the testing – after which this database can be dropped. There may be further requirement to blow the database away and have another copy of production data in case another round of further testing is required.

We all know how long in a normal case it will take to fulfill this requirement – days and maybe even weeks – and the amount of work involved to provision this database.

Now with Database as a Service (DBaaS) being offered via EM12c, the developer can get his clone of the production database created when he wants with a click of a button using the Cloud Self Service Portal. The DBA does not need to go and perform a backup of the database or be involved in any way.

But at the same time the DBA has total control over the service which is offered to the developer – which host the clone will get created on, the size of the database, disk space it can occupy and even init.ora parameters which will be used to create the cloned database among other things.

In this example we have a source 12.1.0.2.’production’ database – db12c – and we will be creating a provisioning profile to clone this database on a target development host. After a week of use the database will be automatically dropped and the developer has the option of requesting creation of another clone in case more testing is required.

We will base the clone on an existing RMAN backup – note that we need to copy the RMAN backupset to the target server where the clone is being created.

In this case we have a Platform as a Service (PaaS) Infrastructure Zone called development_zone and that zone has two member hosts.Let us say one of the hosts is the production host where the source database is located and the other host is the development or test host where the clone will be created.

To see the details of the PaaS Infrastructure Zone highlight the zone and click on the Edit button.

We now create a Database Pool in the PaaS zone – the Database Resource Pool will be a non-CDB based pool.

We enter the details of the PaaS Infrastructure Zone and the OS platform and based on the database version (in this case 12.1.0.2) the available Oracle Homes located on both hosts which make up the PaaS zone are displayed.

In this case the database will be made available only for a period of 7 days so we specify the retention duration under the Request Settings.

In the Quotas section we limit the number of database requests the SSA user can make as well as the total amount of machine RAM and disk space which will be made available to the user. This is being regulated via the CLOUD_SSA role.

Next is that we create a Database Provisioning Profile. In the Profiles section click on Create.

Select the reference or source database on which the profile will be based.

A clone of the reference database (db12c) will be created using an existing RMAN backup.

After the profile is created we now have to create a Service Template based on that profile.

Note that the Shared Location on the Reference Host (which is the host where the clone is being created) is the location where we have copied the RMAN backup sets. This is the RMAN backup of the source database on which the clone is going to be based.

The location of the data files can be defined as well as the prefix which will be used for the database name. Other things like listener port and passwords can also be defined.

Here we can change some of the init.ora parameters if required – for example we may want to ensure that the clone database is allocated a smaller SGA or PGA than the source production database or change some other parameter related to performance or availability.

 
Let us now connect as the Cloud Self Service user.
 
The user requests the creation of a database and selects the Service Template which is available to him.

After submitting the request the user can monitor the progress of the request

After the database has been created we can see the instance dev00000 up and running and can also see the amount of resources like RAM and Disk Space the user has used so far and how much is still available for future requests.

The Patch Deployment feature in EM12c can greatly help in automating the rolling out of patches when we have to deploy the patch on a large number of targets – this significantly reduces both the time and complexity involved in the process.

Let us look at an example of deploying the JAN 2015 PSU patch using EM12c.

We first need to upload the JAN 2015 PSU patch 19769480 as well as the opatch version 12.1.0.1.6 to the EM12c Software Library – as we are operating in Offline Patching mode.

Note that we have to upload the Patch Metadata file as well along with the patch

 
Click on 19769480 link in the Patch Name column
 

We will add the patch to a New Patch Plan
 

 
We can deploy the PSU patch on all available hosts with 12.1.0.2 Oracle databases – in this example we are deploying the PSU to just a single host.
 

 

 
After the patch plan has been created we edit the patch plan via the Patches & Updates menu
 

 

 
The Patch deployment can be In Place or Out of Place. In this case we will be applying the PSU patch to the existing Oracle Home. Provide the required Normal and Privileged Credentials and validate the same.
 

 

 
We then need to run an Analyze of the PSU patch. The patch is staged and checks are performed to determine if all the patch prerequisites are met.
 

 

 

 

 

 
After the patch has been successfully analyzed, we can see that it is now ready for deployment.
 

 
Review the patch plan and then click on the Deploy button
 

 
Since the PSU patch deployment will require a database outage we can schedule the patch to be deployed at a specfic time or it can be deployed to start immediately.
 

 

 
While the patch deployment is in progress we can view the different actions being performed at each step and have very good visibility of the patch application as it proceeds.
 

 

 

 

 

 
After the PSU patch application we can now see that the Post SQL script is being applied while the database has been started in Upgrade Mode.
 

 
A Blackout is created automatically as patch of the patch deployment and Blackout is cleared as one of the last steps in the patch deployment plan.
 

 
If we select the relevant 12.1.0.2 Oracle Home in the Targets menu, we can see under the Patches Applied tab that patch 19769480 has been successfully applied and we also can see the various bugs which have been fixed by this PSU patch.