Steps to Apply Combo Patch (Oct 2022) on Clusterware in Two Node RAC in Oracle

December 4, 2022

Steps to Apply Combo Patch (Oct 2022) on Clusterware in Two Node RAC in Oracle

In this article, we are going to demonstrate the Steps to Apply combo patch 34449117 (19.17.0.0.221018) of October 2022 on Oracle Clusterware in Two Node RAC version 19c. Here we will apply a combo patch 34449117 that contains two patches 34416665 (GRID) and 34411846 (OJVM). Below is the current version of the Clusterware/database, from both NODE/s.

[grid@dbsgururc01 ~]$ . oraenv
ORACLE_SID = [grid] ? +ASM1
The Oracle base has been set to /u01/app/grid1
[grid@dbsgururc01 ~]$ sqlplus -v

SQL*Plus: Release 19.0.0.0.0 - Production
Version 19.16.0.0.0

[grid@dbsgururc01 ~]$ 
[oracle@dbsgururc01 ~]$ . oraenv
ORACLE_SID = [oracle] ? dbsgucdb1
The Oracle base has been set to /u02/app/oracle
[oracle@dbsgururc01 ~]$ sqlplus -v

SQL*Plus: Release 19.0.0.0.0 - Production
Version 19.16.0.0.0

[oracle@dbsgururc01 ~]$
[grid@dbsgururc02 ~]$ . oraenv
ORACLE_SID = [grid] ? +ASM2
The Oracle base has been set to /u01/app/grid1
[grid@dbsgururc02 ~]$ sqlplus -v

SQL*Plus: Release 19.0.0.0.0 - Production
Version 19.16.0.0.0

[grid@dbsgururc02 ~]$
[oracle@dbsgururc02 ~]$ . oraenv
ORACLE_SID = [oracle] ? dbsgucdb2
The Oracle base has been set to /u02/app/oracle
[oracle@dbsgururc02 ~]$ sqlplus -v

SQL*Plus: Release 19.0.0.0.0 - Production
Version 19.16.0.0.0

[oracle@dbsgururc02 ~]$

Follow the high-level plan to apply patch 34449117 (19.17.0.0.221018).

Go through the read-me document/s of patch IDs before proceeding with the patch apply.
Download the required patche/s.
Perform prechecks (Cluster/DB level) like patch conflict, Space, running instances along with listener, etc.)
Validate the version of the current OPATCH utility and upgrade it if required.
Take the backup of GRID/ORACLE HOME along with DBs backup of the associated home.
Blackout in OEM, disable jobs in crontab & DBMS scheduler jobs, stop applications, etc. (if any applicable for all databases of same home).
Stop all running jobs of ORACLE HOME.
Apply GRID patch, node-wise.
Validate applied patch in inventory.
Stop all instances/databases associated with ORACLE HOME.
Apply OJVM patch, node-wise.
Start all instances/databases associated with ORACLE HOME.
Execute post patch (datapatch) on each database and validate.
Start all services associated with ORACLE HOME.
Delete blackout in OEM, enable jobs in crontab & DBMS scheduler jobs, start applications, etc. (if any applicable for all databases of same home).
Handover to applications checkout.

Below are the details of the environment for patching which is used in this demonstration:

COMPONENTS	DETAILS
GRID Home	/u01app/grid/19c/gi_1
DB Home	/u02/app/oracle/product/19c/dbhome_1
DB Version	19.16.0.0.0
Opatch Version	12.2.0.1.32
DB Type	RAC, Multitanenat Architecure
DB Name/s, PDB/s	dbsgururc, DBS_PDB1
Patch Level	896235792, 19.16.0.0.0

1. Prerequisite

NODE1: dbsgururc01

1.1: Download the grid/database patch specific to the version and OS platform. Click here for Oracle Critical Patch IDs along with the enabled download link for October 2022.

[grid@dbsgururc01 ~]$ mkdir 19c_patch_Oct2022
[grid@dbsgururc01 ~]$ chmod 775 -R 19c_patch_Oct2022
[grid@dbsgururc01 ~]$ ls -ld 19c_patch_Oct2022
drwxrwxr-x. 2 grid oinstall 47 Nov 29 16:09 19c_patch_Oct2022
[grid@dbsgururc01 ~]$ cd 19c_patch_Oct2022
[grid@dbsgururc01 19c_patch_Oct2022]$ ls -lrt
total 2683964
-rwxrwxr-x. 1 grid oinstall 2748377244 Nov 29 16:12 p34449117_190000_Linux-x86-64.zip
[grid@dbsgururc01 19c_patch_Oct2022]$

1.2: After downloading the required patch, transfer/copy the downloaded patch to the desired directory and extract it. Here I am using Linux 64 Bit and the GRID version is 19c.

[grid@dbsgururc01 19c_patch_Oct2022]$ unzip -q p34449117_190000_Linux-x86-64.zip
[grid@dbsgururc01 19c_patch_Oct2022]$ ls -lrt
total 2685768
drwxr-xr-x. 4 grid oinstall         57 Oct 19 11:22 34449117
-rw-rw-r--. 1 grid oinstall    1846194 Oct 19 12:17 PatchSearch.xml
-rwxrwxr-x. 1 grid oinstall 2748377244 Nov 29 16:12 p34449117_190000_Linux-x86-64.zip
[grid@dbsgururc01 19c_patch_Oct2022]$ cd 34449117
[grid@dbsgururc01 34449117]$ ls -lrt
total 28
drwxr-xr-x. 4 grid oinstall    67 Sep 27 22:16 34411846   ==> OJVM
drwxr-x---. 8 grid oinstall  4096 Oct 17 12:25 34416665  ==> GRID
-rw-r--r--. 1 grid oinstall 22583 Oct 19 11:22 README.html
[grid@dbsgururc01 34449117]$

1.3: Capture information of running instances, listener, patch inventory, oratab, patch level, etc. of associated HOME/s.

grid@dbsgururc01 19c_patch_Oct2022]$ mkdir logs_patch_Oct2022
[grid@dbsgururc01 19c_patch_Oct2022]$ ls -ld logs_patch_Oct2022
drwxr-xr-x. 2 grid oinstall 6 Nov 29 16:38 logs_patch_Oct2022
[grid@dbsgururc01 19c_patch_Oct2022]$ cd logs_patch_Oct2022
[grid@dbsgururc01 logs_patch_Oct2022]$ pwd
/home/grid/19c_patch_Oct2022/logs_patch_Oct2022
[grid@dbsgururc01 logs_patch_Oct2022]$ . oraenv
ORACLE_SID = [grid] ? +ASM1
The Oracle base has been set to /u01/app/grid1

[grid@dbsgururc01 logs_patch_Oct2022]$ export PATH=$ORACLE_HOME/OPatch:$PATH
[grid@dbsgururc01 logs_patch_Oct2022]$ which opatch
/u01/app/grid/19c/gi_1/OPatch/opatch
[grid@dbsgururc01 logs_patch_Oct2022]$ ps -ef | grep pmon > current_instance_Oct22_19cpatch.txt
[grid@dbsgururc01 logs_patch_Oct2022]$ ps -ef | grep tns > current_tns_Oct22_19cpatch.txt
[grid@dbsgururc01 logs_patch_Oct2022]$ cat /etc/oratab > current_oratab_Oct22_19cpatch.txt
[grid@dbsgururc01 logs_patch_Oct2022]$ $ORACLE_HOME/OPatch/opatch lsinventory > before_current_lsinventory_Oct22_19cpatch.txt
[grid@dbsgururc01 logs_patch_Oct2022]$ opatch lspatches
34318175;TOMCAT RELEASE UPDATE 19.0.0.0.0 (34318175)
34160635;OCW RELEASE UPDATE 19.16.0.0.0 (34160635)
34139601;ACFS RELEASE UPDATE 19.16.0.0.0 (34139601)
34133642;Database Release Update : 19.16.0.0.220719 (34133642)
33575402;DBWLM RELEASE UPDATE 19.0.0.0.0 (33575402)

OPatch succeeded.
[grid@dbsgururc01 logs_patch_Oct2022]$ ls -lrt
total 120
-rw-r--r--. 1 grid oinstall    266 Nov 29 16:42 current_instance_Oct22_19cpatch.txt
-rw-r--r--. 1 grid oinstall    725 Nov 29 16:42 current_tns_Oct22_19cpatch.txt
-rw-r--r--. 1 grid oinstall   1012 Nov 29 16:42 current_oratab_Oct22_19cpatch.txt
-rw-r--r--. 1 grid oinstall 109341 Nov 29 16:43 before_current_lsinventory_Oct22_19cpatch.txt
[grid@dbsgururc01 logs_patch_Oct2022]$

[grid@dbsgururc01 logs_patch_Oct2022]$ crsctl query crs activeversion -f
Oracle Clusterware active version on the cluster is [19.0.0.0.0]. The cluster upgrade state is [UPGRADE FINAL]. The cluster active patch level is [896235792].
[grid@dbsgururc01 logs_patch_Oct2022]$ crsctl query crs softwareversion
Oracle Clusterware version on node [dbsgururc01] is [19.0.0.0.0]
[grid@dbsgururc01 logs_patch_Oct2022]$ crsctl query crs releasepatch
Oracle Clusterware release patch level is [896235792] and the complete list of patches [33575402 34133642 34139601 34160635 34318175 ] have been applied on the local node. The release patch string is [19.16.0.0.0].
[grid@dbsgururc01 logs_patch_Oct2022]$ crsctl query crs softwarepatch
Oracle Clusterware patch level on node dbsgururc01 is [896235792].
[grid@dbsgururc01 logs_patch_Oct2022]$ asmcmd -V
asmcmd version 19.16.0.0.0
[grid@dbsgururc01 logs_patch_Oct2022]$ crsctl stat res -t
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.LISTENER.lsnr
               ONLINE  ONLINE       dbsgururc01              STABLE
               ONLINE  ONLINE       dbsgururc02              STABLE
ora.chad
               ONLINE  ONLINE       dbsgururc01              STABLE
               ONLINE  ONLINE       dbsgururc02              STABLE
ora.net1.network
               ONLINE  ONLINE       dbsgururc01              STABLE
               ONLINE  ONLINE       dbsgururc02              STABLE
ora.ons
               ONLINE  ONLINE       dbsgururc01              STABLE
               ONLINE  ONLINE       dbsgururc02              STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.ASMNET1LSNR_ASM.lsnr(ora.asmgroup)
      1        ONLINE  ONLINE       dbsgururc01              STABLE
      2        ONLINE  ONLINE       dbsgururc02              STABLE
      3        ONLINE  OFFLINE                               STABLE
ora.DATA1.dg(ora.asmgroup)
      1        ONLINE  ONLINE       dbsgururc01              STABLE
      2        ONLINE  ONLINE       dbsgururc02              STABLE
      3        OFFLINE OFFLINE                               STABLE
ora.FRA1.dg(ora.asmgroup)
      1        ONLINE  ONLINE       dbsgururc01              STABLE
      2        ONLINE  ONLINE       dbsgururc02              STABLE
      3        OFFLINE OFFLINE                               STABLE
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       dbsgururc02              STABLE
ora.LISTENER_SCAN2.lsnr
      1        ONLINE  ONLINE       dbsgururc01              STABLE
ora.LISTENER_SCAN3.lsnr
      1        ONLINE  ONLINE       dbsgururc01              STABLE
ora.MGMTLSNR
      1        ONLINE  ONLINE       dbsgururc01              169.254.2.113 192.16
                                                             8.10.100,STABLE
ora.VOTE1.dg(ora.asmgroup)
      1        ONLINE  ONLINE       dbsgururc01              STABLE
      2        ONLINE  ONLINE       dbsgururc02              STABLE
      3        OFFLINE OFFLINE                               STABLE
ora.asm(ora.asmgroup)
      1        ONLINE  ONLINE       dbsgururc01              Started,STABLE
      2        ONLINE  ONLINE       dbsgururc02              Started,STABLE
      3        ONLINE  OFFLINE                               STABLE
ora.asmnet1.asmnetwork(ora.asmgroup)
      1        ONLINE  ONLINE       dbsgururc01              STABLE
      2        ONLINE  ONLINE       dbsgururc02              STABLE
      3        ONLINE  OFFLINE                               STABLE
ora.cvu
      1        ONLINE  ONLINE       dbsgururc01              STABLE
ora.dbsgucdb.db
      1        ONLINE  ONLINE       dbsgururc01              Open,HOME=/u02/app/o
                                                             racle/product/19c/db
                                                             home_1,STABLE
      2        ONLINE  ONLINE       dbsgururc02              Open,HOME=/u02/app/o
                                                             racle/product/19c/db
                                                             home_1,STABLE
ora.dbsgururc01.vip
      1        ONLINE  ONLINE       dbsgururc01              STABLE
ora.dbsgururc02.vip
      1        ONLINE  ONLINE       dbsgururc02              STABLE
ora.mgmtdb
      1        ONLINE  ONLINE       dbsgururc01              Open,STABLE
ora.qosmserver
      1        ONLINE  ONLINE       dbsgururc01              STABLE
ora.scan1.vip
      1        ONLINE  ONLINE       dbsgururc02              STABLE
ora.scan2.vip
      1        ONLINE  ONLINE       dbsgururc01              STABLE
ora.scan3.vip
      1        ONLINE  ONLINE       dbsgururc01              STABLE
--------------------------------------------------------------------------------
[grid@dbsgururc01 logs_patch_Oct2022]$

1.4: Validate the version of the utility OPATCH and upgrade it if needed, It should meet the minimum version as per the readme doc of the specified patch ID. In this demonstration, it meets to require an opatch version. Click here for Step by Step OPATCH Upgrade in Oracle.

[grid@dbsgururc01 logs_patch_Oct2022]$ opatch version ==> As a GRID user
OPatch Version: 12.2.0.1.32

OPatch succeeded.
[grid@dbsgururc01 logs_patch_Oct2022]$
[oracle@dbsgururc01 ~]$ /u02/app/oracle/product/19c/dbhome_1/OPatch/opatch version ==> As an Oracle user
OPatch Version: 12.2.0.1.32

OPatch succeeded.
[oracle@dbsgururc01 ~]$

GRID Patch 34416665

1.5: Determine whether any currently installed interim patches conflict with the patch being installed along with the system space check.

1.5.1: Patch Conflict Check

As a GRID user:

[grid@dbsgururc01 logs_patch_Oct2022]$ $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /home/grid/19c_patch_Oct2022/34449117/34416665/34419443
Oracle Interim Patch Installer version 12.2.0.1.32
Copyright (c) 2022, Oracle Corporation. All rights reserved.

PREREQ session

Oracle Home : /u01/app/grid/19c/gi_1
Central Inventory : /u01/app/oraInventory
from : /u01/app/grid/19c/gi_1/oraInst.loc
OPatch version : 12.2.0.1.32
OUI version : 12.2.0.7.0
Log file location : /u01/app/grid/19c/gi_1/cfgtoollogs/opatch/opatch2022-11-29_17-04-54PM_1.log

Invoking prereq "checkconflictagainstohwithdetail"

Prereq "checkConflictAgainstOHWithDetail" passed.

OPatch succeeded.
[grid@dbsgururc01 logs_patch_Oct2022]$ $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /home/grid/19c_patch_Oct2022/34449117/34416665/34444834
Oracle Interim Patch Installer version 12.2.0.1.32
Copyright (c) 2022, Oracle Corporation. All rights reserved.

PREREQ session

Invoking prereq "checkconflictagainstohwithdetail"

Prereq "checkConflictAgainstOHWithDetail" passed.

OPatch succeeded.
[grid@dbsgururc01 logs_patch_Oct2022]$ $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /home/grid/19c_patch_Oct2022/34449117/34416665/34428761
Oracle Interim Patch Installer version 12.2.0.1.32
Copyright (c) 2022, Oracle Corporation. All rights reserved.

PREREQ session

Invoking prereq "checkconflictagainstohwithdetail"

Prereq "checkConflictAgainstOHWithDetail" passed.

OPatch succeeded.
[grid@dbsgururc01 logs_patch_Oct2022]$ $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /home/grid/19c_patch_Oct2022/34449117/34416665/34580338
Oracle Interim Patch Installer version 12.2.0.1.32
Copyright (c) 2022, Oracle Corporation. All rights reserved.

PREREQ session

Invoking prereq "checkconflictagainstohwithdetail"

Prereq "checkConflictAgainstOHWithDetail" passed.

OPatch succeeded.
[grid@dbsgururc01 logs_patch_Oct2022]$ $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /home/grid/19c_patch_Oct2022/34449117/34416665/33575402
Oracle Interim Patch Installer version 12.2.0.1.32
Copyright (c) 2022, Oracle Corporation. All rights reserved.

PREREQ session

Invoking prereq "checkconflictagainstohwithdetail"

Prereq "checkConflictAgainstOHWithDetail" passed.

OPatch succeeded.
[grid@dbsgururc01 logs_patch_Oct2022]$

As an ORACLE user:

[oracle@dbsgururc01 ~]$ . oraenv
ORACLE_SID = [oracle] ? dbsgucdb1
The Oracle base has been set to /u02/app/oracle
[oracle@dbsgururc01 ~]$ $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /home/grid/19c_patch_Oct2022/34449117/34416665/34419443
Oracle Interim Patch Installer version 12.2.0.1.32
Copyright (c) 2022, Oracle Corporation. All rights reserved.

PREREQ session

Oracle Home : /u02/app/oracle/product/19c/dbhome_1
Central Inventory : /u01/app/oraInventory
from : /u02/app/oracle/product/19c/dbhome_1/oraInst.loc
OPatch version : 12.2.0.1.32
OUI version : 12.2.0.7.0
Log file location : /u02/app/oracle/product/19c/dbhome_1/cfgtoollogs/opatch/opatch2022-11-29_17-14-54PM_1.log

Invoking prereq "checkconflictagainstohwithdetail"

Prereq "checkConflictAgainstOHWithDetail" passed.

OPatch succeeded.
[oracle@dbsgururc01 ~]$ $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /home/grid/19c_patch_Oct2022/34449117/34416665/34444834
Oracle Interim Patch Installer version 12.2.0.1.32
Copyright (c) 2022, Oracle Corporation. All rights reserved.

PREREQ session

Invoking prereq "checkconflictagainstohwithdetail"

Prereq "checkConflictAgainstOHWithDetail" passed.

OPatch succeeded.
[oracle@dbsgururc01 ~]$

1.5.2: System Space Check

As a GRID user:

grid@dbsgururc01 logs_patch_Oct2022]$ vi gridhome_space.txt
[grid@dbsgururc01 logs_patch_Oct2022]$ cat gridhome_space.txt
/home/grid/19c_patch_Oct2022/34449117/34416665/34419443
/home/grid/19c_patch_Oct2022/34449117/34416665/34444834
/home/grid/19c_patch_Oct2022/34449117/34416665/34428761
/home/grid/19c_patch_Oct2022/34449117/34416665/34580338
/home/grid/19c_patch_Oct2022/34449117/34416665/33575402
[grid@dbsgururc01 logs_patch_Oct2022]$ pwd
/home/grid/19c_patch_Oct2022/logs_patch_Oct2022
[grid@dbsgururc01 logs_patch_Oct2022]$ $ORACLE_HOME/OPatch/opatch prereq CheckSystemSpace -phBaseFile /home/grid/19c_patch_Oct2022/logs_patch_Oct2022/gridhome_space.txt
Oracle Interim Patch Installer version 12.2.0.1.32
Copyright (c) 2022, Oracle Corporation.  All rights reserved.

PREREQ session

Oracle Home       : /u01/app/grid/19c/gi_1
Central Inventory : /u01/app/oraInventory
   from           : /u01/app/grid/19c/gi_1/oraInst.loc
OPatch version    : 12.2.0.1.32
OUI version       : 12.2.0.7.0
Log file location : /u01/app/grid/19c/gi_1/cfgtoollogs/opatch/opatch2022-11-29_17-24-53PM_1.log

Invoking prereq "checksystemspace"

Prereq "checkSystemSpace" failed.

The details are:
Required amount of space(9867.579MB) is not available.

OPatch succeeded.

Oops……System space check gt failed for GRID home. So to fix this issue we release space manually and re-run the system check space again.

[grid@dbsgururc01 logs_patch_Oct2022]$ $ORACLE_HOME/OPatch/opatch prereq CheckSystemSpace -phBaseFile /home/grid/19c_patch_Oct2022/logs_patch_Oct2022/gridhome_space.txt
Oracle Interim Patch Installer version 12.2.0.1.32
Copyright (c) 2022, Oracle Corporation.  All rights reserved.

PREREQ session

Oracle Home       : /u01/app/grid/19c/gi_1
Central Inventory : /u01/app/oraInventory
   from           : /u01/app/grid/19c/gi_1/oraInst.loc
OPatch version    : 12.2.0.1.32
OUI version       : 12.2.0.7.0
Log file location : /u01/app/grid/19c/gi_1/cfgtoollogs/opatch/opatch2022-11-29_17-39-49PM_1.log

Invoking prereq "checksystemspace"

Prereq "checkSystemSpace" passed.

OPatch succeeded.
[grid@dbsgururc01 logs_patch_Oct2022]$

As an ORACLE user:

[grid@dbsgururc01 logs_patch_Oct2022]$ vi orahome_space.txt
[grid@dbsgururc01 logs_patch_Oct2022]$ cat orahome_space.txt
/home/grid/19c_patch_Oct2022/34449117/34416665/34419443
/home/grid/19c_patch_Oct2022/34449117/34416665/34444834
[grid@dbsgururc01 logs_patch_Oct2022]$
[oracle@dbsgururc01 ~]$ $ORACLE_HOME/OPatch/opatch prereq CheckSystemSpace -phBaseFile /home/grid/19c_patch_Oct2022/logs_patch_Oct2022/orahome_space.txt
Oracle Interim Patch Installer version 12.2.0.1.32
Copyright (c) 2022, Oracle Corporation.  All rights reserved.

PREREQ session

Oracle Home       : /u02/app/oracle/product/19c/dbhome_1
Central Inventory : /u01/app/oraInventory
   from           : /u02/app/oracle/product/19c/dbhome_1/oraInst.loc
OPatch version    : 12.2.0.1.32
OUI version       : 12.2.0.7.0
Log file location : /u02/app/oracle/product/19c/dbhome_1/cfgtoollogs/opatch/opatch2022-11-29_17-45-38PM_1.log

Invoking prereq "checksystemspace"

Prereq "checkSystemSpace" passed.

OPatch succeeded.
[oracle@dbsgururc01 ~]$

1.5.3: One-off Patch Conflict Detection

As a ROOT user:

[root@dbsgururc01 ~]# . oraenv
ORACLE_SID = [root] ? +ASM1
The Oracle base has been set to /u01/app/grid1
[root@dbsgururc01 ~]# export PATH=$ORACLE_HOME/OPatch:$PATH
[root@dbsgururc01 ~]# which opatchauto
/u01/app/grid/19c/gi_1/OPatch/opatchauto
[root@dbsgururc01 ~]# opatchauto version
Oracle OPatchAuto Version 12.2.1.32.0
Copyright (c) 2016, Oracle Corporation.  All rights reserved.

1. OPatchAuto version 12.2.1.32.0
2. OpatchautoDB version 12.2.0.1.32

[root@dbsgururc01 ~]# opatchauto apply /home/grid/19c_patch_Oct2022/34449117/34416665 -analyze

OPatchauto session is initiated at Tue Nov 29 18:16:31 2022

System initialization log file is /u01/app/grid/19c/gi_1/cfgtoollogs/opatchautodb/systemconfig2022-11-29_06-16-41PM.log.

Session log file is /u01/app/grid/19c/gi_1/cfgtoollogs/opatchauto/opatchauto2022-11-29_06-17-18PM.log
The id for this session is KW5U

Executing OPatch prereq operations to verify patch applicability on home /u01/app/grid/19c/gi_1

Executing OPatch prereq operations to verify patch applicability on home /u02/app/oracle/product/19c/dbhome_1
Patch applicability verified successfully on home /u02/app/oracle/product/19c/dbhome_1

Patch applicability verified successfully on home /u01/app/grid/19c/gi_1


Executing patch validation checks on home /u01/app/grid/19c/gi_1
Patch validation checks successfully completed on home /u01/app/grid/19c/gi_1


Executing patch validation checks on home /u02/app/oracle/product/19c/dbhome_1
Patch validation checks successfully completed on home /u02/app/oracle/product/19c/dbhome_1


Verifying SQL patch applicability on home /u02/app/oracle/product/19c/dbhome_1
SQL patch applicability verified successfully on home /u02/app/oracle/product/19c/dbhome_1

OPatchAuto successful.

--------------------------------Summary--------------------------------

Analysis for applying patches has completed successfully:

Host:dbsgururc01
CRS Home:/u01/app/grid/19c/gi_1
Version:19.0.0.0.0


==Following patches were SKIPPED:

Patch: /home/grid/19c_patch_Oct2022/34449117/34416665/33575402
Log: /u01/app/grid/19c/gi_1/cfgtoollogs/opatchauto/core/opatch/opatch2022-11-29_18-17-42PM_1.log
Reason: /home/grid/19c_patch_Oct2022/34449117/34416665/33575402 is not required to be applied to oracle home /u01/app/grid/19c/gi_1


==Following patches were SUCCESSFULLY analyzed to be applied:

Patch: /home/grid/19c_patch_Oct2022/34449117/34416665/34444834
Log: /u01/app/grid/19c/gi_1/cfgtoollogs/opatchauto/core/opatch/opatch2022-11-29_18-17-42PM_1.log

Patch: /home/grid/19c_patch_Oct2022/34449117/34416665/34428761
Log: /u01/app/grid/19c/gi_1/cfgtoollogs/opatchauto/core/opatch/opatch2022-11-29_18-17-42PM_1.log

Patch: /home/grid/19c_patch_Oct2022/34449117/34416665/34580338
Log: /u01/app/grid/19c/gi_1/cfgtoollogs/opatchauto/core/opatch/opatch2022-11-29_18-17-42PM_1.log

Patch: /home/grid/19c_patch_Oct2022/34449117/34416665/34419443
Log: /u01/app/grid/19c/gi_1/cfgtoollogs/opatchauto/core/opatch/opatch2022-11-29_18-17-42PM_1.log


Host:dbsgururc01
RAC Home:/u02/app/oracle/product/19c/dbhome_1
Version:19.0.0.0.0


==Following patches were SKIPPED:

Patch: /home/grid/19c_patch_Oct2022/34449117/34416665/34428761
Reason: This patch is not applicable to this specified target type - "rac_database"

Patch: /home/grid/19c_patch_Oct2022/34449117/34416665/33575402
Reason: This patch is not applicable to this specified target type - "rac_database"

Patch: /home/grid/19c_patch_Oct2022/34449117/34416665/34580338
Reason: This patch is not applicable to this specified target type - "rac_database"


==Following patches were SUCCESSFULLY analyzed to be applied:

Patch: /home/grid/19c_patch_Oct2022/34449117/34416665/34444834
Log: /u02/app/oracle/product/19c/dbhome_1/cfgtoollogs/opatchauto/core/opatch/opatch2022-11-29_18-17-42PM_1.log

Patch: /home/grid/19c_patch_Oct2022/34449117/34416665/34419443
Log: /u02/app/oracle/product/19c/dbhome_1/cfgtoollogs/opatchauto/core/opatch/opatch2022-11-29_18-17-42PM_1.log



OPatchauto session completed at Tue Nov 29 18:36:31 2022
Time taken to complete the session 20 minutes, 0 second
[root@dbsgururc01 ~]#

NODE02: dbsgururc02

Follow all steps of 1. Prerequisite on NODE02, here I am sharing command output from NODE 02.

1.1:

[grid@dbsgururc02 ~]$ chmod 775 -R 19c_patch_Oct2022
[grid@dbsgururc02 ~]$ ls -ld 19c_patch_Oct2022
drwxrwxr-x. 2 grid oinstall 6 Nov 29 18:25 19c_patch_Oct2022
[grid@dbsgururc02 ~]$ cd 19c_patch_Oct2022
[grid@dbsgururc02 19c_patch_Oct2022]$ pwd
/home/grid/19c_patch_Oct2022
[grid@dbsgururc02 ~]$

1.2:

[grid@dbsgururc02 19c_patch_Oct2022]$ unzip -q p34449117_190000_Linux-x86-64.zip
[grid@dbsgururc02 19c_patch_Oct2022]$ ls -lrt
total 2685768
drwxr-xr-x. 4 grid oinstall         57 Oct 19 11:22 34449117
-rw-rw-r--. 1 grid oinstall    1846194 Oct 19 12:17 PatchSearch.xml
-rw-r--r--. 1 grid oinstall 2748377244 Nov 29 18:33 p34449117_190000_Linux-x86-64.zip
[grid@dbsgururc02 19c_patch_Oct2022]$ cd 34449117
[grid@dbsgururc02 34449117]$ ls -lrt
total 28
drwxr-xr-x. 4 grid oinstall    67 Sep 27 22:16 34411846
drwxr-x---. 8 grid oinstall  4096 Oct 17 12:25 34416665
-rw-r--r--. 1 grid oinstall 22583 Oct 19 11:22 README.html
[grid@dbsgururc02 34449117]$

1.3:

[grid@dbsgururc02 34449117]$ mkdir logs_patch_Oct2022
[grid@dbsgururc02 34449117]$ ls -ld logs_patch_Oct2022
drwxr-xr-x. 2 grid oinstall 6 Nov 29 18:43 logs_patch_Oct2022
[grid@dbsgururc02 34449117]$ cd logs_patch_Oct2022
[grid@dbsgururc02 logs_patch_Oct2022]$ pwd
/home/grid/19c_patch_Oct2022/34449117/logs_patch_Oct2022
[grid@dbsgururc02 logs_patch_Oct2022]$ . oraenv
ORACLE_SID = [grid] ? +ASM2
The Oracle base has been set to /u01/app/grid1
[grid@dbsgururc02 logs_patch_Oct2022]$ export PATH=$ORACLE_HOME/OPatch:$PATH
[grid@dbsgururc02 logs_patch_Oct2022]$ which opatch
/u01/app/grid/19c/gi_1/OPatch/opatch
[grid@dbsgururc02 logs_patch_Oct2022]$ ps -ef | grep pmon > current_instance_Oct22_19cpatch.txt
[grid@dbsgururc02 logs_patch_Oct2022]$ ps -ef | grep tns > current_tns_Oct22_19cpatch.txt
[grid@dbsgururc02 logs_patch_Oct2022]$ cat /etc/oratab > current_oratab_Oct22_19cpatch.txt
[grid@dbsgururc02 logs_patch_Oct2022]$ $ORACLE_HOME/OPatch/opatch lsinventory > before_current_lsinventory_Oct22_19cpatch.txt
[grid@dbsgururc02 logs_patch_Oct2022]$ opatch lspatches
34318175;TOMCAT RELEASE UPDATE 19.0.0.0.0 (34318175)
34160635;OCW RELEASE UPDATE 19.16.0.0.0 (34160635)
34139601;ACFS RELEASE UPDATE 19.16.0.0.0 (34139601)
34133642;Database Release Update : 19.16.0.0.220719 (34133642)
33575402;DBWLM RELEASE UPDATE 19.0.0.0.0 (33575402)

OPatch succeeded.
[grid@dbsgururc02 logs_patch_Oct2022]$ ls -lrt
total 120
-rw-r--r--. 1 grid oinstall    201 Nov 29 18:45 current_instance_Oct22_19cpatch.txt
-rw-r--r--. 1 grid oinstall    487 Nov 29 18:45 current_tns_Oct22_19cpatch.txt
-rw-r--r--. 1 grid oinstall   1009 Nov 29 18:45 current_oratab_Oct22_19cpatch.txt
-rw-r--r--. 1 grid oinstall 109341 Nov 29 18:46 before_current_lsinventory_Oct22_19cpatch.txt
[grid@dbsgururc02 logs_patch_Oct2022]$ crsctl query crs activeversion -f
Oracle Clusterware active version on the cluster is [19.0.0.0.0]. The cluster upgrade state is [UPGRADE FINAL]. The cluster active patch level is [896235792].
[grid@dbsgururc02 logs_patch_Oct2022]$ crsctl query crs softwareversion
Oracle Clusterware version on node [dbsgururc02] is [19.0.0.0.0]
[grid@dbsgururc02 logs_patch_Oct2022]$ crsctl query crs releasepatch
Oracle Clusterware release patch level is [896235792] and the complete list of patches [33575402 34133642 34139601 34160635 34318175 ] have been applied on the local node. The release patch string is [19.16.0.0.0].
[grid@dbsgururc02 logs_patch_Oct2022]$ crsctl query crs softwarepatch
Oracle Clusterware patch level on node dbsgururc02 is [896235792].
[grid@dbsgururc02 logs_patch_Oct2022]$ asmcmd -V
asmcmd version 19.16.0.0.0
[grid@dbsgururc02 logs_patch_Oct2022]$ crsctl stat res -t
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.LISTENER.lsnr
               ONLINE  ONLINE       dbsgururc01              STABLE
               ONLINE  ONLINE       dbsgururc02              STABLE
ora.chad
               ONLINE  ONLINE       dbsgururc01              STABLE
               ONLINE  ONLINE       dbsgururc02              STABLE
ora.net1.network
               ONLINE  ONLINE       dbsgururc01              STABLE
               ONLINE  ONLINE       dbsgururc02              STABLE
ora.ons
               ONLINE  ONLINE       dbsgururc01              STABLE
               ONLINE  ONLINE       dbsgururc02              STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.ASMNET1LSNR_ASM.lsnr(ora.asmgroup)
      1        ONLINE  ONLINE       dbsgururc01              STABLE
      2        ONLINE  ONLINE       dbsgururc02              STABLE
      3        ONLINE  OFFLINE                               STABLE
ora.DATA1.dg(ora.asmgroup)
      1        ONLINE  ONLINE       dbsgururc01              STABLE
      2        ONLINE  ONLINE       dbsgururc02              STABLE
      3        OFFLINE OFFLINE                               STABLE
ora.FRA1.dg(ora.asmgroup)
      1        ONLINE  ONLINE       dbsgururc01              STABLE
      2        ONLINE  ONLINE       dbsgururc02              STABLE
      3        OFFLINE OFFLINE                               STABLE
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       dbsgururc02              STABLE
ora.LISTENER_SCAN2.lsnr
      1        ONLINE  ONLINE       dbsgururc01              STABLE
ora.LISTENER_SCAN3.lsnr
      1        ONLINE  ONLINE       dbsgururc01              STABLE
ora.MGMTLSNR
      1        ONLINE  ONLINE       dbsgururc01              169.254.2.113 192.16
                                                             8.10.100,STABLE
ora.VOTE1.dg(ora.asmgroup)
      1        ONLINE  ONLINE       dbsgururc01              STABLE
      2        ONLINE  ONLINE       dbsgururc02              STABLE
      3        OFFLINE OFFLINE                               STABLE
ora.asm(ora.asmgroup)
      1        ONLINE  ONLINE       dbsgururc01              Started,STABLE
      2        ONLINE  ONLINE       dbsgururc02              Started,STABLE
      3        ONLINE  OFFLINE                               STABLE
ora.asmnet1.asmnetwork(ora.asmgroup)
      1        ONLINE  ONLINE       dbsgururc01              STABLE
      2        ONLINE  ONLINE       dbsgururc02              STABLE
      3        ONLINE  OFFLINE                               STABLE
ora.cvu
      1        ONLINE  ONLINE       dbsgururc01              STABLE
ora.dbsgucdb.db
      1        ONLINE  ONLINE       dbsgururc01              Open,HOME=/u02/app/o
                                                             racle/product/19c/db
                                                             home_1,STABLE
      2        ONLINE  ONLINE       dbsgururc02              Open,HOME=/u02/app/o
                                                             racle/product/19c/db
                                                             home_1,STABLE
ora.dbsgururc01.vip
      1        ONLINE  ONLINE       dbsgururc01              STABLE
ora.dbsgururc02.vip
      1        ONLINE  ONLINE       dbsgururc02              STABLE
ora.mgmtdb
      1        ONLINE  ONLINE       dbsgururc01              Open,STABLE
ora.qosmserver
      1        ONLINE  ONLINE       dbsgururc01              STABLE
ora.scan1.vip
      1        ONLINE  ONLINE       dbsgururc02              STABLE
ora.scan2.vip
      1        ONLINE  ONLINE       dbsgururc01              STABLE
ora.scan3.vip
      1        ONLINE  ONLINE       dbsgururc01              STABLE
--------------------------------------------------------------------------------
[grid@dbsgururc02 logs_patch_Oct2022]$

1.4:

[grid@dbsgururc02 logs_patch_Oct2022]$ opatch version  ==>As a GRID user
OPatch Version: 12.2.0.1.32

OPatch succeeded.
[grid@dbsgururc02 logs_patch_Oct2022]$ 
[oracle@dbsgururc02 ~]$ /u02/app/oracle/product/19c/dbhome_1/OPatch/opatch version ==> As an Oracle user
OPatch Version: 12.2.0.1.32

OPatch succeeded.
[oracle@dbsgururc02 ~]$

1.5.1:

[grid@dbsgururc02 logs_patch_Oct2022]$ $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /home/grid/19c_patch_Oct2022/34449117/34416665/34419443
Oracle Interim Patch Installer version 12.2.0.1.32
Copyright (c) 2022, Oracle Corporation. All rights reserved.

PREREQ session

Invoking prereq "checkconflictagainstohwithdetail"

Prereq "checkConflictAgainstOHWithDetail" passed.

OPatch succeeded.
[grid@dbsgururc02 logs_patch_Oct2022]$ $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /home/grid/19c_patch_Oct2022/34449117/34416665/34444834
Oracle Interim Patch Installer version 12.2.0.1.32
Copyright (c) 2022, Oracle Corporation. All rights reserved.

PREREQ session

Invoking prereq "checkconflictagainstohwithdetail"

Prereq "checkConflictAgainstOHWithDetail" passed.

OPatch succeeded.
[grid@dbsgururc02 logs_patch_Oct2022]$ $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /home/grid/19c_patch_Oct2022/34449117/34416665/34428761
Oracle Interim Patch Installer version 12.2.0.1.32
Copyright (c) 2022, Oracle Corporation. All rights reserved.

PREREQ session

Invoking prereq "checkconflictagainstohwithdetail"

Prereq "checkConflictAgainstOHWithDetail" passed.

OPatch succeeded.
[grid@dbsgururc02 logs_patch_Oct2022]$ $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /home/grid/19c_patch_Oct2022/34449117/34416665/34580338
Oracle Interim Patch Installer version 12.2.0.1.32
Copyright (c) 2022, Oracle Corporation. All rights reserved.

PREREQ session

Invoking prereq "checkconflictagainstohwithdetail"

Prereq "checkConflictAgainstOHWithDetail" passed.

OPatch succeeded.
[grid@dbsgururc02 logs_patch_Oct2022]$ $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /home/grid/19c_patch_Oct2022/34449117/34416665/33575402
Oracle Interim Patch Installer version 12.2.0.1.32
Copyright (c) 2022, Oracle Corporation. All rights reserved.

PREREQ session

Invoking prereq "checkconflictagainstohwithdetail"

Prereq "checkConflictAgainstOHWithDetail" passed.

OPatch succeeded.
[grid@dbsgururc02 logs_patch_Oct2022]$

[oracle@dbsgururc02 ~]$ . oraenv
ORACLE_SID = [oracle] ? dbsgucdb2
The Oracle base has been set to /u02/app/oracle
[oracle@dbsgururc02 ~]$ $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /home/grid/19c_patch_Oct2022/34449117/34416665/34419443
Oracle Interim Patch Installer version 12.2.0.1.32
Copyright (c) 2022, Oracle Corporation. All rights reserved.

PREREQ session

Invoking prereq "checkconflictagainstohwithdetail"

Prereq "checkConflictAgainstOHWithDetail" passed.

OPatch succeeded.
[oracle@dbsgururc02 ~]$ $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /home/grid/19c_patch_Oct2022/34449117/34416665/34444834
Oracle Interim Patch Installer version 12.2.0.1.32
Copyright (c) 2022, Oracle Corporation. All rights reserved.

PREREQ session

Invoking prereq "checkconflictagainstohwithdetail"

Prereq "checkConflictAgainstOHWithDetail" passed.

OPatch succeeded.
[oracle@dbsgururc02 ~]$

1.5.2:

[grid@dbsgururc02 logs_patch_Oct2022]$ pwd
/home/grid/19c_patch_Oct2022/34449117/logs_patch_Oct2022
[grid@dbsgururc02 logs_patch_Oct2022]$ $ORACLE_HOME/OPatch/opatch prereq CheckSystemSpace -phBaseFile /home/grid/19c_patch_Oct2022/34449117/logs_patch_Oct2022/gridhome_space.txt
Oracle Interim Patch Installer version 12.2.0.1.32
Copyright (c) 2022, Oracle Corporation.  All rights reserved.

PREREQ session

Oracle Home       : /u01/app/grid/19c/gi_1
Central Inventory : /u01/app/oraInventory
   from           : /u01/app/grid/19c/gi_1/oraInst.loc
OPatch version    : 12.2.0.1.32
OUI version       : 12.2.0.7.0
Log file location : /u01/app/grid/19c/gi_1/cfgtoollogs/opatch/opatch2022-11-29_18-58-20PM_1.log

Invoking prereq "checksystemspace"

Prereq "checkSystemSpace" passed.

OPatch succeeded.
[grid@dbsgururc02 logs_patch_Oct2022]$
[oracle@dbsgururc02 ~]$ $ORACLE_HOME/OPatch/opatch prereq CheckSystemSpace -phBaseFile /home/grid/19c_patch_Oct2022/34449117/logs_patch_Oct2022/orahome_space.txt
Oracle Interim Patch Installer version 12.2.0.1.32
Copyright (c) 2022, Oracle Corporation.  All rights reserved.

PREREQ session

Oracle Home       : /u02/app/oracle/product/19c/dbhome_1
Central Inventory : /u01/app/oraInventory
   from           : /u02/app/oracle/product/19c/dbhome_1/oraInst.loc
OPatch version    : 12.2.0.1.32
OUI version       : 12.2.0.7.0
Log file location : /u02/app/oracle/product/19c/dbhome_1/cfgtoollogs/opatch/opatch2022-11-29_19-08-20PM_1.log

Invoking prereq "checksystemspace"

Prereq "checkSystemSpace" passed.

OPatch succeeded.
[oracle@dbsgururc02 ~]$

1.5.3:

[root@dbsgururc02 ~]# . oraenv
ORACLE_SID = [root] ? +ASM2
The Oracle base has been set to /u01/app/grid1
[root@dbsgururc02 ~]# export PATH=$ORACLE_HOME/OPatch:$PATH
[root@dbsgururc02 ~]# which opatchauto
/u01/app/grid/19c/gi_1/OPatch/opatchauto
[root@dbsgururc02 ~]# opatchauto version
Oracle OPatchAuto Version 12.2.1.32.0
Copyright (c) 2016, Oracle Corporation.  All rights reserved.

1. OPatchAuto version 12.2.1.32.0
2. OpatchautoDB version 12.2.0.1.32

[root@dbsgururc02 ~]# opatchauto apply /home/grid/19c_patch_Oct2022/34449117/34416665 -analyze

OPatchauto session is initiated at Tue Nov 29 19:10:37 2022

System initialization log file is /u01/app/grid/19c/gi_1/cfgtoollogs/opatchautodb/systemconfig2022-11-29_07-10-43PM.log.

Session log file is /u01/app/grid/19c/gi_1/cfgtoollogs/opatchauto/opatchauto2022-11-29_07-11-18PM.log
The id for this session is SPLX

Executing OPatch prereq operations to verify patch applicability on home /u01/app/grid/19c/gi_1

Executing OPatch prereq operations to verify patch applicability on home /u02/app/oracle/product/19c/dbhome_1
Patch applicability verified successfully on home /u01/app/grid/19c/gi_1

Patch applicability verified successfully on home /u02/app/oracle/product/19c/dbhome_1


Executing patch validation checks on home /u01/app/grid/19c/gi_1
Patch validation checks successfully completed on home /u01/app/grid/19c/gi_1


Executing patch validation checks on home /u02/app/oracle/product/19c/dbhome_1
Patch validation checks successfully completed on home /u02/app/oracle/product/19c/dbhome_1


Verifying SQL patch applicability on home /u02/app/oracle/product/19c/dbhome_1
SQL patch applicability verified successfully on home /u02/app/oracle/product/19c/dbhome_1

OPatchAuto successful.

--------------------------------Summary--------------------------------

Analysis for applying patches has completed successfully:

Host:dbsgururc02
CRS Home:/u01/app/grid/19c/gi_1
Version:19.0.0.0.0


==Following patches were SKIPPED:

Patch: /home/grid/19c_patch_Oct2022/34449117/34416665/33575402
Log: /u01/app/grid/19c/gi_1/cfgtoollogs/opatchauto/core/opatch/opatch2022-11-29_19-11-36PM_1.log
Reason: /home/grid/19c_patch_Oct2022/34449117/34416665/33575402 is not required to be applied to oracle home /u01/app/grid/19c/gi_1


==Following patches were SUCCESSFULLY analyzed to be applied:

Patch: /home/grid/19c_patch_Oct2022/34449117/34416665/34444834
Log: /u01/app/grid/19c/gi_1/cfgtoollogs/opatchauto/core/opatch/opatch2022-11-29_19-11-36PM_1.log

Patch: /home/grid/19c_patch_Oct2022/34449117/34416665/34428761
Log: /u01/app/grid/19c/gi_1/cfgtoollogs/opatchauto/core/opatch/opatch2022-11-29_19-11-36PM_1.log

Patch: /home/grid/19c_patch_Oct2022/34449117/34416665/34580338
Log: /u01/app/grid/19c/gi_1/cfgtoollogs/opatchauto/core/opatch/opatch2022-11-29_19-11-36PM_1.log

Patch: /home/grid/19c_patch_Oct2022/34449117/34416665/34419443
Log: /u01/app/grid/19c/gi_1/cfgtoollogs/opatchauto/core/opatch/opatch2022-11-29_19-11-36PM_1.log


Host:dbsgururc02
RAC Home:/u02/app/oracle/product/19c/dbhome_1
Version:19.0.0.0.0


==Following patches were SKIPPED:

Patch: /home/grid/19c_patch_Oct2022/34449117/34416665/34428761
Reason: This patch is not applicable to this specified target type - "rac_database"

Patch: /home/grid/19c_patch_Oct2022/34449117/34416665/33575402
Reason: This patch is not applicable to this specified target type - "rac_database"

Patch: /home/grid/19c_patch_Oct2022/34449117/34416665/34580338
Reason: This patch is not applicable to this specified target type - "rac_database"


==Following patches were SUCCESSFULLY analyzed to be applied:

Patch: /home/grid/19c_patch_Oct2022/34449117/34416665/34444834
Log: /u02/app/oracle/product/19c/dbhome_1/cfgtoollogs/opatchauto/core/opatch/opatch2022-11-29_19-11-38PM_1.log

Patch: /home/grid/19c_patch_Oct2022/34449117/34416665/34419443
Log: /u02/app/oracle/product/19c/dbhome_1/cfgtoollogs/opatchauto/core/opatch/opatch2022-11-29_19-11-38PM_1.log



OPatchauto session completed at Tue Nov 29 19:15:12 2022
Time taken to complete the session 4 minutes, 35 seconds
[root@dbsgururc02 ~]#

OJVM Patch 34411846

NODE01: dbsgururc01

1.6.1: OJVM component status.

SQL> SELECT version, status FROM dba_registry WHERE comp_id='JAVAVM';

VERSION                        STATUS
------------------------------ --------------------------------------------
19.0.0.0.0                     VALID

1.6.2: OJVM utilization.

SQL> select count(*) from x$kglob where KGLOBTYP = 29 OR KGLOBTYP = 56;

  COUNT(*)
----------
         0

1.6.3: Patch Conflict Check

[oracle@dbsgururc01 ~]$ cd /home/grid/19c_patch_Oct2022/34449117/34411846/
[oracle@dbsgururc01 34411846]$ ls -lrt
total 60
drwxr-x---. 6 grid oinstall    60 Sep 27 22:16 files
drwxr-x---. 3 grid oinstall    20 Sep 27 22:16 etc
-rw-r--r--. 1 grid oinstall    11 Sep 27 22:16 README.txt
-rw-rw-r--. 1 grid oinstall 56618 Oct 17 21:48 README.html
[oracle@dbsgururc01 34411846]$
[oracle@dbsgururc01 ~]$ export PATH=$ORACLE_HOME/OPatch:$PATH
[oracle@dbsgururc01 ~]$ which opatch
/u02/app/oracle/product/19c/dbhome_1/OPatch/opatch
[oracle@dbsgururc01 34411846]$ opatch prereq CheckConflictAgainstOHWithDetail -ph ./
Oracle Interim Patch Installer version 12.2.0.1.32
Copyright (c) 2022, Oracle Corporation.  All rights reserved.

PREREQ session

Oracle Home       : /u02/app/oracle/product/19c/dbhome_1
Central Inventory : /u01/app/oraInventory
   from           : /u02/app/oracle/product/19c/dbhome_1/oraInst.loc
OPatch version    : 12.2.0.1.32
OUI version       : 12.2.0.7.0
Log file location : /u02/app/oracle/product/19c/dbhome_1/cfgtoollogs/opatch/opatch2022-11-29_22-20-35PM_1.log

Invoking prereq "checkconflictagainstohwithdetail"

Prereq "checkConflictAgainstOHWithDetail" passed.

OPatch succeeded.
[oracle@dbsgururc01 34411846]$

Repeat the same steps of NODE01 1.6.1 through 1.6.3 on NODE02.

NODE02: dbsgururc02

1.6.1:

SQL> SELECT version, status FROM dba_registry WHERE comp_id='JAVAVM';

VERSION                        STATUS
------------------------------ --------------------------------------------
19.0.0.0.0                     VALID

1.6.2:

SQL> select count(*) from x$kglob where KGLOBTYP = 29 OR KGLOBTYP = 56;

  COUNT(*)
----------
     11890

1.6.3:

[oracle@dbsgururc02 ~]$ cd /home/grid/19c_patch_Oct2022/34449117/34411846/
[oracle@dbsgururc02 34411846]$ ls -lrt
total 60
drwxr-x---. 6 grid oinstall    60 Sep 27 22:16 files
drwxr-x---. 3 grid oinstall    20 Sep 27 22:16 etc
-rw-r--r--. 1 grid oinstall    11 Sep 27 22:16 README.txt
-rw-rw-r--. 1 grid oinstall 56618 Oct 17 21:48 README.html
[oracle@dbsgururc02 ~]$ export PATH=$ORACLE_HOME/OPatch:$PATH
[oracle@dbsgururc02 ~]$ which opatch
/u02/app/oracle/product/19c/dbhome_1/OPatch/opatch
[oracle@dbsgururc02 34411846]$ opatch prereq CheckConflictAgainstOHWithDetail -ph ./
Oracle Interim Patch Installer version 12.2.0.1.32
Copyright (c) 2022, Oracle Corporation.  All rights reserved.

PREREQ session

Oracle Home       : /u02/app/oracle/product/19c/dbhome_1
Central Inventory : /u01/app/oraInventory
   from           : /u02/app/oracle/product/19c/dbhome_1/oraInst.loc
OPatch version    : 12.2.0.1.32
OUI version       : 12.2.0.7.0
Log file location : /u02/app/oracle/product/19c/dbhome_1/cfgtoollogs/opatch/opatch2022-11-29_22-32-32PM_1.log

Invoking prereq "checkconflictagainstohwithdetail"

Prereq "checkConflictAgainstOHWithDetail" passed.

OPatch succeeded.
[oracle@dbsgururc02 34411846]$

2. Backup Plan

2.1: It’s always recommended to have a backup before starting any major changes. So make sure you have a backup of ORACLE HOME as a backup plan if require to restore in case of failure.

[grid@dbsgururc01 logs_patch_Oct2022]$ tar -cvf grid_home_29Oct2022.tar $ORACLE_HOME  ==> As a GRID user
[oracle@dbsgururc01 ~]$ tar -cvf oracle_home_29Oct2022.tar $ORACLE_HOME  ==> As an ORACLE user

2.2: Additionally make sure to have a successful FULL DB backup of the database. Click here to get RMAN Full Database Backup Scripts in Oracle.

3. Disable Jobs

3.1: Enable blackout in OEM, Disable jobs in crontab & DBMS scheduler, etc. if applicable. Click here to know more about Target Blackouts in OEM13c using EMCTL.

3.2: Stop all dependent applications/jobs, as applicable.

4. GRID Patch Install 34416665

The OPatch mileage has automated the patch operation for the Oracle Grid structure( Grid) home and the Oracle RAC database homes. It operates by querying configurations and automating the way needed for doctoring each Oracle RAC database home of the same interpretation and the Grid home.

The mileage must be executed by an operating system( OS) stoner with root privileges, and it must be executed on each node in the cluster if the Grid home or Oracle RAC database home is an in a non-shared storehouse. The mileage can be run in parallel on the cluster bumps except for the first ( any) node.

Depending on the command line options specified, one incantation of OPatchAuto can patch the Grid home, Oracle RAC database homes, or both Grid and Oracle RAC database homes of the same Oracle release interpretation as the patch. You can also roll back the patch with the same selectivity. So let’s start the GRID patch 34416665 and apply it first on NODE01 dbsgururc01 and at a time one node only then moves to the next NODE 02.

NODE01: dbsgururc01

4.1: Apply GRID patch (34416665).

[root@dbsgururc01 ~]# env | grep ORA
ORACLE_SID=+ASM1
ORACLE_BASE=/u01/app/grid1
ORACLE_HOME=/u01/app/grid/19c/gi_1
[root@dbsgururc01 ~]# opatchauto apply /home/grid/19c_patch_Oct2022/34449117/34416665/

OPatchauto session is initiated at Tue Nov 29 19:20:01 2022

System initialization log file is /u01/app/grid/19c/gi_1/cfgtoollogs/opatchautodb/systemconfig2022-11-29_07-20-07PM.log.

Session log file is /u01/app/grid/19c/gi_1/cfgtoollogs/opatchauto/opatchauto2022-11-29_07-20-31PM.log
The id for this session is X4F7

Executing OPatch prereq operations to verify patch applicability on home /u01/app/grid/19c/gi_1

Executing OPatch prereq operations to verify patch applicability on home /u02/app/oracle/product/19c/dbhome_1
Patch applicability verified successfully on home /u02/app/oracle/product/19c/dbhome_1

Patch applicability verified successfully on home /u01/app/grid/19c/gi_1


Executing patch validation checks on home /u01/app/grid/19c/gi_1
Patch validation checks successfully completed on home /u01/app/grid/19c/gi_1


Executing patch validation checks on home /u02/app/oracle/product/19c/dbhome_1
Patch validation checks successfully completed on home /u02/app/oracle/product/19c/dbhome_1


Verifying SQL patch applicability on home /u02/app/oracle/product/19c/dbhome_1
SQL patch applicability verified successfully on home /u02/app/oracle/product/19c/dbhome_1


Preparing to bring down database service on home /u02/app/oracle/product/19c/dbhome_1
Successfully prepared home /u02/app/oracle/product/19c/dbhome_1 to bring down database service


Performing prepatch operations on CRS - bringing down CRS service on home /u01/app/grid/19c/gi_1
Prepatch operation log file location: /u01/app/grid1/crsdata/dbsgururc01/crsconfig/crs_prepatch_apply_inplace_dbsgururc01_2022-11-29_07-25-41PM.log
CRS service brought down successfully on home /u01/app/grid/19c/gi_1


Performing prepatch operation on home /u02/app/oracle/product/19c/dbhome_1
Prepatch operation completed successfully on home /u02/app/oracle/product/19c/dbhome_1


Start applying binary patch on home /u02/app/oracle/product/19c/dbhome_1
Binary patch applied successfully on home /u02/app/oracle/product/19c/dbhome_1


Performing postpatch operation on home /u02/app/oracle/product/19c/dbhome_1
Postpatch operation completed successfully on home /u02/app/oracle/product/19c/dbhome_1


Start applying binary patch on home /u01/app/grid/19c/gi_1
Binary patch applied successfully on home /u01/app/grid/19c/gi_1


Performing postpatch operations on CRS - starting CRS service on home /u01/app/grid/19c/gi_1
Postpatch operation log file location: /u01/app/grid1/crsdata/dbsgururc01/crsconfig/crs_postpatch_apply_inplace_dbsgururc01_2022-11-29_07-58-29PM.log
CRS service started successfully on home /u01/app/grid/19c/gi_1


Preparing home /u02/app/oracle/product/19c/dbhome_1 after database service restarted
No step execution required.........


Trying to apply SQL patch on home /u02/app/oracle/product/19c/dbhome_1
SQL patch applied successfully on home /u02/app/oracle/product/19c/dbhome_1

OPatchAuto successful.

--------------------------------Summary--------------------------------

Patching is completed successfully. Please find the summary as follows:

Host:dbsgururc01
RAC Home:/u02/app/oracle/product/19c/dbhome_1
Version:19.0.0.0.0
Summary:

==Following patches were SKIPPED:

Patch: /home/grid/19c_patch_Oct2022/34449117/34416665/34428761
Reason: This patch is not applicable to this specified target type - "rac_database"

Patch: /home/grid/19c_patch_Oct2022/34449117/34416665/33575402
Reason: This patch is not applicable to this specified target type - "rac_database"

Patch: /home/grid/19c_patch_Oct2022/34449117/34416665/34580338
Reason: This patch is not applicable to this specified target type - "rac_database"


==Following patches were SUCCESSFULLY applied:

Patch: /home/grid/19c_patch_Oct2022/34449117/34416665/34419443
Log: /u02/app/oracle/product/19c/dbhome_1/cfgtoollogs/opatchauto/core/opatch/opatch2022-11-29_19-36-20PM_1.log

Patch: /home/grid/19c_patch_Oct2022/34449117/34416665/34444834
Log: /u02/app/oracle/product/19c/dbhome_1/cfgtoollogs/opatchauto/core/opatch/opatch2022-11-29_19-36-20PM_1.log


Host:dbsgururc01
CRS Home:/u01/app/grid/19c/gi_1
Version:19.0.0.0.0
Summary:

==Following patches were SKIPPED:

Patch: /home/grid/19c_patch_Oct2022/34449117/34416665/33575402
Log: /u01/app/grid/19c/gi_1/cfgtoollogs/opatchauto/core/opatch/opatch2022-11-29_19-20-52PM_1.log
Reason: /home/grid/19c_patch_Oct2022/34449117/34416665/33575402 is not required to be applied to oracle home /u01/app/grid/19c/gi_1


==Following patches were SUCCESSFULLY applied:

Patch: /home/grid/19c_patch_Oct2022/34449117/34416665/34419443
Log: /u01/app/grid/19c/gi_1/cfgtoollogs/opatchauto/core/opatch/opatch2022-11-29_19-47-46PM_1.log

Patch: /home/grid/19c_patch_Oct2022/34449117/34416665/34428761
Log: /u01/app/grid/19c/gi_1/cfgtoollogs/opatchauto/core/opatch/opatch2022-11-29_19-47-46PM_1.log

Patch: /home/grid/19c_patch_Oct2022/34449117/34416665/34444834
Log: /u01/app/grid/19c/gi_1/cfgtoollogs/opatchauto/core/opatch/opatch2022-11-29_19-47-46PM_1.log

Patch: /home/grid/19c_patch_Oct2022/34449117/34416665/34580338
Log: /u01/app/grid/19c/gi_1/cfgtoollogs/opatchauto/core/opatch/opatch2022-11-29_19-47-46PM_1.log



OPatchauto session completed at Tue Nov 29 20:10:03 2022
Time taken to complete the session 50 minutes, 2 seconds
[root@dbsgururc01 ~]#

4.2: Validate the applied patches on HOME.

[grid@dbsgururc01 logs_patch_Oct2022]$ opatch lspatches
34580338;TOMCAT RELEASE UPDATE 19.0.0.0.0 (34580338)
34444834;OCW RELEASE UPDATE 19.17.0.0.0 (34444834)
34428761;ACFS RELEASE UPDATE 19.17.0.0.0 (34428761)
34419443;Database Release Update : 19.17.0.0.221018 (34419443)
33575402;DBWLM RELEASE UPDATE 19.0.0.0.0 (33575402)

OPatch succeeded.
[grid@dbsgururc01 logs_patch_Oct2022]$ opatch lsinventory -detail -oh $ORACLE_HOME > after_current_grid_lsinventory_Oct22_19cpatch.txt
[grid@dbsgururc01 logs_patch_Oct2022]$ ls -lrt after_current_grid_lsinventory_Oct22_19cpatch.txt
-rw-r--r--. 1 grid oinstall 1824857 Nov 30 16:37 after_current_grid_lsinventory_Oct22_19cpatch.txt
[grid@dbsgururc01 logs_patch_Oct2022]$

NODE02: dbsgururc02

Repeat the same set of steps of NODE01.

4.1:

[root@dbsgururc02 ~]# env | grep ORA
ORACLE_SID=+ASM2
ORACLE_BASE=/u01/app/grid1
ORACLE_HOME=/u01/app/grid/19c/gi_1
[root@dbsgururc02 ~]# opatchauto apply /home/grid/19c_patch_Oct2022/34449117/34416665/

OPatchauto session is initiated at Tue Nov 29 20:17:23 2022

System initialization log file is /u01/app/grid/19c/gi_1/cfgtoollogs/opatchautodb/systemconfig2022-11-29_08-17-44PM.log.

Session log file is /u01/app/grid/19c/gi_1/cfgtoollogs/opatchauto/opatchauto2022-11-29_08-18-30PM.log
The id for this session is 1R8L

Executing OPatch prereq operations to verify patch applicability on home /u01/app/grid/19c/gi_1

Executing OPatch prereq operations to verify patch applicability on home /u02/app/oracle/product/19c/dbhome_1
Patch applicability verified successfully on home /u01/app/grid/19c/gi_1

Patch applicability verified successfully on home /u02/app/oracle/product/19c/dbhome_1


Executing patch validation checks on home /u01/app/grid/19c/gi_1
Patch validation checks successfully completed on home /u01/app/grid/19c/gi_1


Executing patch validation checks on home /u02/app/oracle/product/19c/dbhome_1
Patch validation checks successfully completed on home /u02/app/oracle/product/19c/dbhome_1


Verifying SQL patch applicability on home /u02/app/oracle/product/19c/dbhome_1
SQL patch applicability verified successfully on home /u02/app/oracle/product/19c/dbhome_1


Preparing to bring down database service on home /u02/app/oracle/product/19c/dbhome_1
Successfully prepared home /u02/app/oracle/product/19c/dbhome_1 to bring down database service


Performing prepatch operations on CRS - bringing down CRS service on home /u01/app/grid/19c/gi_1
Prepatch operation log file location: /u01/app/grid1/crsdata/dbsgururc02/crsconfig/crs_prepatch_apply_inplace_dbsgururc02_2022-11-29_08-31-09PM.log
CRS service brought down successfully on home /u01/app/grid/19c/gi_1


Performing prepatch operation on home /u02/app/oracle/product/19c/dbhome_1
Prepatch operation completed successfully on home /u02/app/oracle/product/19c/dbhome_1


Start applying binary patch on home /u02/app/oracle/product/19c/dbhome_1
Binary patch applied successfully on home /u02/app/oracle/product/19c/dbhome_1


Performing postpatch operation on home /u02/app/oracle/product/19c/dbhome_1
Postpatch operation completed successfully on home /u02/app/oracle/product/19c/dbhome_1


Start applying binary patch on home /u01/app/grid/19c/gi_1
Binary patch applied successfully on home /u01/app/grid/19c/gi_1

Checking shared status of home.....

Performing postpatch operations on CRS - starting CRS service on home /u01/app/grid/19c/gi_1




Postpatch operation log file location: /u01/app/grid1/crsdata/dbsgururc02/crsconfig/crs_postpatch_apply_inplace_dbsgururc02_2022-11-29_09-10-20PM.log
CRS service started successfully on home /u01/app/grid/19c/gi_1


Preparing home /u02/app/oracle/product/19c/dbhome_1 after database service restarted
No step execution required.........


Trying to apply SQL patch on home /u02/app/oracle/product/19c/dbhome_1
SQL patch applied successfully on home /u02/app/oracle/product/19c/dbhome_1

OPatchAuto successful.

--------------------------------Summary--------------------------------

Patching is completed successfully. Please find the summary as follows:

Host:dbsgururc02
RAC Home:/u02/app/oracle/product/19c/dbhome_1
Version:19.0.0.0.0
Summary:

==Following patches were SKIPPED:

Patch: /home/grid/19c_patch_Oct2022/34449117/34416665/34428761
Reason: This patch is not applicable to this specified target type - "rac_database"

Patch: /home/grid/19c_patch_Oct2022/34449117/34416665/33575402
Reason: This patch is not applicable to this specified target type - "rac_database"

Patch: /home/grid/19c_patch_Oct2022/34449117/34416665/34580338
Reason: This patch is not applicable to this specified target type - "rac_database"


==Following patches were SUCCESSFULLY applied:

Patch: /home/grid/19c_patch_Oct2022/34449117/34416665/34419443
Log: /u02/app/oracle/product/19c/dbhome_1/cfgtoollogs/opatchauto/core/opatch/opatch2022-11-29_20-42-52PM_1.log

Patch: /home/grid/19c_patch_Oct2022/34449117/34416665/34444834
Log: /u02/app/oracle/product/19c/dbhome_1/cfgtoollogs/opatchauto/core/opatch/opatch2022-11-29_20-42-52PM_1.log


Host:dbsgururc02
CRS Home:/u01/app/grid/19c/gi_1
Version:19.0.0.0.0
Summary:

==Following patches were SKIPPED:

Patch: /home/grid/19c_patch_Oct2022/34449117/34416665/33575402
Log: /u01/app/grid/19c/gi_1/cfgtoollogs/opatchauto/core/opatch/opatch2022-11-29_20-18-53PM_1.log
Reason: /home/grid/19c_patch_Oct2022/34449117/34416665/33575402 is not required to be applied to oracle home /u01/app/grid/19c/gi_1


==Following patches were SUCCESSFULLY applied:

Patch: /home/grid/19c_patch_Oct2022/34449117/34416665/34419443
Log: /u01/app/grid/19c/gi_1/cfgtoollogs/opatchauto/core/opatch/opatch2022-11-29_20-59-05PM_1.log

Patch: /home/grid/19c_patch_Oct2022/34449117/34416665/34428761
Log: /u01/app/grid/19c/gi_1/cfgtoollogs/opatchauto/core/opatch/opatch2022-11-29_20-59-05PM_1.log

Patch: /home/grid/19c_patch_Oct2022/34449117/34416665/34444834
Log: /u01/app/grid/19c/gi_1/cfgtoollogs/opatchauto/core/opatch/opatch2022-11-29_20-59-05PM_1.log

Patch: /home/grid/19c_patch_Oct2022/34449117/34416665/34580338
Log: /u01/app/grid/19c/gi_1/cfgtoollogs/opatchauto/core/opatch/opatch2022-11-29_20-59-05PM_1.log



OPatchauto session completed at Tue Nov 29 21:46:41 2022
Time taken to complete the session 47 minutes, 53 seconds
[root@dbsgururc02 ~]#

4.2:

[grid@dbsgururc02 logs_patch_Oct2022]$ opatch lspatches
34580338;TOMCAT RELEASE UPDATE 19.0.0.0.0 (34580338)
34444834;OCW RELEASE UPDATE 19.17.0.0.0 (34444834)
34428761;ACFS RELEASE UPDATE 19.17.0.0.0 (34428761)
34419443;Database Release Update : 19.17.0.0.221018 (34419443)
33575402;DBWLM RELEASE UPDATE 19.0.0.0.0 (33575402)

OPatch succeeded.
[grid@dbsgururc02 logs_patch_Oct2022]$ opatch lsinventory -detail -oh $ORACLE_HOME > after_current_grid_lsinventory_Oct22_19cpatch.txt
[grid@dbsgururc02 logs_patch_Oct2022]$ ls -lrt after_current_grid_lsinventory_Oct22_19cpatch.txt
-rw-r--r--. 1 grid oinstall 1824857 Nov 30 16:56 after_current_grid_lsinventory_Oct22_19cpatch.txt
[grid@dbsgururc02 logs_patch_Oct2022]$

5. OJVM Patch Install 34411846

NODE01: dbsgururc01

5.1: Make sure that you have completely shut down all the services running from the ORACLE HOME where you are performing patch activity on NODE01. Shutdown instances and other services running associated from the same ORACLE HOME.

[oracle@dbsgururc01 34411846]$ srvctl status database -db dbsgucdb
Instance dbsgucdb1 is running on node dbsgururc01
Instance dbsgucdb2 is running on node dbsgururc02
[oracle@dbsgururc01 34411846]$ srvctl stop instance -db dbsgucdb -i dbsgucdb1
[oracle@dbsgururc01 34411846]$ srvctl status database -db dbsgucdb
Instance dbsgucdb1 is not running on node dbsgururc01
Instance dbsgucdb2 is running on node dbsgururc02

5.2: Apply OJVM patch (33192694):

Verifying environment and performing prerequisite checks...
OPatch continues with these patches: 34411846

Do you want to proceed? [y|n]
y
User Responded with: Y
All checks passed.

Please shutdown Oracle instances running out of this ORACLE_HOME on the local system.
(Oracle Home = '/u02/app/oracle/product/19c/dbhome_1')

Is the local system ready for patching? [y|n]
y
User Responded with: Y
Backing up files...
Applying interim patch '34411846' to OH '/u02/app/oracle/product/19c/dbhome_1'

Patching component oracle.javavm.server, 19.0.0.0.0...

Patching component oracle.javavm.server.core, 19.0.0.0.0...

Patching component oracle.rdbms.dbscripts, 19.0.0.0.0...

Patching component oracle.rdbms, 19.0.0.0.0...

Patching component oracle.javavm.client, 19.0.0.0.0...
Patch 34411846 successfully applied.
Sub-set patch [34086870] has become inactive due to the application of a super-set patch [34411846].
Please refer to Doc ID 2161861.1 for any possible further required actions.
Log file location: /u02/app/oracle/product/19c/dbhome_1/cfgtoollogs/opatch/opatch2022-11-29_22-22-45PM_1.log

OPatch succeeded.
[oracle@dbsgururc01 34411846]$

5.3: Validate the applied patch in HOME.

[oracle@dbsgururc01 34411846]$ opatch lspatches
34411846;OJVM RELEASE UPDATE: 19.17.0.0.221018 (34411846)
34444834;OCW RELEASE UPDATE 19.17.0.0.0 (34444834)
34419443;Database Release Update : 19.17.0.0.221018 (34419443)

OPatch succeeded.
[oracle@dbsgururc01 logs_patch_Oct2022]$ $ORACLE_HOME/OPatch/opatch lsinventory -detail -oh $ORACLE_HOME > after_current_ora_lsinventory_Oct22_19cpatch.txt
[oracle@dbsgururc01 logs_patch_Oct2022]$ ls -lrt after_current_ora_lsinventory_Oct22_19cpatch.txt
-rw-r--r--. 1 oracle oinstall 1809138 Nov 30 16:52 after_current_ora_lsinventory_Oct22_19cpatch.txt
[oracle@dbsgururc01 logs_patch_Oct2022]$

5.4: Start instances.

[oracle@dbsgururc01 34411846]$ srvctl start instance -db dbsgucdb -i dbsgucdb1
[oracle@dbsgururc01 34411846]$ srvctl status database -db dbsgucdb
Instance dbsgucdb1 is running on node dbsgururc01
Instance dbsgucdb2 is running on node dbsgururc02
[oracle@dbsgururc01 34411846]$

5.5.: Execute utlrp.sql to make it VALID if any invalid objects after the applied patch.

SQL> SQL> @$ORACLE_HOME/rdbms/admin/utlrp.sql

Session altered.


TIMESTAMP
--------------------------------------------------------------------------------
COMP_TIMESTAMP UTLRP_BGN              2022-12-04 16:32:17

DOC>   The following PL/SQL block invokes UTL_RECOMP to recompile invalid
DOC>   objects in the database. Recompilation time is proportional to the
DOC>   number of invalid objects in the database, so this command may take
DOC>   a long time to execute on a database with a large number of invalid
DOC>   objects.
DOC>
DOC>   Use the following queries to track recompilation progress:
DOC>
DOC>   1. Query returning the number of invalid objects remaining. This
DOC>      number should decrease with time.
DOC>         SELECT COUNT(*) FROM obj$ WHERE status IN (4, 5, 6);
DOC>
DOC>   2. Query returning the number of objects compiled so far. This number
DOC>      should increase with time.
DOC>         SELECT COUNT(*) FROM UTL_RECOMP_COMPILED;
DOC>
DOC>   This script automatically chooses serial or parallel recompilation
DOC>   based on the number of CPUs available (parameter cpu_count) multiplied
DOC>   by the number of threads per CPU (parameter parallel_threads_per_cpu).
DOC>   On RAC, this number is added across all RAC nodes.
DOC>
DOC>   UTL_RECOMP uses DBMS_SCHEDULER to create jobs for parallel
DOC>   recompilation. Jobs are created without instance affinity so that they
DOC>   can migrate across RAC nodes. Use the following queries to verify
DOC>   whether UTL_RECOMP jobs are being created and run correctly:
DOC>
DOC>   1. Query showing jobs created by UTL_RECOMP
DOC>         SELECT job_name FROM dba_scheduler_jobs
DOC>            WHERE job_name like 'UTL_RECOMP_SLAVE_%';
DOC>
DOC>   2. Query showing UTL_RECOMP jobs that are running
DOC>         SELECT job_name FROM dba_scheduler_running_jobs
DOC>            WHERE job_name like 'UTL_RECOMP_SLAVE_%';
DOC>#

PL/SQL procedure successfully completed.


TIMESTAMP
--------------------------------------------------------------------------------
COMP_TIMESTAMP UTLRP_END              2022-12-04 16:32:20

DOC> The following query reports the number of invalid objects.
DOC>
DOC> If the number is higher than expected, please examine the error
DOC> messages reported with each object (using SHOW ERRORS) to see if they
DOC> point to system misconfiguration or resource constraints that must be
DOC> fixed before attempting to recompile these objects.
DOC>#

OBJECTS WITH ERRORS
-------------------
                  0

DOC> The following query reports the number of exceptions caught during
DOC> recompilation. If this number is non-zero, please query the error
DOC> messages in the table UTL_RECOMP_ERRORS to see if any of these errors
DOC> are due to misconfiguration or resource constraints that must be
DOC> fixed before objects can compile successfully.
DOC> Note: Typical compilation errors (due to coding errors) are not
DOC>       logged into this table: they go into DBA_ERRORS instead.
DOC>#

ERRORS DURING RECOMPILATION
---------------------------
                          0


Function created.


PL/SQL procedure successfully completed.


Function dropped.


PL/SQL procedure successfully completed.

SQL>

NODE02: dbsgururc02

Repeat the same set of steps from 5.1 through 5.5 on NODE02.

5.1:

[oracle@dbsgururc02 34411846]$ srvctl status database -db dbsgucdb
Instance dbsgucdb1 is running on node dbsgururc01
Instance dbsgucdb2 is running on node dbsgururc02
[oracle@dbsgururc02 34411846]$ srvctl stop instance -db dbsgucdb -i dbsgucdb2
[oracle@dbsgururc02 34411846]$ srvctl status database -db dbsgucdb
Instance dbsgucdb1 is running on node dbsgururc01
Instance dbsgucdb2 is not running on node dbsgururc02
[oracle@dbsgururc02 34411846]$

5.2:

Verifying environment and performing prerequisite checks...
OPatch continues with these patches: 34411846

Do you want to proceed? [y|n]
y
User Responded with: Y
All checks passed.

Please shutdown Oracle instances running out of this ORACLE_HOME on the local system.
(Oracle Home = '/u02/app/oracle/product/19c/dbhome_1')

Is the local system ready for patching? [y|n]
y
User Responded with: Y
Backing up files...
Applying interim patch '34411846' to OH '/u02/app/oracle/product/19c/dbhome_1'

Patching component oracle.javavm.server, 19.0.0.0.0...

Patching component oracle.javavm.server.core, 19.0.0.0.0...

Patching component oracle.rdbms.dbscripts, 19.0.0.0.0...

Patching component oracle.rdbms, 19.0.0.0.0...

OPatch succeeded.
[oracle@dbsgururc02 34411846]$

5.3:

[oracle@dbsgururc02 34411846]$ opatch lspatches
34411846;OJVM RELEASE UPDATE: 19.17.0.0.221018 (34411846)
34444834;OCW RELEASE UPDATE 19.17.0.0.0 (34444834)
34419443;Database Release Update : 19.17.0.0.221018 (34419443)

OPatch succeeded.
[oracle@dbsgururc02 34411846]$
[oracle@dbsgururc02 logs_patch_Oct2022]$ $ORACLE_HOME/OPatch/opatch lsinventory -detail -oh $ORACLE_HOME > after_current_ora_lsinventory_Oct22_19cpatch.txt
[oracle@dbsgururc02 logs_patch_Oct2022]$ ls -lrt after_current_ora_lsinventory_Oct22_19cpatch.txt
-rw-r--r--. 1 oracle oinstall 1809138 Nov 30 16:54 after_current_ora_lsinventory_Oct22_19cpatch.txt
[oracle@dbsgururc02 logs_patch_Oct2022]$

5.4:

[oracle@dbsgururc02 34411846]$ srvctl start instance -db dbsgucdb -i dbsgucdb2
[oracle@dbsgururc02 34411846]$ srvctl status database -db dbsgucdb
Instance dbsgucdb1 is running on node dbsgururc01
Instance dbsgucdb2 is running on node dbsgururc02
[oracle@dbsgururc02 34411846]$

5.5:

SQL> @$ORACLE_HOME/rdbms/admin/utlrp.sql

Session altered.


TIMESTAMP
--------------------------------------------------------------------------------
COMP_TIMESTAMP UTLRP_BGN              2022-12-04 16:32:59

DOC>   The following PL/SQL block invokes UTL_RECOMP to recompile invalid
DOC>   objects in the database. Recompilation time is proportional to the
DOC>   number of invalid objects in the database, so this command may take
DOC>   a long time to execute on a database with a large number of invalid
DOC>   objects.
DOC>
DOC>   Use the following queries to track recompilation progress:
DOC>
DOC>   1. Query returning the number of invalid objects remaining. This
DOC>      number should decrease with time.
DOC>         SELECT COUNT(*) FROM obj$ WHERE status IN (4, 5, 6);
DOC>
DOC>   2. Query returning the number of objects compiled so far. This number
DOC>      should increase with time.
DOC>         SELECT COUNT(*) FROM UTL_RECOMP_COMPILED;
DOC>
DOC>   This script automatically chooses serial or parallel recompilation
DOC>   based on the number of CPUs available (parameter cpu_count) multiplied
DOC>   by the number of threads per CPU (parameter parallel_threads_per_cpu).
DOC>   On RAC, this number is added across all RAC nodes.
DOC>
DOC>   UTL_RECOMP uses DBMS_SCHEDULER to create jobs for parallel
DOC>   recompilation. Jobs are created without instance affinity so that they
DOC>   can migrate across RAC nodes. Use the following queries to verify
DOC>   whether UTL_RECOMP jobs are being created and run correctly:
DOC>
DOC>   1. Query showing jobs created by UTL_RECOMP
DOC>         SELECT job_name FROM dba_scheduler_jobs
DOC>            WHERE job_name like 'UTL_RECOMP_SLAVE_%';
DOC>
DOC>   2. Query showing UTL_RECOMP jobs that are running
DOC>         SELECT job_name FROM dba_scheduler_running_jobs
DOC>            WHERE job_name like 'UTL_RECOMP_SLAVE_%';
DOC>#

PL/SQL procedure successfully completed.


TIMESTAMP
--------------------------------------------------------------------------------
COMP_TIMESTAMP UTLRP_END              2022-12-04 16:33:06

DOC> The following query reports the number of invalid objects.
DOC>
DOC> If the number is higher than expected, please examine the error
DOC> messages reported with each object (using SHOW ERRORS) to see if they
DOC> point to system misconfiguration or resource constraints that must be
DOC> fixed before attempting to recompile these objects.
DOC>#

OBJECTS WITH ERRORS
-------------------
                  0

DOC> The following query reports the number of exceptions caught during
DOC> recompilation. If this number is non-zero, please query the error
DOC> messages in the table UTL_RECOMP_ERRORS to see if any of these errors
DOC> are due to misconfiguration or resource constraints that must be
DOC> fixed before objects can compile successfully.
DOC> Note: Typical compilation errors (due to coding errors) are not
DOC>       logged into this table: they go into DBA_ERRORS instead.
DOC>#

ERRORS DURING RECOMPILATION
---------------------------
                          0


Function created.


PL/SQL procedure successfully completed.


Function dropped.


PL/SQL procedure successfully completed.

SQL>

6. Post Patch Apply

Apply datapatch only on any one node’s instance in the RAC environment, applicable to all running instances of patched ORACLE home.

6.1: Login to an individual instance and open all pluggable databases, applicable to only Multitenant architecture.

[oracle@dbsgururc02 34411846]$ sqlplus / as sysdba

SQL*Plus: Release 19.0.0.0.0 - Production on Tue Nov 29 22:39:02 2022
Version 19.17.0.0.0

Copyright (c) 1982, 2022, Oracle.  All rights reserved.


Connected to:
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.17.0.0.0

SQL> alter pluggable database all open;

Pluggable database altered.

SQL> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       READ ONLY  NO
         3 DBS_PDB1                       READ WRITE NO
SQL> exit
Disconnected from Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.17.0.0.0

6.2: Apply patch (datapatch) to the database.

[oracle@dbsgururc02 34411846]$  cd $ORACLE_HOME/OPatch
[oracle@dbsgururc02 OPatch]$ ./datapatch -verbose
SQL Patching tool version 19.17.0.0.0 Production on Tue Nov 29 22:40:07 2022
Copyright (c) 2012, 2022, Oracle.  All rights reserved.

Log file for this invocation: /u02/app/oracle/cfgtoollogs/sqlpatch/sqlpatch_30670_2022_11_29_22_40_07/sqlpatch_invocation.log

Connecting to database...OK
Gathering database info...done

Note:  Datapatch will only apply or rollback SQL fixes for PDBs
       that are in an open state, no patches will be applied to closed PDBs.
       Please refer to Note: Datapatch: Database 12c Post Patch SQL Automation
       (Doc ID 1585822.1)

Bootstrapping registry and package to current versions...done
Determining current state...done

Current state of interim SQL patches:
Interim patch 34086870 (OJVM RELEASE UPDATE: 19.16.0.0.220719 (34086870)):
  Binary registry: Not installed
  PDB CDB$ROOT: Applied successfully on 30-JUL-22 04.58.11.063879 PM
  PDB DBS_PDB1: Applied successfully on 30-JUL-22 04.58.16.838841 PM
  PDB PDB$SEED: Applied successfully on 30-JUL-22 04.58.11.387029 PM
Interim patch 34411846 (OJVM RELEASE UPDATE: 19.17.0.0.221018 (34411846)):
  Binary registry: Installed
  PDB CDB$ROOT: Not installed
  PDB DBS_PDB1: Not installed
  PDB PDB$SEED: Not installed

Current state of release update SQL patches:
  Binary registry:
    19.17.0.0.0 Release_Update 220924224051: Installed
  PDB CDB$ROOT:
    Applied 19.17.0.0.0 Release_Update 220924224051 successfully on 29-NOV-22 09.46.00.756194 PM
  PDB DBS_PDB1:
    Applied 19.16.0.0.0 Release_Update 220703022223 successfully on 30-JUL-22 04.58.16.812606 PM
  PDB PDB$SEED:
    Applied 19.17.0.0.0 Release_Update 220924224051 successfully on 29-NOV-22 09.46.01.308823 PM

Adding patches to installation queue and performing prereq checks...done
Installation queue:
  For the following PDBs: CDB$ROOT PDB$SEED
    The following interim patches will be rolled back:
      34086870 (OJVM RELEASE UPDATE: 19.16.0.0.220719 (34086870))
    No release update patches need to be installed
    The following interim patches will be applied:
      34411846 (OJVM RELEASE UPDATE: 19.17.0.0.221018 (34411846))
  For the following PDBs: DBS_PDB1
    The following interim patches will be rolled back:
      34086870 (OJVM RELEASE UPDATE: 19.16.0.0.220719 (34086870))
    Patch 34419443 (Database Release Update : 19.17.0.0.221018 (34419443)):
      Apply from 19.16.0.0.0 Release_Update 220703022223 to 19.17.0.0.0 Release_Update 220924224051
    The following interim patches will be applied:
      34411846 (OJVM RELEASE UPDATE: 19.17.0.0.221018 (34411846))

Installing patches...
Patch installation complete.  Total patches installed: 7

Validating logfiles...done
Patch 34086870 rollback (pdb CDB$ROOT): SUCCESS
  logfile: /u02/app/oracle/cfgtoollogs/sqlpatch/34086870/24803071/34086870_rollback_DBSGUCDB_CDBROOT_2022Nov29_22_42_01.log (no errors)
Patch 34411846 apply (pdb CDB$ROOT): SUCCESS
  logfile: /u02/app/oracle/cfgtoollogs/sqlpatch/34411846/24917919/34411846_apply_DBSGUCDB_CDBROOT_2022Nov29_22_42_10.log (no errors)
Patch 34086870 rollback (pdb PDB$SEED): SUCCESS
  logfile: /u02/app/oracle/cfgtoollogs/sqlpatch/34086870/24803071/34086870_rollback_DBSGUCDB_PDBSEED_2022Nov29_22_42_16.log (no errors)
Patch 34411846 apply (pdb PDB$SEED): SUCCESS
  logfile: /u02/app/oracle/cfgtoollogs/sqlpatch/34411846/24917919/34411846_apply_DBSGUCDB_PDBSEED_2022Nov29_22_42_51.log (no errors)
Patch 34086870 rollback (pdb DBS_PDB1): SUCCESS
  logfile: /u02/app/oracle/cfgtoollogs/sqlpatch/34086870/24803071/34086870_rollback_DBSGUCDB_DBS_PDB1_2022Nov29_22_43_05.log (no errors)
Patch 34419443 apply (pdb DBS_PDB1): SUCCESS
  logfile: /u02/app/oracle/cfgtoollogs/sqlpatch/34419443/24972075/34419443_apply_DBSGUCDB_DBS_PDB1_2022Nov29_22_43_10.log (no errors)
Patch 34411846 apply (pdb DBS_PDB1): SUCCESS
  logfile: /u02/app/oracle/cfgtoollogs/sqlpatch/34411846/24917919/34411846_apply_DBSGUCDB_DBS_PDB1_2022Nov29_22_43_10.log (no errors)
SQL Patching tool complete on Tue Nov 29 22:48:37 2022
[oracle@dbsgururc02 OPatch]$

Apply datapatch only on any one node’s instance in the RAC environment, applicable to all running instances of patched ORACLE home.

6.3: Validate applied patch in databases. Here we performed in a multitenant database (CDB, PDBs).

[oracle@dbsgururc01 34411846]$ sqlplus / as sysdba

SQL*Plus: Release 19.0.0.0.0 - Production on Tue Nov 29 23:12:24 2022
Version 19.17.0.0.0

Copyright (c) 1982, 2022, Oracle.  All rights reserved.


Connected to:
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.17.0.0.0

SQL> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       READ ONLY  NO
         3 DBS_PDB1                       READ WRITE NO
SQL> SET PAGES 55
SET LINESIZE 601
COLUMN ACTION_TIME FORMAT A21
COLUMN ACTION FORMAT A11
COLUMN STATUS FORMAT A11
COLUMN DESCRIPTION FORMAT A55
COLUMN VERSION FORMAT A11
COLUMN BUNDLE_SERIES FORMAT A11

SELECT CON_ID,TO_CHAR(ACTION_TIME, 'DD-MON-YYYY HH24:MI:SS') AS ACTION_TIME, PATCH_TYPE,
ACTION,STATUS,DESCRIPTION, SOURCE_VERSION,TARGET_VERSION, PATCH_ID FROM SYS.CDB_REGISTRY_SQLPATCH ORDER BY CON_ID,ACTION_TIME DESC;
prompt$$$$$$$**Welcome to DBsGuru!**Share Learn Grow**$$$$$$$SQL> SQL> SQL> SQL> SQL> SQL> SQL> SQL> SQL>   2
    CON_ID ACTION_TIME           PATCH_TYPE ACTION      STATUS      DESCRIPTION                                             SOURCE_VERSION  TARGET_VERSION    PATCH_ID
---------- --------------------- ---------- ----------- ----------- ------------------------------------------------------- --------------- --------------- ----------
         1 30-JUL-2022 16:58:11  INTERIM    APPLY       SUCCESS     OJVM RELEASE UPDATE: 19.16.0.0.220719 (34086870)        19.16.0.0.0     19.16.0.0.0       34086870
         1 29-NOV-2022 22:48:16  INTERIM    ROLLBACK    SUCCESS     OJVM RELEASE UPDATE: 19.16.0.0.220719 (34086870)        19.17.0.0.0     19.17.0.0.0       34086870
         1 29-NOV-2022 22:48:16  INTERIM    APPLY       SUCCESS     OJVM RELEASE UPDATE: 19.17.0.0.221018 (34411846)        19.17.0.0.0     19.17.0.0.0       34411846
         1 29-NOV-2022 21:46:00  RU         APPLY       SUCCESS     Database Release Update : 19.17.0.0.221018 (34419443)   19.16.0.0.0     19.17.0.0.0       34419443
         1 29-JUL-2022 13:25:37  RU         APPLY       SUCCESS     Database Release Update : 19.16.0.0.220719 (34133642)   19.3.0.0.0      19.16.0.0.0       34133642
         1 28-JUL-2022 10:58:04  RU         APPLY       SUCCESS     Database Release Update : 19.3.0.0.190416 (29517242)    19.1.0.0.0      19.3.0.0.0        29517242
         3 30-JUL-2022 16:58:16  INTERIM    APPLY       SUCCESS     OJVM RELEASE UPDATE: 19.16.0.0.220719 (34086870)        19.3.0.0.0      19.3.0.0.0        34086870
         3 30-JUL-2022 16:58:16  RU         APPLY       SUCCESS     Database Release Update : 19.16.0.0.220719 (34133642)   19.3.0.0.0      19.16.0.0.0       34133642
         3 29-NOV-2022 22:48:18  INTERIM    APPLY       SUCCESS     OJVM RELEASE UPDATE: 19.17.0.0.221018 (34411846)        19.16.0.0.0     19.16.0.0.0       34411846
         3 29-NOV-2022 22:48:18  RU         APPLY       SUCCESS     Database Release Update : 19.17.0.0.221018 (34419443)   19.16.0.0.0     19.17.0.0.0       34419443
         3 29-NOV-2022 22:48:16  INTERIM    ROLLBACK    SUCCESS     OJVM RELEASE UPDATE: 19.16.0.0.220719 (34086870)        19.17.0.0.0     19.17.0.0.0       34086870
         3 28-JUL-2022 11:11:46  RU         APPLY       SUCCESS     Database Release Update : 19.3.0.0.190416 (29517242)    19.1.0.0.0      19.3.0.0.0        29517242

12 rows selected.

SQL>
$$$$$$$**Welcome to DBsGuru!**Share Learn Grow**$$$$$$$

Follow the same steps 6.1 through 6.3 to all instances/databases running from the same patched home.

Click here for various options to validate applied patches in ORACLE HOME & DATABASE.

7.: Validate the patch level & other components on each NODE and make sure after the patch is applied patch level should be the same across all NODES along with all services that should be started of patched home.

NODE01: dbsgururc01

[grid@dbsgururc01 logs_patch_Oct2022]$ crsctl query crs activeversion -f
Oracle Clusterware active version on the cluster is [19.0.0.0.0]. The cluster upgrade state is [NORMAL]. The cluster active patch level is [3007461856].
[grid@dbsgururc01 logs_patch_Oct2022]$ crsctl query crs softwareversion
Oracle Clusterware version on node [dbsgururc01] is [19.0.0.0.0]
[grid@dbsgururc01 logs_patch_Oct2022]$ crsctl query crs releasepatch
Oracle Clusterware release patch level is [3007461856] and the complete list of patches [33575402 34419443 34428761 34444834 34580338 ] have been applied on the local node. The release patch string is [19.17.0.0.0].
[grid@dbsgururc01 logs_patch_Oct2022]$ crsctl query crs softwarepatch
Oracle Clusterware patch level on node dbsgururc01 is [3007461856].
[grid@dbsgururc01 logs_patch_Oct2022]$ asmcmd -V
asmcmd version 19.17.0.0.0
[grid@dbsgururc01 logs_patch_Oct2022]$ crsctl stat res -t
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.LISTENER.lsnr
               ONLINE  ONLINE       dbsgururc01              STABLE
               ONLINE  ONLINE       dbsgururc02              STABLE
ora.chad
               ONLINE  ONLINE       dbsgururc01              STABLE
               ONLINE  ONLINE       dbsgururc02              STABLE
ora.net1.network
               ONLINE  ONLINE       dbsgururc01              STABLE
               ONLINE  ONLINE       dbsgururc02              STABLE
ora.ons
               ONLINE  ONLINE       dbsgururc01              STABLE
               ONLINE  ONLINE       dbsgururc02              STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.ASMNET1LSNR_ASM.lsnr(ora.asmgroup)
      1        ONLINE  ONLINE       dbsgururc01              STABLE
      2        ONLINE  ONLINE       dbsgururc02              STABLE
      3        ONLINE  OFFLINE                               STABLE
ora.DATA1.dg(ora.asmgroup)
      1        ONLINE  ONLINE       dbsgururc01              STABLE
      2        ONLINE  ONLINE       dbsgururc02              STABLE
      3        OFFLINE OFFLINE                               STABLE
ora.FRA1.dg(ora.asmgroup)
      1        ONLINE  ONLINE       dbsgururc01              STABLE
      2        ONLINE  ONLINE       dbsgururc02              STABLE
      3        OFFLINE OFFLINE                               STABLE
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       dbsgururc02              STABLE
ora.LISTENER_SCAN2.lsnr
      1        ONLINE  ONLINE       dbsgururc01              STABLE
ora.LISTENER_SCAN3.lsnr
      1        ONLINE  ONLINE       dbsgururc01              STABLE
ora.MGMTLSNR
      1        ONLINE  ONLINE       dbsgururc01              169.254.2.113 192.16
                                                             8.10.100,STABLE
ora.VOTE1.dg(ora.asmgroup)
      1        ONLINE  ONLINE       dbsgururc01              STABLE
      2        ONLINE  ONLINE       dbsgururc02              STABLE
      3        OFFLINE OFFLINE                               STABLE
ora.asm(ora.asmgroup)
      1        ONLINE  ONLINE       dbsgururc01              Started,STABLE
      2        ONLINE  ONLINE       dbsgururc02              Started,STABLE
      3        ONLINE  OFFLINE                               STABLE
ora.asmnet1.asmnetwork(ora.asmgroup)
      1        ONLINE  ONLINE       dbsgururc01              STABLE
      2        ONLINE  ONLINE       dbsgururc02              STABLE
      3        ONLINE  OFFLINE                               STABLE
ora.cvu
      1        ONLINE  ONLINE       dbsgururc01              STABLE
ora.dbsgucdb.db
      1        ONLINE  ONLINE       dbsgururc01              Open,HOME=/u02/app/o
                                                             racle/product/19c/db
                                                             home_1,STABLE
      2        ONLINE  ONLINE       dbsgururc02              Open,HOME=/u02/app/o
                                                             racle/product/19c/db
                                                             home_1,STABLE
ora.dbsgururc01.vip
      1        ONLINE  ONLINE       dbsgururc01              STABLE
ora.dbsgururc02.vip
      1        ONLINE  ONLINE       dbsgururc02              STABLE
ora.mgmtdb
      1        ONLINE  ONLINE       dbsgururc01              Open,STABLE
ora.qosmserver
      1        ONLINE  ONLINE       dbsgururc01              STABLE
ora.scan1.vip
      1        ONLINE  ONLINE       dbsgururc02              STABLE
ora.scan2.vip
      1        ONLINE  ONLINE       dbsgururc01              STABLE
ora.scan3.vip
      1        ONLINE  ONLINE       dbsgururc01              STABLE
--------------------------------------------------------------------------------
[grid@dbsgururc01 logs_patch_Oct2022]$

NODE02: dbsgururc02

[grid@dbsgururc02 logs_patch_Oct2022]$ crsctl query crs activeversion -f
Oracle Clusterware active version on the cluster is [19.0.0.0.0]. The cluster upgrade state is [NORMAL]. The cluster active patch level is [3007461856].
[grid@dbsgururc02 logs_patch_Oct2022]$ crsctl query crs softwareversion
Oracle Clusterware version on node [dbsgururc02] is [19.0.0.0.0]
[grid@dbsgururc02 logs_patch_Oct2022]$ crsctl query crs releasepatch
Oracle Clusterware release patch level is [3007461856] and the complete list of patches [33575402 34419443 34428761 34444834 34580338 ] have been applied on the local node. The release patch string is [19.17.0.0.0].
[grid@dbsgururc02 logs_patch_Oct2022]$ crsctl query crs softwarepatch
Oracle Clusterware patch level on node dbsgururc02 is [3007461856].
[grid@dbsgururc02 logs_patch_Oct2022]$ asmcmd -V
asmcmd version 19.17.0.0.0
[grid@dbsgururc02 logs_patch_Oct2022]$ crsctl stat res -t
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.LISTENER.lsnr
               ONLINE  ONLINE       dbsgururc01              STABLE
               ONLINE  ONLINE       dbsgururc02              STABLE
ora.chad
               ONLINE  ONLINE       dbsgururc01              STABLE
               ONLINE  ONLINE       dbsgururc02              STABLE
ora.net1.network
               ONLINE  ONLINE       dbsgururc01              STABLE
               ONLINE  ONLINE       dbsgururc02              STABLE
ora.ons
               ONLINE  ONLINE       dbsgururc01              STABLE
               ONLINE  ONLINE       dbsgururc02              STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.ASMNET1LSNR_ASM.lsnr(ora.asmgroup)
      1        ONLINE  ONLINE       dbsgururc01              STABLE
      2        ONLINE  ONLINE       dbsgururc02              STABLE
      3        ONLINE  OFFLINE                               STABLE
ora.DATA1.dg(ora.asmgroup)
      1        ONLINE  ONLINE       dbsgururc01              STABLE
      2        ONLINE  ONLINE       dbsgururc02              STABLE
      3        OFFLINE OFFLINE                               STABLE
ora.FRA1.dg(ora.asmgroup)
      1        ONLINE  ONLINE       dbsgururc01              STABLE
      2        ONLINE  ONLINE       dbsgururc02              STABLE
      3        OFFLINE OFFLINE                               STABLE
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       dbsgururc02              STABLE
ora.LISTENER_SCAN2.lsnr
      1        ONLINE  ONLINE       dbsgururc01              STABLE
ora.LISTENER_SCAN3.lsnr
      1        ONLINE  ONLINE       dbsgururc01              STABLE
ora.MGMTLSNR
      1        ONLINE  ONLINE       dbsgururc01              169.254.2.113 192.16
                                                             8.10.100,STABLE
ora.VOTE1.dg(ora.asmgroup)
      1        ONLINE  ONLINE       dbsgururc01              STABLE
      2        ONLINE  ONLINE       dbsgururc02              STABLE
      3        OFFLINE OFFLINE                               STABLE
ora.asm(ora.asmgroup)
      1        ONLINE  ONLINE       dbsgururc01              Started,STABLE
      2        ONLINE  ONLINE       dbsgururc02              Started,STABLE
      3        ONLINE  OFFLINE                               STABLE
ora.asmnet1.asmnetwork(ora.asmgroup)
      1        ONLINE  ONLINE       dbsgururc01              STABLE
      2        ONLINE  ONLINE       dbsgururc02              STABLE
      3        ONLINE  OFFLINE                               STABLE
ora.cvu
      1        ONLINE  ONLINE       dbsgururc01              STABLE
ora.dbsgucdb.db
      1        ONLINE  ONLINE       dbsgururc01              Open,HOME=/u02/app/o
                                                             racle/product/19c/db
                                                             home_1,STABLE
      2        ONLINE  ONLINE       dbsgururc02              Open,HOME=/u02/app/o
                                                             racle/product/19c/db
                                                             home_1,STABLE
ora.dbsgururc01.vip
      1        ONLINE  ONLINE       dbsgururc01              STABLE
ora.dbsgururc02.vip
      1        ONLINE  ONLINE       dbsgururc02              STABLE
ora.mgmtdb
      1        ONLINE  ONLINE       dbsgururc01              Open,STABLE
ora.qosmserver
      1        ONLINE  ONLINE       dbsgururc01              STABLE
ora.scan1.vip
      1        ONLINE  ONLINE       dbsgururc02              STABLE
ora.scan2.vip
      1        ONLINE  ONLINE       dbsgururc01              STABLE
ora.scan3.vip
      1        ONLINE  ONLINE       dbsgururc01              STABLE
--------------------------------------------------------------------------------
[grid@dbsgururc02 logs_patch_Oct2022]$

7.: Delete blackout in OEM, Enable jobs in crontab & DBMS scheduler, etc. if applicable. Click here to know more about Target Blackouts enable-disable in OEM.

8.: Handover to application checkout to start dependent applications/jobs, if applicable.

Click here to get Oracle-related patching articles.

This document is only for learning purpose and always validate in the LAB environment first before applying in the LIVE environment.

Hope so you like this article!
Please share your valuable feedback/comments/subscribe and follow us below and don’t forget to click on the bell icon to get the most recent update. Click here to know more about our pursuit.

ASM, COMBO PATCH, DATABASE, INSTALL, OJVM, ORACLE, ORACLE 19C, PATCHING, RAC, UPGRADE