Hope you are doing well & safe.
Today I will be writing on one of the issue I faced today while applying “GI Release Update 22.214.171.124.201020″(31750108) patch on GI Home of one of servers.
I used opatchauto utility to apply this patch. Following command was run for the same:
$<GI_HOME>/OPatch/opatchauto apply <PATCH DIRECTORY> -oh <GI_HOME>
After waiting for the sometime, I found my command was stuck without any progress. It was not moving further after printing below status information:
OPatchauto session is initiated at Sat Dec 12 15:23:18 2020 System initialization log file is /u01/app/126.96.36.199/grid/cfgtoollogs/opatchautodb/systemconfig2020-12-12_03-23-54PM.log. Session log file is /u01/app/188.8.131.52/grid/cfgtoollogs/opatchauto/opatchauto2020-12-12_03-24-19PM.log The id for this session is JKL8 Executing OPatch prereq operations to verify patch applicability on home /u01/app/184.108.40.206/grid Patch applicability verified successfully on home /u01/app/220.127.116.11/grid Bringing down CRS service on home /u01/app/18.104.22.168/grid Prepatch operation log file location: /u01/app/oracle/crsdata/test-server-1/crsconfig/hapatch_2020-12-12_03-26-03PM.log CRS service brought down successfully on home /u01/app/22.214.171.124/grid Start applying binary patch on home /u01/app/126.96.36.199/grid Successfully executed command: /usr/sbin/slibclean
I checked lot many relevant logfiles associated with this opatchauto session, I found none of were updated with proper information what the session is currently doing.
So after traversing in between multiple directories & files I found following logfile which has information on what the session is waiting for:
Logfile had repeated warnings as given below:
Do you want to proceed? [y|n] [Dec 12, 2020 3:28:55 PM] [INFO] Y (auto-answered by -silent) [Dec 12, 2020 3:28:55 PM] [INFO] User Responded with: Y [Dec 12, 2020 3:28:55 PM] [INFO] OPatch will sleep for few seconds, before re-trying to get the lock… [Dec 12, 2020 3:30:55 PM] [INFO] Unable to lock Central Inventory. OPatch will attempt to re-lock. Do you want to proceed? [y|n] [Dec 12, 2020 3:30:55 PM] [INFO] Y (auto-answered by -silent) [Dec 12, 2020 3:30:55 PM] [INFO] User Responded with: Y [Dec 12, 2020 3:30:55 PM] [INFO] OPatch will sleep for few seconds, before re-trying to get the lock… [Dec 12, 2020 3:32:55 PM] [INFO] Unable to lock Central Inventory. OPatch will attempt to re-lock.
I found the culprit. Patching session was unable to lock central inventory. After checking I found locks directory had incorrect ownership. After changing its ownership back to Oracle user, patching session moved further.
Following commands were used to fix the issue:
As a root user: cd <oraInventory location> chown -R oracle:oinstall locks
Patching was successfully completed in 35 Minutes.
Hope u will find this post very useful.