Startup RAC database using SRVCTL fails with : ORA-12547: TNS:lost contact & CRS-2632

Two days before while starting our UAT database using SRVCTL, I received following error:


[oracle@srv-uat-db-gen1 ~] Ora:BIIBDPRD $ srvctl start database -d BIIBDPRD
PRCR-1079 : Failed to start resource ora.biibdprd.db
ORA-12547: TNS:lost contact
CRS-5017: The resource action "ora.biibdprd.db start" encountered the following error:
ORA-12547: TNS:lost contact
. For details refer to "(:CLSN00107:)" in "/u01/app/11.2.0.3/grid/log/srv-uat-db-gen2/agent/crsd/oraagent_oracle/oraagent_oracle.log".
CRS-2674: Start of 'ora.biibdprd.db' on 'srv-uat-db-gen2' failed
ORA-12547: TNS:lost contact
CRS-2632: There are no more servers to try to place resource 'ora.biibdprd.db' on that would satisfy its placement policy
CRS-2674: Start of 'ora.biibdprd.db' on 'srv-uat-db-gen1' failed
CRS-2680: Clean of 'ora.biibdprd.db' on 'srv-uat-db-gen1' failed
CRS-5804: Communication error with agent process

I tried connecting instance on node 2 through sqlplus, I received the same error:


[oracle@srv-uat-db-gen2 ~] Ora:BIIBDPRD $ sqlplus "/ as sysdba"
SQL*Plus: Release 11.2.0.3.0 Production on Thu Jun 18 17:09:23 2015
Copyright (c) 1982, 2011, Oracle. All rights reserved.
ERROR:
ORA-12547: TNS:lost contact

After searching on metalink for possible causes of ORA-12547: TNS:Lost Contact, I found following note which explains various scenarios for ORA-12547.                                                                                                                               Local SQL*Plus Connection and DBCA Fails With: ORA-12547: TNS:Lost Contact (Doc ID 422173.1)

One of those scenarios : Incorrect permissions on the ORACLE executable was matching in my case as on node 2 : oracle executable was owned by root instead of oracle.

After correcting permissions on oracle executable my issue resolved!!


root@srv-uat-db-gen2:/u01/app/oracle/BIIBDPRD/11.2.0.3/bin# chown oracle:oinstall oracle
root@srv-uat-db-gen2:/u01/app/oracle/BIIBDPRD/11.2.0.3/bin# ls -lrt oracle
-rwsr-s--x 1 oracle oinstall 250373280 Jun 18 17:23 oracle
[oracle@srv-uat-db-gen2 bin] Ora:BIIBDPRD $ sss
SQL*Plus: Release 11.2.0.3.0 Production on Thu Jun 18 17:25:23 2015
Copyright (c) 1982, 2011, Oracle. All rights reserved.
Connected to an idle instance.
SQL> Disconnected

This time I was able to start my database using SRVCTL without any issues 🙂

Hope so u will find this post very useful 🙂

Cheers

Regards,

Adityanath

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s