Pivotal Knowledge Base

Follow

dca_failover manual command fails

Environment

Product Version
DCA (GPDB) 1.2.x , 2.1.x
OS RHEL 6.x
Others  

Symptom

Manual dca_failover command to failover to standby master on DCA fails when there are issues with gpstop.

Error Message:

Note: These log entries use a volume named 12345678-abcdefg as an example:

[root@smdw ~]# dca_failover --stopmasterdb --noninteractive --vip 10.39.236.110 --gateway 10.39.236.127 --netmask 255.255.255.224
Using /data/master/gpseg-1/ as master data directory from healthmon config
Using 5432 as db port from healthmon config
Stopping the master database on the remote machine.

20160627:12:32:18:531035 gpactivatestandby:smdw:gpadmin-[WARNING]:-Appears that there is an active postgres process on mdw.gphd.local port=5432
20160627:12:32:18:531035 gpactivatestandby:smdw:gpadmin-[WARNING]:-Need to stop current master before activating the standby.
20160627:12:32:18:531035 gpactivatestandby:smdw:gpadmin-[WARNING]:-Rare possibility is just /tmp/.s.PGSQL.5432 and /tmp/.s.PGSQL.5432.* files exist on mdw.gphd.local.
20160627:12:32:18:531035 gpactivatestandby:smdw:gpadmin-[WARNING]:-Need to delete these files
20160627:12:32:18:531035 gpactivatestandby:smdw:gpadmin-[WARNING]:-Then call this utility again.
20160627:12:32:18:531035 gpactivatestandby:smdw:gpadmin-[CRITICAL]:-Error activating standby master: Active postgres process on master
20160627:12:32:18:530891 dca_failover_main.py:smdw:root-[ERROR]:-Unsuccessful in activating standby
Unsuccessful in activating standby
20160627:12:32:18:530891 dca_failover_main.py:smdw:root-[ERROR]:-Could not activate the standby

Resolution

dca_failover fails when there is an issue with stopping the master or if the gpstop takes longer. Hence the workaround is to manually stop the master using : gpstop -am

Then run the dca_failover without using the stop master option of "--stopmasterdb":

dca_failover --noninteractive --vip 10.39.236.110 --gateway 10.39.236.127 --netmask 255.255.255.224

The failover completed successfully. Stop and restart the database to make sure VIP is running on the right master host.

Comments

Powered by Zendesk