Pivotal Knowledge Base

Follow

icm deployment hung with no error in GPHDNodeInstaller log

Environment

PHD 1.0.1

Symptom

When you do icm_client dpeloy or icm_client uninstall to install or uninstall a PHD cluster, you may see one or more cluster nodes hung and icm_client timed out eventually. From /tmp/GPHDNodeInstaller.log on a problematic cluster node, you may find the following lines:

[INFO] Cleaning up old status files [INFO] Checking yum repo file /etc/puppet/puppet.conf.icmbackup.1392926025 [INFO] Cleaning up old status files [INFO] Checking yum repo file /etc/puppet/puppet.conf.icmbackup.1392926027 [INFO] Cleaning up old status files [INFO] Checking yum repo file /etc/puppet/puppet.conf.icmbackup.1392926029
[INFO] Cleaning up old status files
[INFO] Checking yum repo file
/etc/puppet/puppet.conf.icmbackup.1392926025
[INFO] Cleaning up old status files
[INFO] Checking yum repo file
/etc/puppet/puppet.conf.icmbackup.1392926027
[INFO] Cleaning up old status files
[INFO] Checking yum repo file
/etc/puppet/puppet.conf.icmbackup.1392926029
There were no other type of log messages or error in this log.
There were no errors indicate this problem in /var/log/gphd/gphdmgrgphdmgr-webservices.log on admin node.
icm_client scanhosts command showed all node are good.
Multiple files like puppet.conf.icmbackup.1392926025 were created in /etc/puppet.
 
Verify 
When running "puppet cert list" from failed cluster node you will see this error:
Got 2 failure(s) while initializing: change from absent to directory failed: Could not set 'directory on ensure: Could not find group puppet; Could not evaluate: Could not find group puppet
Cause
User account  and group puppet doesn't exist on the cluster node.
 
fix
To fix manually use command useradd and groupadd to add the puppet user and group on the problematic cluster node. It's recommended that you grep the puppet from /etc/passwd and /etc/group on a good node, then use the same  info such as home dir etc on the /etc/passwd and /etc/group on the problematic node. Keep the UID ang GID that was creatd by useradd and groupadd  commands in those files.
 
 

Comments

Powered by Zendesk