Pivotal Knowledge Base

Follow

GPCRONDUMP Error "Lost Response from Dump Agent..."

Environment 

 Product  Version
 Pivotal Greenplum   4.3.x
 OS  RHEL 6.x

Overview

GPCRONDUMP fails when using DDBoost with the error:

Lost response from dump agent with dbid XXX on host sdwXXX after 10 minutes. 

This error is repeated for all segments on (1) or more hosts. 

Cause 

GPCRONDUMP uses an implementation of RSA lockbox to store the username, password, and configuration for the Data Domain Boost. The hash to decrypt the lockbox includes information about the hardware.

If the lockbox file is unavailable or is otherwise unreadable due to a change of hardware on the host, segments on that host will not be able to connect to Data Domain and the operation will fail.

The connectivity and lockbox can be verified on the master and the segments using the following command:

gpddboost --verify

Hosts that are unable to open or read the lockbox file may present an error similar to the following:

[gpadmin@sdwXX ~]$ gpddboost --verify 
20161101:15:07:50|ddboost-[DEBUG]:-Libraries were loaded successfully 
20161101:15:07:50|ddboost-[INFO]:-opening LB on /home/gpadmin/DDBOOST_CONFIG 
20161101:15:07:51|ddboost-[ERROR]:-The lockbox file was not found. 
20161101:15:07:51|ddboost-[ERROR]:-Parsing DDBoost login credentials failed 
20161101:15:07:51|gpddboost-[ERROR]:-Could not connect to DD_host with DD_user and the DD_password. 

Resolution

You must have the username and password required to connect to the Data Domain controller in order to rebuild and redistribute the configuration from the Master.

Use the following command:

gpcrondump --ddboost-host <hostname> --ddboost-user <ddboost user> --ddboost-backupdir <directory>

You will be prompted to provide the password and once authenticated, the lockbox configuration is redistributed to all the hosts in the cluster.

 

 

Comments

  • Avatar
    Shawn Yan

    Sometimes when this issue happens, the "gpddboost --verify" command would only hang instead of error out. In that situation, we can still apply the same workaround (re-create ddboost credentials ).

Powered by Zendesk