Pivotal Knowledge Base

Follow

gpinitstandby Fails with Error "pg_basebackup: could not read COPY data: server closed the connection unexpectedly" due to Lack of Disk Space

Environment

 Product  Version
 Pivotal Greenplum  4.2.x - 4.3.x

Symptom

During the standby initialisation, you encounter the following error message:

gpinitstandby:mdw:gpadmin-[ERROR]:-Error initializing standby master: ExecutionError: 'non-zero rc: 1' occured. Details: 'ssh -o 'StrictHostKeyChecking no' smdw ". /usr/local/greenplum-db/./greenplum_path.sh; pg_basebackup -x -R -E ./pg_log -E ./gp_dumps -E ./gpperfmon/data -D /data/master/gpseg-1 -h mdw -p 5432"' cmd had rc=1 completed=True halted=False
stdout=''
stderr='pg_basebackup -x -R -E ./pg_log -E ./gp_dumps -E ./gpperfmon/data -D /data/master/gpseg-1 -h mdw -p 5432
pg_basebackup: could not read COPY data: server closed the connection unexpectedly

This probably means that the server terminated abnormally before or while processing the request.

Cause

It was observed that the master data directory ($MASTER_DATA_DIRECTORY) on the master host was of the size greater than the size of the disk on the standby master (the directory on which the copy attempt is made), thus causing the copy to fail.

Solution

  • In this instance, there was a big script file that was occupying the space. Moving it out of the master data directory and attempting to re-initialise the standby, helped to overcome the problem. So, move any unrelated database files out of the master data directory and into the master host.
  • Ensure there is enough disk space (i.e., greater than the size of the master data directory on the master host).

Comments

  • Avatar
    Michael Mulcahy

    Another possibility for this issue is that there may be a file owned by the root user in the MASTER_DATA_DIRECTORY on mdw. Moving this file to outside MASTER_DATA_DIRECTORY will help resolve the issue.

  • Avatar
    Faisal Ali

    Hi Michael,

    The issue is already covered on a separate article "https://discuss.zendesk.com/hc/en-us/articles/206441237"

    Thanks

Powered by Zendesk