Pivotal Knowledge Base

Follow

gpcrondump failed with error message "'ssh_exchange_identification: read: Connection reset by peer"

Environment

Product Version
Pivotal Greenplum (GPDB) 4.3.x
OS RHEL 6.x
Others  

Symptom

gpcrondump failed with the following error message:

20160701:14:34:01:055166 gpcrondump:linux:dbuser-[ERROR]:-ExecutionError: 'non-zero rc: 255' occured. Details: 'ssh -o 'StrictHostKeyChecking no' localhost ". /opt/ossdata/greenplum-db/./greenplumpath.sh; $GPHOME/sbin/gpoperation.py"'
cmd had rc=255 completed=True halted=False
stdout=''
stderr='ssh_exchange_identification: read: Connection reset by peer
'
Traceback (most recent call last):
File "/opt/ossdata/greenplum-db/lib/python/gppylib/commands/base.py", line 211, in run
self.cmd.run()
File "/opt/ossdata/greenplum-db/lib/python/gppylib/operations/
init_.py", line 49, in run
self.ret = self.execute()
File "/opt/ossdata/greenplum-db/lib/python/gppylib/operations/utils.py", line 48, in execute
cmd.run(validateAfter=True)
File "/opt/ossdata/greenplum-db/lib/python/gppylib/commands/base.py", line 702, in run
self.validate()
File "/opt/ossdata/greenplum-db/lib/python/gppylib/commands/base.py", line 737, in validate
raise ExecutionError("non-zero rc: %d" % self.results.rc, self)
ExecutionError: ExecutionError: 'non-zero rc: 255' occured. Details: 'ssh -o 'StrictHostKeyChecking no' localhost ". /opt/ossdata/greenplum-db/./greenplum_path.sh; $GPHOME/sbin/gpoperation.py"' cmd had rc=255 completed=True halted=Fals
e
stdout=''
stderr='ssh_exchange_identification: read: Connection reset by peer

Cause

This issue happened on a single node cluster, with six primary instances. The backup always fails when running in the afternoon (while there are user connections). This SSH error could be caused by many reasons, one of them being too many SSH connections. 

Resolution 

Set the parameter Maxstartups to 100 in the /etc/ssh/sshd_config file.

Additional Information

Here is more information regarding the Maxstartups parameter.

MaxStartups 10:30:60
10: Number of unauthenticated connections before we start dropping
30: Percentage chance of dropping once we reach 10 (increases linearly for more than 10)
60: Maximum number of connections at which we start dropping everything

 

Comments

Powered by Zendesk