Pivotal Knowledge Base

Follow

"PANIC","XX000","ERRORDATA_STACK_SIZE exceeded

Environment

Product Version
Pivotal Greenplum (GPDB) <4.3.10.0
OS  
Others  

Symptom

Crash in a segment and PANIC with the following string: ERRORDATA_STACK_SIZE exceeded

Error Message:

Master log

2016-08-29 08:13:26.142117 EDT,"502590828","sourcing",p142687,th-1648052448,"3.48.33.20","56325",2016-08-29 06:41:09 EDT,543242407,con143677,cmd2,seg-1,,dx1921613,x543242407,sx1,"LOG","58M01","Master unable to connect to seg0 alpsdw1gp1dev.idc.ge.com:1025 with options FATAL:  the database system is in recovery mode",,,,,,"BEGIN",0,,"cdbgang.c",437,
2016-08-29 08:13:26.142156 EDT,"502590828","sourcing",p142687,th-1648052448,"3.48.33.20","56325",2016-08-29 06:41:09 EDT,543242407,con143677,cmd2,seg-1,,dx1921613,x543242407,sx1,"LOG","00000","Failed connection to seg1 alpsdw1gp1dev.idc.ge.com:1026",,,,,,"BEGIN",0,,"cdbgang.c",430,

or

2017-02-20 10:09:30.547422 GMT,"wsr","DQ_db",p36982,th-875772160,"10.183.222.169","53655",2017-02-20 10:00:49 GMT,53495192,con60383,cmd1273,seg-1,,dx77529,x53495192,sx1,"LOG","00000","Dispatcher encountered connection error on seg10 00212SG1LGRE25:40002 pid=60287: server closed the connection unexpectedly
        This probably means the server terminated abnormally
        before or while processing the request.
",,,,,,,0,,,,

Segment log

2016-08-29 08:13:14.315970 EDT,"502427902","finance",p569632,th-1505868000,"172.28.8.250","26515",2016-08-29 08:06:05 EDT,613804256,con142028,,seg0,,,x613804256,sx1,"ERROR","58030","could not close temporary file base/5346845/pgsql_tmp/workfile_set_HashJoin_Slice0.XXXX1bJhGV/spillfile_f45: No such file or directory",,,,,,,0,,"bfz.c",442,
2016-08-29 08:13:14.316021 EDT,"502427902","finance",p569632,th-1505868000,"172.28.8.250","26515",2016-08-29 08:06:05 EDT,613804256,con142028,,seg0,,,x613804256,sx1,"ERROR","58030","could not cl
ose temporary file base/5346845/pgsql_tmp/workfile_set_HashJoin_Slice0.XXXX1bJhGV/spillfile_f43: No such file or directory",,,,,,,0,,"bfz.c",442,
2016-08-29 08:13:14.609128 EDT,"502427902","finance",p569632,th-1505868000,"172.28.8.250","26515",2016-08-29 08:06:05 EDT,613804256,con142028,,seg0,,,x613804256,sx1,"PANIC","XX000","ERRORDATA_STACK_SIZE exceeded (elog.c:1409)",,,,,,,0,,"elog.c",1409,"Stack trace:
1    0xb0802a postgres <symbol not found> + 0xb0802a
2    0xb0a2a3 postgres elog_start + 0xa3
3    0x5458a7 postgres AbortTransaction + 0xb7
4    0x549225 postgres AbortCurrentTransaction + 0x25
5    0x99be07 postgres PostgresMain + 0x537
6    0x8fbcee postgres <symbol not found> + 0x8fbcee
7    0x8fea70 postgres PostmasterMain + 0xff0
8    0x800aff postgres main + 0x44f
9    0x7f7ca1bedd5d libc.so.6 __libc_start_main + 0xfd
10   0x4c4db9 postgres <symbol not found> + 0x4c4db9

Cause

The reason for the failure is that in the affected versions, when a transaction is cancelled while a query is spilling to disk, Greenplum initially removes the directory that contains the temporary spill files, and then tries to close these files. However, since the directory has been deleted, all the close operations fail, complaining of a missing directory.

Resolution

The issue is fixed in 4.3.10.0. The issue number to track this is 26574.

Comments

Powered by Zendesk