Pivotal Knowledge Base

Follow

gprecoverseg failed with "could not open file position"

Environment

Product Version
Pivotal Greenplum (GPDB) 4.3.x, 4.2.x
OS RHEL 6.x

Symptom
Incremental gprecoverseg reports the following error message:

2016-08-26 01:10:47.380578,,,p75103,th-595642592,,,,0,,,seg-1,,,,,"WARNING","58P01","mirror failure, could not open file position:240764600 file flags '2' file mode '600' : No such file or directory, failover requested","identifier 'base/17144/2544455.2' operation 'open' relation type 'append only' message count '1315657'","run gprecoverseg to re-establish mirror connectivity",,,"mirroring role 'mirror role' mirroring state 'resync' segment state 'up and running' process name(pid) 'mirror consumer append only process(75103)' filerep state 'up and running' ",,0,,"cdbfilerepmirror.c",949,

 When looking at the failed instance, some of them were found to be missing the .1 file while others were missing the .2 file:

[gpadmin@hdm1 gpseg3]$ ls -l /data/*/gpseg*/base/17144/2544455* 
-rw------- 1 gpadmin gpadmin 0 Jul 18 18:55 /data/mirror/gpseg12/base/17144/2544455
-rw------- 1 gpadmin gpadmin 3397978616 Aug 26 00:08 /data/mirror/gpseg12/base/17144/2544455.2
-rw------- 1 gpadmin gpadmin 0 Jul 18 18:55 /data/mirror/gpseg13/base/17144/2544455
-rw------- 1 gpadmin gpadmin 3411895152 Aug 26 00:32 /data/mirror/gpseg13/base/17144/2544455.2
-rw------- 1 gpadmin gpadmin 0 Jul 18 18:55 /data/mirror/gpseg14/base/17144/2544455
-rw------- 1 gpadmin gpadmin 236963064 Aug 4 05:34 /data/mirror/gpseg14/base/17144/2544455.2
-rw------- 1 gpadmin gpadmin 0 Jul 18 18:55 /data/primary/gpseg0/base/17144/2544455
-rw------- 1 gpadmin gpadmin 3415391256 Aug 24 00:09 /data/primary/gpseg0/base/17144/2544455.2
-rw------- 1 gpadmin gpadmin 0 Jul 18 18:55 /data/primary/gpseg3/base/17144/2544455
-rw------- 1 gpadmin gpadmin 1726204856 Aug 4 05:34 /data/primary/gpseg3/base/17144/2544455.1

Resolution
The customer mentioned that there was a hardware issue at the hdm1 host before. This might have caused the file to be missing. Touch the missing file at the related segment and inform the customer of the affected table/partition name. They may need to re-create that object or reload the data from another source.

For example, here it is required to touch this file : 

/data/primary/gpseg3/base/17144/2544455.2



 

Comments

Powered by Zendesk