Pivotal Knowledge Base

Follow

gpdbrestore Fails due to Filesystem where the Dump is Located being Read-Only

Environment

Pivotal Greenplum Database (GPDB) 4.3.2.x

Symptom

gpdbrestore fails with the following error:

Error Message

{restore failed to start for source dbid 1, target dbid 1 on host xxx in the required time interval
}
[.....]
[.....]
20150909:20:16:46|gp_restore-[INFO]:-Successfully launched Greenplum Database restore on dbid 1 to restore dbid 1
20150909:20:26:46|gp_restore-[ERROR]:-see error report for details
20150909:20:26:46|gp_restore-[WARN]:-Cannot open report file /mnt/xxx/xxx/db_dumps/20150908/gp_restore_20150908200609.rpt for writing.  Will use stdout instead.

There is no report file or the status file generated to identify the cause of the failure

On the master log, there is an error message stating that it couldn't create a status file due to the file system being the read-only file system.

2015-09-09 20:16:13.730244 EST,,,p21612,th-719968512,,,,0,,,seg-1,,,,,"LOG","00000","3rd party error log: cannot create '/mnt/xxx/xxx/db_dumps/20150908/gp_restore_status_1_1_20150908200609': Read-only file system

Cause

When you run a restore, gpdbrestore tends to create a status file on the same directory of the dumps for each segment, now when the mount point is read-only its unable to create one and hence the failure.

Resolution

Upgrade to GPDB>4.3.2.x where the gpdbrestore has a new parameter "--report-status-dir" introduced to specify the new location of the status file directory.

--report-status-dir report_directory

Specifies the absolute path to the directory on the each Greenplum Database host (master and segment hosts) where gpdbrestore writes report status files for a restore operation. If report_directory does not exist or is not writable, gpdbrestore returns an error and stops.
If this option is not specified and the -u option is specified, report status files are written to the location specified by the -u option if the -u location is writable. If the location specified by -u option is not writable, the report status files are written to segment data directories.

Workaround

  • Make the filesystem read-write
  • Move the dumps to a new location
  • Create a soft link to the filesystem from a read-write file system and specify that location to gpdbrestore.

Comments

Powered by Zendesk