Pivotal Knowledge Base

Follow

gpdbrestore fails with error "(Reason=' ') exiting..." due to null entry on gp_cdatabase* file

Environment

Product Version
Pivotal Greenplum All Versions

Problem

gpdbrestore using "-s" (i.e database name) fails with a blank reason, error message looks like

gpdbrestore failed. (Reason='') exiting...

When restoring from a local backup the following is seen in the verbose output

[gpadmin@mdw 20150915]$ gpdbrestore -s testdb -v
20150922:08:28:09:019113 gpdbrestore:mdw:gpadmin-[INFO]:-Starting gpdbrestore with args: -s testdb -v
20150922:08:28:09:019113 gpdbrestore:mdw:gpadmin-[DEBUG]:-Checking if MASTER_DATA_DIRECTORY env variable is set.
[.....]
20150922:08:28:09:019113 gpdbrestore:mdw:gpadmin-[ERROR]:-gpdbrestore failed.  exiting...
Traceback (most recent call last):
  File "/usr/local/greenplum-db/lib/python/gppylib/mainUtils.py", line 280, in simple_main_locked
    exitCode = commandObject.run()
  File "/usr/local/greenplum-db/lib/python/gppylib/operations/__init__.py", line 49, in run
    self.ret = self.execute()
  File "/usr/local/greenplum-db/./bin/gpdbrestore", line 197, in execute
    info = self._gather_info()
  File "/usr/local/greenplum-db/./bin/gpdbrestore", line 385, in _gather_info
    (restore_timestamp, restore_db, compress) = self._search_for_latest()
  File "/usr/local/greenplum-db/./bin/gpdbrestore", line 483, in _search_for_latest
    db_name = GetDbName(os.path.join(path, filename)).run()
  File "/usr/local/greenplum-db/lib/python/gppylib/operations/__init__.py", line 49, in run
    self.ret = self.execute()
  File "/usr/local/greenplum-db/lib/python/gppylib/operations/restore.py", line 996, in execute
    raise GetDbName.DbNameNotFound()
DbNameNotFound

If DDBoost is used

[gpadmin@gpmdw pg_log]$ gpdbrestore -s test --ddboost -v
20150921:10:39:18:285329 gpdbrestore:gpmdw:gpadmin-[INFO]:-Starting gpdbrestore with args: -s test --ddboost -e -v
[......]
20150921:10:39:26:285329 gpdbrestore:gpmdw:gpadmin-[ERROR]:-gpdbrestore failed.  exiting...
Traceback (most recent call last):
  File "/usr/local/greenplum-db/lib/python/gppylib/mainUtils.py", line 280, in simple_main_locked
    exitCode = commandObject.run()
  File "/usr/local/greenplum-db/lib/python/gppylib/operations/__init__.py", line 49, in run
    self.ret = self.execute()
  File "/usr/local/greenplum-db/./bin/gpdbrestore", line 197, in execute
    info = self._gather_info()
  File "/usr/local/greenplum-db/./bin/gpdbrestore", line 385, in _gather_info
    (restore_timestamp, restore_db, compress) = self._search_for_latest()
  File "/usr/local/greenplum-db/./bin/gpdbrestore", line 483, in _search_for_latest
    db_name = GetDbName(os.path.join(path, filename)).run()
  File "/usr/local/greenplum-db/lib/python/gppylib/operations/__init__.py", line 49, in run
    self.ret = self.execute()
  File "/usr/local/greenplum-db/lib/python/gppylib/operations/restore.py", line 996, in execute
    raise GetDbName.DbNameNotFound()
DbNameNotFound 

Cause

The cause of the error is triggered from the below part of the code in the restore.py file

[......]
    979 class GetDbName(Operation):
    980     def __init__(self, createdb_file):
    981         self.createdb_file = createdb_file
    982     def execute(self):
    983         f = open(self.createdb_file, 'r')
    984         # assumption: 'CREATE DATABASE' line will reside within the first 50 lines of the gp_cdatabase_1_1_* file
    985         for line_no in range(0, 50):
    986             line = f.readline()
    987             if not line:
    988                 break
    989             if line.startswith("CREATE DATABASE"):
    990                 restore_db = line.split()[2]    
    991                 if restore_db is None:
    992                     raise Exception('Expected database name after CREATE DATABASE in line "%s" of file "%s"' % (line, self.createdb_file))
    993                 return restore_db.strip().strip('"')
    994         else:
    995             raise GetDbName.DbNameGiveUp()
    996         raise GetDbName.DbNameNotFound()
    997 
    998     class DbNameNotFound(Exception): pass
    999     class DbNameGiveUp(Exception): pass
[.....]

The code reads all the directory on the local file system dump location ( even if its ddboost ), which is normally $MASTER_DATA_DIRECTORY/db_dumps/<timestamp> and identify all the timestamp key which has the database name (using the file gp_cdatabase_1_1_* file ) mentioned along with gpdbrestore , if there is a issue in reading the content thats when the error below is returned

, line 996, in execute
    raise GetDbName.DbNameNotFound()
DbNameNotFound 

Here the files (gp_cdatabase_1_1_*) on one of the dump directory was zero or had NULL contents.

Solution

Find and delete the 0 byte gp_cdatabase_1_1_* files or remove that old invalid dump directory

To find the files run the following command on the master:

# find ${MASTER_DATA_DIRECTORY} -size 0 -name gp_cdatabase_1_1_\* -ls

Verify that it is ok to delete all the files listed above and delete them with the following command:

# find ${MASTER_DATA_DIRECTORY} -size 0 -name gp_cdatabase_1_1_\* -exec rm {} \;

Comments

Powered by Zendesk