Pivotal Knowledge Base

Follow

gptransfer Fails on Multiple "Distributed By" Columns

Environment

  • Pivotal Greenplum Database (GPDB) 4.3.x
  • Operating System (OS)- Red Hat Enterprise Linux (RHEL) 6.x

Symptom

gptransfer fails when the table has multiple "Distributed By" columns. This issue has been fixed in 4.3.11.3 version of Greenplum. 

Error Message:

20170112:14:57:47:028186 gptransfer:mdw:gpadmin-[DEBUG]:-[worker1] finished cmd: transfer of gpadmin.public.sample cmdStr='None'  had result: cmd had rc=1 completed=False halted=False
  stdout='error 'ERROR:  column "id1, id2" named in 'DISTRIBUTED BY' clause does not exist

Cause 

There is a known bug in gptransfer where if the table is "Distributed By" multiple columns it would fail.

For example:

gpadmin=# create table sample (id1 int, id2 int, name text) distributed by (id1,id2); 
CREATE TABLE 
gpadmin=# \d sample; 
Table "public.sample" 
Column | Type | Modifiers 
--------+---------+----------- 
id1 | integer | 
id2 | integer | 
name | text | 
Distributed by: (id1, id2)

gptransfer -t gpadmin.public.sample --dest-host=mdw   --source-host=mdw --dest-database=gpadmin1   --dest-port=6171 --source-port=6171 --verbose

Error message details:

20170112:14:57:47:028186 gptransfer:mdw:gpadmin-[ERROR]:-Failed to transfer table gpadmin.public.sample
20170112:14:57:47:028186 gptransfer:mdw:gpadmin-[INFO]:-Remaining 1 of 1 tables
20170112:14:57:47:028186 gptransfer:mdw:gpadmin-[DEBUG]:-[worker1] finished cmd: transfer of gpadmin.public.sample cmdStr='None'  had result: cmd had rc=1 completed=False halted=False
  stdout='error 'ERROR:  column "id1, id2" named in 'DISTRIBUTED BY' clause does not exist
' in 'CREATE WRITABLE EXTERNAL WEB TABLE gptransfer.w_ext_sample_e9d8a4dc28e58a6e74ec145d754faa9a (LIKE "public"."sample")
                   EXECUTE 'cat > /home/gpadmin/gptransfer_28186/gpadmin.public.sample/gpadmin.public.sample.pipe.$GP_SEGMENT_ID'
                   FORMAT 'CSV'
                 (DELIMITER AS ',' QUOTE AS E'') ENCODING 'UTF8' DISTRIBUTED BY ("id1, id2")''
  stderr='None'
20170112:14:57:47:028186 gptransfer:mdw:gpadmin-[WARNING]:-1 tables failed to transfer.  A list of these tables

Resolution

The issue has been fixed in 4.3.11.3. In case you cannot upgrade, the workaround is to change the table distribution policy to a single column and then transfer and change the distribution policy back.

 

Comments

Powered by Zendesk