Pivotal Knowledge Base

Follow

GPTRANSFER Fails on Multiple "Distributed By" Columns

Environment

 Product  Version
 Pivotal Greenplum  4.3.x upto 4.3.11.3
 OS  RHEL 6.x

Symptom

gptransfer fails when the table has multiple "Distributed By" columns. This issue has been fixed in 4.3.11.3 version of Greenplum. 

Error Message:

20170112:14:57:47:028186 gptransfer:mdw:gpadmin-[DEBUG]:-[worker1] finished cmd: transfer of gpadmin.public.sample cmdStr='None'  had result: cmd had rc=1 completed=False halted=False
  stdout='error 'ERROR:  column "id1, id2" named in 'DISTRIBUTED BY' clause does not exist

Cause 

There is a known bug in gptransfer where if the table is "Distributed By" multiple columns it would fail.

For example:

gpadmin=# create table sample (id1 int, id2 int, name text) distributed by (id1,id2); 
CREATE TABLE 
gpadmin=# \d sample; 
Table "public.sample" 
Column | Type | Modifiers 
--------+---------+----------- 
id1 | integer | 
id2 | integer | 
name | text | 
Distributed by: (id1, id2)

gptransfer -t gpadmin.public.sample --dest-host=mdw   --source-host=mdw --dest-database=gpadmin1   --dest-port=6171 --source-port=6171 --verbose

Error message details:

20170112:14:57:47:028186 gptransfer:mdw:gpadmin-[ERROR]:-Failed to transfer table gpadmin.public.sample
20170112:14:57:47:028186 gptransfer:mdw:gpadmin-[INFO]:-Remaining 1 of 1 tables
20170112:14:57:47:028186 gptransfer:mdw:gpadmin-[DEBUG]:-[worker1] finished cmd: transfer of gpadmin.public.sample cmdStr='None'  had result: cmd had rc=1 completed=False halted=False
  stdout='error 'ERROR:  column "id1, id2" named in 'DISTRIBUTED BY' clause does not exist
' in 'CREATE WRITABLE EXTERNAL WEB TABLE gptransfer.w_ext_sample_e9d8a4dc28e58a6e74ec145d754faa9a (LIKE "public"."sample")
                   EXECUTE 'cat > /home/gpadmin/gptransfer_28186/gpadmin.public.sample/gpadmin.public.sample.pipe.$GP_SEGMENT_ID'
                   FORMAT 'CSV'
                 (DELIMITER AS ',' QUOTE AS E'') ENCODING 'UTF8' DISTRIBUTED BY ("id1, id2")''
  stderr='None'
20170112:14:57:47:028186 gptransfer:mdw:gpadmin-[WARNING]:-1 tables failed to transfer.  A list of these tables

Workaround

The issue has been fixed in 4.3.11.3. Incase you cannot upgrade, the workaround is to change the table distribution policy to a single column and then transfer and change the distribution policy back.

 

Comments

Powered by Zendesk