Pivotal Knowledge Base

Follow

Writable External Tables failed to send Request to GPFDIST

Environment

 Product  Version
 Pivotal Greenplum  4.3.x
 OS  RHEL 6.x

Symptom

Inserting data into writable external table fails with the following error.

Error message:

psql -c "insert into callhour_ext_unld1_local (start_date) values('2017-02-22')" wicdwh
WARNING: http://xx.xx.xx.xx:8081/xxxx error (2 - Failed initialization) (seg10 led00651.wic.west.com:40010 pid=5182)
WARNING: failed to send request to gpfdist (http://xx.xx.xx.xx:8081/xxxx), will retry after 1 seconds (seg10 led00651.wic.west.com:40010 pid=5182)
WARNING: http://xx.xx.xx.xx:8081/xxxx error (2 - Failed initialization) (seg10 led00651.wic.west.com:40010 pid=5182)


There were no issue with READABLE external tables and data can be selected from it. The error was consistent with WRITABLE external tables. 

Cause

"Failed initialization" error message has been encountered since the database was using a different version of libcurl

You can check from the postmaster process maps what the libcurl version is. In the example below, it shows a different version of libcurl for Master's postmaster process PID 22895:

[gpadmin@led00653 ~]$ cat /proc/22895/maps | grep curl
7fad896b0000-7fad8970d000 r-xp 00000000 fd:00 35016610     /usr/local/greenplum-db-4.3.9.1/ext/R-3.3.1/extlib/libcurl.so.4
7fad8970d000-7fad8990c000 ---p 0005d000 fd:00 35016610     /usr/local/greenplum-db-4.3.9.1/ext/R-3.3.1/extlib/libcurl.so.4
7fad8990c000-7fad8990f000 rw-p 0005c000 fd:00 35016610     /usr/local/greenplum-db-4.3.9.1/ext/R-3.3.1/extlib/libcurl.so.4
[gpadmin@led00653 ~]$ 

 However, An ideal version of libcurl should be using the library from $GPHOME/lib as shown below: 

[gpadmin@mdw ~]$ cat /proc/16236/maps | grep curl 
2b915b2c3000-2b915b316000 r-xp 00000000 08:02 9207864     /usr/local/GP-4.3.9.1/lib/libcurl.so.4.2.0
2b915b316000-2b915b515000 ---p 00053000 08:02 9207864     /usr/local/GP-4.3.9.1/lib/libcurl.so.4.2.0
2b915b515000-2b915b518000 rw-p 00052000 08:02 9207864     /usr/local/GP-4.3.9.1/lib/libcurl.so.4.2.0

 

Since the cluster has been started using different libraries, it could be an issue with the source file where LD_LIBRARY_PATH has been defined. Also, it would be good to check user profile files (.bashrc etc) for if the variable has been set there explicitly.

In this case, it was an issue with $GPHOME/greenplum_path.sh which has been updated through gppkg utility during R installation. Below were the issue related entries updated by gppkg:  

export R_HOME=$GPHOME/ext/R-3.3.1
export LD_LIBRARY_PATH=$GPHOME/ext/R-3.3.1/lib:$GPHOME/ext/R-3.3.1/extlib:$LD_LIBRARY_PATH
export PATH=$GPHOME/ext/R-3.3.1/bin:$PATH


As a result, the postmaster gets started using R's libcurl, breaking the external writable table loading. A bug has been filed with engineering for gppkg installation. 

Resolution

  • Configure $GPHOME/greenplum_path.sh on master and segments to append R path after the LD_LIBRARY_PATH as shown below:
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$GPHOME/ext/R-3.3.1/lib:$GPHOME/ext/R-3.3.1/extlib
  • Restart the database.
  • Check the writable external table.
  • Also, check the R functions if there is any issues after the change.

 

Comments

Powered by Zendesk