Pivotal Knowledge Base

Follow

Installation of pg_crypto Fails with "ERROR: could not load library "xxxx/pgcrypto.so": xxxx/lib/postgresql/pgcrypto.so: undefined symbol: FIPS_mode_set"

Environment

Product Version
Pivotal Greenplum 4.3.x, 4.2.x
OS RHEL 6.x

Symptom

Installation of pgcrypto fails with the error message below. 

Error Message:

[gpadmin@gpdbsne lib64]$ psql -d <database-name> -f $GPHOME/share/postgresql/contrib/pgcrypto.sql
SET
psql:/usr/local/greenplum-db/./share/postgresql/contrib/pgcrypto.sql:9: ERROR: could not load library "/usr/local/greenplum-db-4.3.5.3/lib/postgresql/pgcrypto.so": /usr/local/greenplum-db-4.3.5.3/lib/postgresql/pgcrypto.so: undefined symbol: FIPS_mode_set
[......]
[......]

Cause 

The issue is caused when the wrong crypto libraries are referenced.

For example, the crypto library "libcrypto.so.0.9.8" references the crypto library in the "greenplum-perfmon-web-4.1.1.5-build-1" installation directory when it should be referencing the Greenplum database libraries.

[gpadmin@gpdbsne lib64]$ ldd /usr/local/greenplum-db-4.3.5.3/lib/postgresql/pgcrypto.so
	linux-vdso.so.1 =>  (0x00007fffa01ff000)
	libcrypto.so.0.9.8 => /usr/local/greenplum-perfmon-web-4.1.1.5-build-1/lib/libcrypto.so.0.9.8 (0x00007f3a9cd21000)
	libz.so.1 => /usr/local/greenplum-cc-web/./lib/libz.so.1 (0x00007f3a9cb0a000)
	libc.so.6 => /lib64/libc.so.6 (0x00007f3a9c75e000)
	libdl.so.2 => /lib64/libdl.so.2 (0x00007f3a9c55a000)
	/lib64/ld-linux-x86-64.so.2 (0x0000003601e00000)

This is caused by the method used to set the environment variables when the user connects to the Unix server. For example, in this case, the LD_LIBRARY_PATH environment variable is set.

[gpadmin@gpdbsne lib64]$ env | grep LD
LD_LIBRARY_PATH=/usr/local/greenplum-perfmon-web-4.1.1.5-build-1/lib:/usr/local/greenplum-cc-web-1.3.0.3-build-18/lib:/usr/local/greenplum-db/./lib:/usr/local/greenplum-db/./ext/python/lib: 

As shown in the setting above, the following locations "/usr/local/greenplum-perfmon-web-4.1.1.5-build-1/lib" & "/usr/local/greenplum-cc-web-1.3.0.3-build-18/lib" are used to reference the crypto libraries first, and this is caused by the method used to set up the environment path in the .bashrc file ( user updated ).

Note: The last line of the source always uses the first batch to check the path of the libraries.

source /usr/local/greenplum-db/greenplum_path.sh
source /usr/local/greenplum-db/gpcc_path.sh
source /usr/local/greenplum-db/gpperfmon_path.sh

The cause here is that the postmaster of the database points to the crypto file of two different installations and not the Greenplum database.

[gpadmin@gpdbsne lib64]$ lsof -p 31949  | grep crypto
postgres 31949 gpadmin  mem    REG              253,1   309353 1191079 /usr/local/greenplum-cc-web-1.3.0.3-build-18/lib/libk5crypto.so.3.1
postgres 31949 gpadmin  mem    REG              253,1  1788415 1325183 /usr/local/greenplum-perfmon-web-4.1.1.5-build-1/lib/libcrypto.so.0.9.8
[gpadmin@gpdbsne lib64]$ 

Resolution

Follow the steps to resolve this issue:

  1. Correct the environment to have the Greenplum library sourced at the end.
    source /usr/local/greenplum-db/gpcc_path.sh
    source /usr/local/greenplum-db/gpperfmon_path.sh
    source /usr/local/greenplum-db/greenplum_path.sh
  2. When you refresh this, the path now points to the correct location.
    [gpadmin@gpdbsne lib64]$ env | grep LD
    LD_LIBRARY_PATH=/usr/local/greenplum-db/./lib:/usr/local/greenplum-db/./ext/python/lib:/usr/local/greenplum-perfmon-web-4.1.1.5-build-1/lib:/usr/local/greenplum-cc-web-1.3.0.3-build-18/lib:
  3. Restart the database and the new postmaster process will now point to the correct libraries.
    [gpadmin@gpdbsne lib64]$ lsof -p 19896 | grep crypto
    postgres 19896 gpadmin  mem    REG              253,1   583652 1071088 /usr/local/greenplum-db-4.3.5.3/lib/libk5crypto.so.3.1
    postgres 19896 gpadmin  mem    REG              253,1  1855858 1071063 /usr/local/greenplum-db-4.3.5.3/lib/libcrypto.so.0.9.8
    [gpadmin@gpdbsne lib64]$ 
  4. Re-execute the pgcrypto file again using the command below.
    psql -d <database-name> -f $GPHOME/share/postgresql/contrib/pgcrypto.sql

Notes:

The other issue that can lead to the problems discussed in this article are the following:

  • When the OpenSSL rpm is updated on the server, if the OpenSSL rpm is updated then the server might need a reboot to reload the packages.
  • Wrong extension package (gppkg) is downloaded for the version of the database.

Comments

Powered by Zendesk