Pivotal Greenplum: 4.3.x
OS: RHEL 6.x
When trying to create a string buffer greater than 1 GB in size, an error is logged.
Cannot enlarge string buffer containing 1073349879 bytes by 700160 more bytes.
There is a hardcoded limit, MaxAllocSize, of 1 GB on the size of a string buffer in Greenplum. There is no guarantee that allocations smaller than MaxAllocSize will succeed. However, allocation requests larger than MaxAllocSize will be definitely denied.
This is deliberately chosen to correspond to the limiting size of variable-length (varlena) objects under TOAST.
This is not indicating an issue with the system, but with the data being processed.
You will need to determine what data is being read that is larger than 1 GB and process the data in a different manner to avoid objects greater than 1 GB.