Pivotal Knowledge Base

Follow

Query Error: "FATAL: Internal error: Using fd > 65535 in MPP_FD_SET (cdbdisp.c:1983)"

Environment

 Product  Version
 Pivotal Greenplum  4.2.6.3
 OS  RHEL 6.x

Symptom

When the database created more than 65535 QEs, the query fails and the following error could be seen:

"FATAL: Internal error: Using fd > 65535 in MPP_FD_SET (cdbdisp.c:1983)"

Cause

The query is failing when the QD tries to check the status of the QE. This is due to the system call "MPP_FD_SET" being used in this piece of code which can not handle more than 65535 file descriptors. 

   #define MPP_FD_SET(fd, set) do{ 
   if (fd > 65535)
    elog(FATAL,"Internal error:  Using fd > 65535 in MPP_FD_SET");
    ((set)->__fds_bits[MPP_FD_WORD(fd)] = ((set)->__fds_bits[MPP_FD_WORD(fd)]) | MPP_FD_BIT(fd));
    } while(0)

A test case ran on a 9 Racks DCA cluster with 8 primary and 8 mirror segments per segment node. Also, there are 9*16*8 = 1152 primary segments in total.

The query for the testing contains 55 slices, hence, there are 55*1153 = 63360 file descriptors that needs to be opened to track all the QEs, along with some other small queries running at the same time lead to the error eventually.

Resolution

These pieces of code to check the status of QEs was rewritten to get rid of this system call "MPP_FD_SET". The limitation with 65535 QEs no longer exists with the new code.

This is fixed as of 4.2.6.3 DT1 and 4.2.7.0.

 

Comments

Powered by Zendesk