Pivotal Knowledge Base

Follow

Pivotal Query Optimizer/ORCA causes PANIC where querying tables with bitmap indexes

Environment 

Product Version
Pivotal Greenplum (GPDB) 4.3.9.0 and older

Symptom

When you have a table with a bitmap index on it, and you have the optimizer turned on, utilizing an IN clause with a subselect such as SELECT * FROM my_table WHERE a IN (SELECT 1); will PANIC GPDB.

  • This is affecting all versions of 4.3.8.x and 4.3.9.0.
  • The legacy optimizer does not experience this behaviour.

Error Message:

Note: These log entries will be returned to the console (psql or client software):

server closed the connection unexpectedly
	This probably means the server terminated abnormally
	before or while processing the request.
The connection to the server was lost. Attempting reset: Failed.

Note: These log entries will be in the $MASTER_DATA_DIRECTORY/pg_log/gpdb<date>.csv file:

"PANIC","XX000","Unexpected internal error: Master process received signal SIGSEGV",,,,,,,0,,,,"1    0x7fe1442fd790 libpthread.so.0 <symbol not found> (??:0)
2    0x7fe144b15468 libgpdbcost.so _ZN8gpdbcost14CCostModelGPDB19CostBitmapTableScanEPN4gpos11IMemoryPoolERN5gpopt17CExpressionHandleEPKS0_PKNS4_10ICostModel12SCostingInfoE (??:0)
3    0x7fe144b18db7 libgpdbcost.so _ZNK8gpdbcost14CCostModelGPDB4CostERN5gpopt17CExpressionHandleEPKNS1_10ICostModel12SCostingInfoE (??:0)
4    0x7fe1447efe4f libgpopt.so _ZN5gpopt12CPartialPlan11CostComputeEPN4gpos11IMemoryPoolE (??:0)
5    0x7fe14474a860 libgpopt.so _ZN5gpopt16CGroupExpression14CostLowerBoundEPN4gpos11IMemoryPoolEPNS_13CReqdPropPlanEPNS_12CCostContextEj (??:0)
6    0x7fe1447409b8 libgpopt.so _ZN5gpopt6CGroup14CostLowerBoundEPN4gpos11IMemoryPoolEPNS_13CReqdPropPlanE (??:0)
7    0x7fe1447ef658 libgpopt.so _ZN5gpopt12CPartialPlan26ExtractChildrenCostingInfoEPN4gpos11IMemoryPoolEPNS_10ICostModelERNS_17CExpressionHandleEPNS4_12SCostingInfoE (??:0)
8    0x7fe1447efc8e libgpopt.so _ZN5gpopt12CPartialPlan11CostComputeEPN4gpos11IMemoryPoolE (??:0)
9    0x7fe14474a860 libgpopt.so _ZN5gpopt16CGroupExpression14CostLowerBoundEPN4gpos11IMemoryPoolEPNS_13CReqdPropPlanEPNS_12CCostContextEj (??:0)
10   0x7fe1447f49b0 libgpopt.so _ZN5gpopt7CEngine12FSafeToPruneEPNS_16CGroupExpressionEPNS_13CReqdPropPlanEPNS_12CCostContextEjPNS_5CCostE (??:0)
11   0x7fe1447370e6 libgpopt.so _ZN5gpopt31CJobGroupExpressionOptimization14EevtInitializeEPNS_17CSchedulerContextEPNS_4CJobE (??:0)
12   0x7fe14473615a libgpopt.so _ZN5gpopt31CJobGroupExpressionOptimization8FExecuteEPNS_17CSchedulerContextE (??:0)
13   0x7fe14472fb25 libgpopt.so _ZN5gpopt10CScheduler11ExecuteJobsEPNS_17CSchedulerContextE (??:0)
14   0x7fe14472fc2a libgpopt.so _ZN5gpopt10CScheduler11ProcessJobsEPNS_17CSchedulerContextE (??:0)
15   0x7fe14472fcb0 libgpopt.so _ZN5gpopt10CScheduler3RunEPv (??:0)
16   0x7fe1447f638c libgpopt.so _ZN5gpopt7CEngine18MainThreadOptimizeEv (??:0)
17   0x7fe1447f7ea8 libgpopt.so _ZN5gpopt7CEngine8OptimizeEv (??:0)
18   0x7fe14474db5c libgpopt.so _ZN5gpopt10COptimizer13PexprOptimizeEPN4gpos11IMemoryPoolEPNS_13CQueryContextEPNS1_16CDynamicPtrArrayINS_12CSearchStageEXadL_ZNS1_13CleanupDeleteIS7_EEvPT_EEEE (??:0)
19   0x7fe14474e389 libgpopt.so _ZN5gpopt10COptimizer13PdxlnOptimizeEPN4gpos11IMemoryPoolEPNS_11CMDAccessorEPKN5gpdxl8CDXLNodeEPKNS1_16CDynamicPtrArrayIS7_XadL_ZNS1_14CleanupReleaseIS7_EEvPT_EEEESG_PNS_19IConstExprEvaluatorEjjjPNSA_INS_12CSearchStageEXadL_ZNS1_13CleanupDeleteISJ_EEvSD_EEEEPNS_16COptimizerConfigEPKc (??:0)
20   0xd2df33 postgres _ZN9COptTasks14PvOptimizeTaskEPv (COptTasks.cpp:1032)
21   0x7fe14589edd3 libgpos.so _ZN4gpos5CTask7ExecuteEv (??:0)
22   0x7fe14589d638 libgpos.so _ZN4gpos7CWorker7ExecuteEPNS_5CTaskE (??:0)
23   0x7fe1458a159d libgpos.so _ZN4gpos14CAutoTaskProxy7ExecuteEPNS_5CTaskE (??:0)
24   0x7fe1458a4dd8 libgpos.so gpos_exec (??:0)
25   0xd298d4 postgres _ZN9COptTasks7ExecuteEPFPvS0_ES0_ (COptTasks.cpp:554)
26   0xd29cfd postgres _ZN9COptTasks15PplstmtOptimizeEP5QueryPb (COptTasks.cpp:1732)
27   0x89f60e postgres planner (planner.c:219)
28   0x998369 postgres <symbol not found> (postgres.c:878)

Cause

  • This issue is caused by software defect MPP-26427.
  • The Pivotal Optimizer/ORCA cost model assumes that the index condition expression always has one used column, which is not suitable for a constant table.  
  • This will cause ORCA to proceed to generate a plan without assuming the capability of the index scan to handle ScalarArrayOpExpr when it does not lead to a PANIC.

RCA 

Set up a table with a BITMAP index and run a query with a subselect of a static value in an IN clause to PANIC the system:

SET optimizer=ON;
CREATE TABLE bitmap_test as SELECT * FROM generate_series(1,1000000) as a distributed randomly;
CREATE INDEX bitmap_index ON bitmap_test USING BITMAP(a);
SELECT * FROM bitmap_test WHERE a IN (SELECT 1);

You will immediately get this back in the console:

server closed the connection unexpectedly
	This probably means the server terminated abnormally
	before or while processing the request.
The connection to the server was lost. Attempting reset: Failed.

Resolution

  • Upgrade to GPDB 4.3.9.1 or newer where this issue is first fixed where a change in the translator will guard against such a plan if the database is unable to execute an Index Scan with a ScalarArrayOpExpr.
  • Do not use the Pivotal Query Optimizer.
  • Do not use a subselect in the IN Clause.

 

Comments

Powered by Zendesk