Pivotal Knowledge Base

Follow

Query failed with <ERROR: unable to read table "t_review_sentiment_ext" file "hdfs:...8020/hawq_data/gpseg31/...>

Problem

The Pivotal HDB(HDB) query failed withe following messages

WARNING:  cannot read from file hdfs://big1hdvip.hd.corp.haier.com:8020/hawq_data/gpseg31/16385/16390/142988.129, errno 5  (seg31 slice1 big1hd06.hd.corp.haier.com:40003 pid=27129)
ERROR:  unable to read table "t_review_sentiment_ext"file "hdfs://big1hdvip.hd.corp.haier.com:8020/hawq_data/gpseg31/16385/16390/142988.129"(errcode 5)  (seg31 slice1  big1hd06.hd.corp.haier.com:40003pid=27129)  

Analysis

Check the segment log file (gpseg31 in this example) if messages look similar to the following

E1114 14:41:13.07645014680Hdfs.cpp:16] read from input stream from position: 1769472read size: 3360read data from local block reader: cannot read data when read and  verify data, bad flag: 0fail flag: 1eof: 1
    @     0x7fce0aaa9ff4  Hdfs::Client::LocalBlockReader::read()
    @     0x7fce0aa836e2  Hdfs::Client::HdfsInputStreamImpl::read()
    @     0x7fce0aa9b6c5  Hdfs::HdfsInputStream::read()
    @     0x7fce0aa727b8  hdfsRead
    @     0x7fcdbad8a8dd  gpfs_hdfs_read
    @           0x8f383b  HdfsRead
    @           0x8e50f2  FileRead
    @           0xb4ba76  BufferedReadGrowBuffer
    @           0xb41015  AppendOnlyStorageRead_GetBuffer
    @           0xa8a988  datumstreamread_block
    @           0x59f6d2  aocs_getnext
    @           0x7318e6  AOCSNext
    @           0x71c407  ExecScan
    @           0x731822  ExecAOCSScan
    @           0x710b4d  ExecProcNode
    @           0x743e76  ExecMotion
    @           0x710a59  ExecProcNode
    @           0x709964  ExecutePlan
    @           0x70b12e  ExecutorRun
    @           0x93f629  PortalRun
    @           0x93893f  PostgresMain
    @           0x893871  BackendStartup
    @           0x899f20  PostmasterMain
    @           0x79de5a  main
In the example above the block file on the data node (big1hd06) might have been corrupted.
 
Solution

Check if there is a parameter "dfs.client.enable.read.from.local" in hdfs-client.xml. If yes, try change this parameter to "true". Just make the change on DataNode reporting the error and run query again. No need to restart DataNode

<name>dfs.client.enable.read.from.local</name> 
<value>true</value> 
hdfs-client.xml typically locates under <HDB install dir>/etc

Comments

Powered by Zendesk