Pivotal Knowledge Base


How gp_log_collector Automates the Collection of Pivotal Greenplum Logs


  • Pivotal Greenplum Database (GPDB) 4.3.x
  • Operating System (OS)- Red Hat Enterprise Linux (RHEL) 6.x


gp_log_collector is the tool part of the GPMT meta-tool.

Usually, it is required to run through ssh, mkdir, scp, and tar just to collect GPDB logs. The purpose of this tool is to make supportability of the GPDB effortless by automating the collection of master and segment log files and configurations. 

It is much easier for customers now to provide logs to Pivotal support using this tool.


    gpmt gp_log_collector [ -failed-segs | -c ID1,ID2,... | -hostfile FILE | -h HOST1,HOST2,... ]
        [ -start YYYY-MM-DD ] [ -end YYYY-MM-DD ]
        [ -dir PATH ] [ -segdir PATH ] [ -a ]

    When run with no options gp_log_collector will collect the master and standby logs for today.
    When the "-hostfile" option is set gp_log_collector will collect master, standby and all segment
    instances running on the specified segment hosts.

    Note: Hostnames provided through -hostfile or -h must match the hostname column in gp_segment_configuration.

    -failed-segs       Query gp_configuration_history for list of faulted content IDs.
    -free-space        default=10  free space threshold which will abort log collection if reached
    -c                 Comma seperated list of content IDs.
    -hostfile          Read hostnames from a hostfile.
    -h                 Comma seperated list of hostnames.
    -start             Start date for logs to collect (defaults to current date)
    -end               End date for logs to collect (defaults to current date)
    -a                 Answer Yes to all prompts
    -dir               Working directory (defaults to current directory)
    -segdir            Segment temporary directory (defaults to /tmp)
-skip-master       Skip master log collection EXAMPLES: Collect master logs for today gpmt gp_log_collector Collect master logs and all segment hosts from hostfile for today gpmt gp_log_collector -hostfile ~/gpconfig/hostfile Collect logs for any segments marked down in gp_segment_configuration_history from 2016-03-21 to today: gpmt gp_log_collector -failed-segs -start 2016-03-21 Collect logs for content 13, 12 and 15 for today: gpmt gp_log_collector -c "13,12,15" Collect logs from 'sdw1.gpdb.local' and 'sdw4.gpdb.local' for today: gpmt gp_log_collector -h sdw1.gpdb.local,sdw4.gpdb.local Collect logs from 'sdw2.gpdb.local' between 2016-03-21 and 2016-03-24: gpmt gp_log_collector -h sdw2.gpdb.local -start 2016-03-21 -end 2016-03-24



In cases where a segment failed over to their mirrors or simply failed for any unexplained reason, this feature can help. Using this flag will tell gp_log_collector to scan gp_configuration_history using the time frame specified by -start and -end flags. Content ID logs will be collected for any identified DBIDs.


When running gp_log_collector the generated tarball could be very large and this is normally because the master logs will contain lots of information.  In some cases, support will only require the segment logs so this option allows users to skip the master logs reducing the size of the resulting tar file. 


Information to be sent to support:

The tool creates an archive as per the given flags in the current working directory using the timestamp in the file name. For example, file "gp_log_collection_2016-05-16_08-50-06.tar" was created on 16th May 2016 at 08:50:06 . Customers need to send the generated archive to Pivotal support.

For informational purpose only

The archive will have the following information collected:  

1. From Database psql:

- version
- uptime   >>>> database uptime
- pg_settings
- pg_resqueue
- pg_database
- gp_segment_configuration
- gp_configuration_history 

2. From segment servers (Master, Primary, and Mirror) :

Command outputs:

- uname-a
- sysctl-a
- psaux
- netstat-rn
- netstat-i
- lsof
- ifconfig
- free
- df-h 

System files:

- /etc/redhat-release
- /etc/sysctl.conf
- /etc/sysconfig/network
- /etc/security/limits.conf 

Database related files:

 - $SEG_DIR/pg_hba.conf      
- $SEG_DIR/pg_log/           
- $SEG_DIRE/postgresql.conf  
- ~/gpAdminLogs



  • Avatar
    Gurupreet Singh Bhatia

    Its awesome

Powered by Zendesk