Pivotal Knowledge Base

Follow

Script to download logs from MYSQL server tile for further troubleshooting

Environment

Product Version
Pivotal Cloud Foundry (PCF)  
MySQL for PCF 1.7.x, 1.8-edge 

Purpose

The download-logs script is used to download logs and other diagnostic files from the MySQL service instances. These logs are necessary for Support to diagnose certain issues.

Procedure

Download script to run, so as to gather logs from the MySQL tile. Then upload to support for further troubleshooting.

Where to find

The download-logs script can be found in the URL below.

Tested

The above script was tested by using the OpsMan VM. It is suggested to use the OpsMan VM to run the script as it usually will have access to the MySQL nodes and it would be a consistent environment from which the script can ran.

Usage 

$ ./download-logs
Usage:
  -n (Required) IP address or hostname of the mysql or proxy node to retrieve logs from (can be specified multiple times)
  -d (Required) The output directory
  -X (Optional) Include audit and binary logs
  -i (Optional) Private SSH key authentication

  example:
    ./download-logs -d /tmp -n 10.0.0.1 -n 10.0.0.2

How to run?

The script leverages SSH to access the instances. So, it must be run from a host that has network access to TCP port 22 on each of the deployed MySQL instances.

  1. Copy the file to a remote host that has ssh access to the MySQL deployment.
    scp download-logs.sh USERNAME@HOST:
  2. If the MySQL instances require an SSH key, you will need to add the key to your local keychain and forward it when connecting or use "./download-logs -i <pemfile>" and pass in the private pem file supplied by your IAAS.  Below is an example of how to add the private key to your local keychain.
    ssh-add PATH_TO_SSH_KEY
  3. Connect to the remote machine, forwarding the SSH key if needed.
    ssh -A USERNAME@HOST
  4. Make download-logs executable.
    chmod +x download-logs.sh
  5. Create an output directory.
    mkdir /tmp/download-logs-output
  6. Run the script with arguments from one of the below examples.  Replace IP_ADDRESS with the relevant instance IP addresses . You can find the instance IPs in the “Status” tab of the MySQL tile in OpsManager. The script is designed to download logs from the MySQL and Proxy instances. Specify each IP using a separate -n argument.
    • Run this command to get a basic collection of syslogs and GRA logs
      ./download-logs.sh -d /tmp/download-logs-output -n IP_ADDRESS -n IP_ADDRESS -n IP_ADDRESS ...
    • (Optional) Run the command with -X to collect the same basic collection as well as the mysql auidt and bin logs. We would want to collect the bin logs in cases where there are transaction errors or deadlock events.  (Warning) Bin logs can range from a few hundred MB to several GB so collection might be large in some cases. 
      ./download-logs.sh -X -d /tmp/download-logs-output -n IP_ADDRESS -n IP_ADDRESS -n IP_ADDRESS ...
  7. If the instances require a password to connect, you will be prompted for a password for each IP specified. The password used will be the "vcap" user password that can be located under the E.R.T tile in Ops Manager. The SSH key requirement varies per IaaS.
  8. The script will prompt you to enter and confirm a passphrase that will be used to encrypt the output file.

What to do with the resulting file

If you do not have one already, file a new Pivotal Support Ticket which can be opened at https://support.pivotal.io/home. The files generated by this script will likely be large and not able to be attached to this ticket. The process for uploading large files is explained here.

What to do with the encryption passphrase for the file

The file generated by the script has been encrypted with a passphrase of your choosing. In order for Pivotal to be able to decrypt and read the files you upload, you will need to send us the encryption passphrase. To protect your files, Pivotal recommends that you do not include the password in the same channel that you're using to send us the file. Our recommendation is to send the files through the Pivotal Secure Dropzone (one channel) and post the passphrase in the support ticket (second channel). If you would rather send the passphrase in another manner, please make a note on the support ticket and the support engineer will work with you to obtain the passphrase.

Impact

The process in this KB will capture a large amount of data, including very sensitive information like queries and user data. While the script will encrypt the files to help protect this information, you may also choose to review and redact the information prior to upload. The output artefact from the download-logs script is a tar file. You can use any tar compatible tool to open the archive, extract files, review, redact and update the archive. If you need assistance opening or updating the archive, please make a note on the support ticket and the assigned support engineer can assist you with the process.

Download the latest download-logs script:

https://raw.githubusercontent.com/cloudfoundry/cf-mysql-release/master/scripts/download-logs

 

Comments

Powered by Zendesk