Pivotal Knowledge Base

Follow

How to Calculate the Loggregators Message Throughput

Environment

 Product  Version
 Pivotal Cloud Foundry® (PCF)  1.6.x and later

Purpose

The purpose of this document is to help in calculating the total number of messages that the Loggregator produces over a given amount of time. This is useful when trying to scale your Loggregator environment to handle the number of messages you are producing.

Prerequisites

Software requited:

  • cf cli [1] 
  • Cloud Foundry Firehose plugin

Note: The following commands should be run from the Ops Manager VM.

How to install the Cloud Foundry Firehose plugin

Installing the plugin is really simple.

1. Log on to the CF CLI with your admin credentials:

cf api <your_cf_api_url>

cf login

2. Add the plugin repository:

cf add-plugin-repo CF-Community http://plugins.cloudfoundry.org/

3. Install the plugin:

cf install-plugin “Firehose Plugin” -r CF-Community 

4. Install Pipe Viewer (PV):

apt-get install pv 

Procedure

Please run the following commands from the Ops Manager VM.

     1.    Connect via Single SHell (SSH) into the Ops Manager VM  

   2.    Log on as admin into the cf cli

   3.    cf nozzle -n | pv -l -i10 -r >/dev/null

[460.4 /s] 

i.e. Loggregator is processing 460.4 messages a second. The output will refresh every 10 seconds. 

Scaling Loggregator [2]

Rule of thumb

1. Dopplers

The number of Dopplers should be approximately the total number of messages per second divided by 2000.

(Dopplers =1 per msgs/sec /2,000)

2. Traffic Controllers

You should have one Traffic Controller for every four Dopplers.

(Traffic Controllers = 1 for every 4 Dopplers)

3. Nozzles 

For a Nozzle such as the firehose-to-syslog [3] Nozzle, you will require one instance per 10,000 msgs/sec.

(Nozzles = 1 for every 10,000 msgs/sec) 

Additional Information

[1] Installing cf cli: https://docs.run.pivotal.io/cf-cli/install-go-cli.html.

[2] Scaling Loggregator: http://docs.pivotal.io/pivotalcf/1-7/opsguide/logging-config-opsman.html#scaling

[3] Firehose-to-syslog: https://github.com/cloudfoundry-community/firehose-to-syslog

Comments

Powered by Zendesk