Pivotal Knowledge Base


PCF Advisory- Internal Certificates Expires 2 Years After the Installation


Pivotal Cloud Foundry versions 1.6 and above.


Certificates in Cloud Foundry have a 2 year expiration period. Certificates should be regenerated within 2 years of installation. 


Beginning in PCF 1.6, internal PCF components communicate using non-configurable certificates over TLS - this is a design choice in order to guarantee secure communication.

If a preexisting foundation has been running for over 2 years without certificate regeneration being performed, then PCF services will fail to start due to "Certificate has expired". This will impact applications running in the environment.


PCF versions 1.6, 1.7, 1.8, 1.9, 1.10, 1.11, 1.12 and 2.0

Please arrange an upgrade to 1.9 or above immediately before certificate expiration occurs.

For detecting expiring certificates: You can simply check one of the certificates. (for example bbs_client_cert) There is no need to check all certificates because in every tile all internal certificates are generated at the same time.

Go to Ops Manager > Elastic Runtime > Credentials > BBS Client Cert, copy the string from “-----BEGIN CERTIFICATE-----” to “-----END CERTIFICATE-----”, execute the command from any Linux console:

 $ openssl x509 -startdate -enddate -noout -in <(printf -- "-----BEGIN CERTIFICATE-----\n… \n-----END CERTIFICATE-----")
notBefore=Dec 21 12:21:44 2017 GMT
notAfter=Dec 21 12:21:44 2019 GMT

PCF versions 1.9 and above

The certificate expiration can be checked and regenerated via the OpsManager API.

Perform the steps:

  1. From your local machine, target your Ops Manager UAA server:

    $ uaac target https://OPS-MAN-FQDN/uaa
  2. If Operations Manager is not integrated with SAML and using local auth, then proceed to this step :

    $ uaac token owner get Client ID: opsman Client secret: [Leave Blank] User name: OPS-MAN-USERNAME (by default 'admin') Password: OPS-MAN-PASSWORD

    Replace OPS-MAN-USERNAME and OPS-MAN-PASSWORD with the credentials that you use to log in to the Ops Manager web interface.

    If Operations Manager is integrated with SAML, then proceed with this step and acquire a passcode from https://${OPSMANWEB}/uaa/passcode endpoint :

    uaac token sso get
    Client ID: opsman
    Client secret: <Leave Blank>
    Passcode: <paste from /uaa/passcode endpoint>
  3. List your tokens:
    $ uaac contexts
    Locate the entry for your Ops Manager FQDN. Under,client_id: opsman record the value for access_token.

  4. Use curl to make an API call to check for certificates expiring on the system within 6 months:

    $ curl "https://OPS-MAN-FQDN/api/v0/deployed/certificates?expires_within=6m" \
    -H "Authorization: Bearer YOUR-UAA-ACCESS-TOKEN"
  5. If certificates are expiring soon, then use curl to regenerate all non-configurable certificates and apply the new CA to your existing Ops Manager Director:

    $ curl "https://OPS-MAN-FQDN/api/v0/certificate_authorities/active/regenerate" \ 
    -X POST \ 
    -H "Authorization: Bearer YOUR-UAA-ACCESS-TOKEN" \ 
    -H "Content-Type: application/json" \ 
    -d '{}'
  6. IMPORTANT Set the number of Consul servers temporarily to 1 in OpsManager > Elastic Runtime > Resource Config. This can be scaled back once the certificate regeneration completes.

  7. Click Apply Changes in OpsManager to complete certificate regeneration.

Additional Information 

Upgrade steps for checking certificate expiration: https://docs.pivotal.io/pivotalcf/1-12/customizing/upgrading-pcf.html#cert-expiry

Article for fixing system with expired certificates: https://discuss.pivotal.io/hc/en-us/articles/115014034408-PCF-services-do-not-start-due-to-x509-certificate-has-expired-or-is-not-yet-valid

Article for rotating ERT UAA certificate if SAML/SSO has been implemented: https://discuss.pivotal.io/hc/en-us/articles/115015717747-PCF-Advisory-SAML-Service-Provider-Credential-Certificates-Expire-after-2-Years

Reference on using OpsManager API https://docs.pivotal.io/pivotalcf/1-10/customizing/ops-man-api.html

Reference guide on rotating and regenerating certs in PCF https://docs.pivotal.io/pivotalcf/1-12/security/pcf-infrastructure/api-cert-rotation.html

NOTE: The certificate renewal will trigger application drain scripts, since each diego cell will be restarted. https://bosh.io/docs/job-lifecycle/ -- see the last section, When stop is issued (or before update and subsequent start happens).


Powered by Zendesk