Pivotal Knowledge Base

Follow

PCF Advisory: Internal certificates expire 2 years after installation

Environment

Pivotal Cloud Foundry 1.6 and above

Purpose

Certificates in Cloud Foundry have a 2 year expiration period. Certificates should be regenerated within 2 years of installation. 

Cause 

Beginning in PCF 1.6, internal PCF components communicate using non-configurable certificates over TLS - this is a design choice in order to guarantee secure communication.

If a preexisting foundation has been running for over 2 years without certificate regeneration being performed, then PCF services will fail to start due to "Certificate has expired". This will impact applications running on the environment.

Procedure

For versions of PCF 1.6-1.8, please arrange an upgrade of system to 1.9+ immediately before certificate expiration occurs.

For versions of PCF 1.9+, the certificate expiration can be checked and regenerated via the OpsManager API.

Perform the steps:

  1. From your local machine, target your Ops Manager UAA server:

    $ uaac target https://OPS-MAN-FQDN/uaa
  2. If Operations Manager is not integrated with SAML and using local auth, then proceed with this step :

     

    $ uaac token owner get
    Client ID: opsman
    Client secret: [Leave Blank]
    User name: OPS-MAN-USERNAME (by default 'admin')
    Password: OPS-MAN-PASSWORD
    

    Replace OPS-MAN-USERNAME and OPS-MAN-PASSWORD with the credentials that you use to log in to the Ops Manager web interface.

    If Operations Manager is integrated with SAML, then proceed with this step and acquire a passcode from https://${OPSMANWEB}/uaa/passcode endpoint :

    uaac token sso get
    Client ID: opsman
    Client secret: <Leave Blank>
    Passcode: <paste from /uaa/passcode endpoint>
  3. List your tokens:
    $ uaac contexts
    
    Locate the entry for your Ops Manager FQDN. Under,client_id: opsman record the value for access_token.

  4. Use curl to make an API call to check for certificates expiring on the system within 6 months:

    $ curl "https://OPS-MAN-FQDN/api/v0/deployed/certificates?expires_within=6m" \
    -H "Authorization: Bearer YOUR-UAA-ACCESS-TOKEN"
  5. If certificates are expiring soon, then use curl to regenerate all non-configurable certificates and apply the new CA to your existing Ops Manager Director:

    $ curl "https://OPS-MAN-FQDN/api/v0/certificate_authorities/active/regenerate" \ 
    -X POST \ 
    -H "Authorization: Bearer YOUR-UAA-ACCESS-TOKEN" \ 
    -H "Content-Type: application/json" \ 
    -d '{}'
  6. Set the number of consul servers temporarily to 1 in OpsManager > Elastic Runtime > Resource Config. This can be scaled back once regeneration completes.

  7. Click Apply Changes in OpsManager to complete certificate regeneration.

Additional Information 

Upgrade steps for checking certificate expiration: https://docs.pivotal.io/pivotalcf/1-12/customizing/upgrading-pcf.html#cert-expiry

Article for fixing system with expired certificates: https://discuss.pivotal.io/hc/en-us/articles/115014034408-PCF-services-do-not-start-due-to-x509-certificate-has-expired-or-is-not-yet-valid

Reference on using OpsManager API https://docs.pivotal.io/pivotalcf/1-10/customizing/ops-man-api.html

Reference guide on rotating and regenerating certs in PCF https://docs.pivotal.io/pivotalcf/1-12/security/pcf-infrastructure/api-cert-rotation.html

Comments

Powered by Zendesk