Pivotal Knowledge Base

Follow

Sending emails via 'notifications' service fails

Environment

Pivotal Cloud Foundry® (PCF) 1.4.x

Symptom

When an invitation for a new user is sent via the app-manager/console UI, the user does not get an email notification.

Looking at the logs via cf cli, we see the following errors :

-> cf logs notifications --recent

--snip-- 2015-06-24T16:04:56.86-0500 [App/0] ERR [MySQL] 2015/06/24 21:04:56 packets.go:118: write tcp 10.17.128.51:3306: broken pipe 2015-06-24T16:04:56.88-0500 [RTR/0] OUT notifications.xyz.com - [24/06/2015:21:04:56 +0000] "POST /emails HTTP/1.1" 200 117 "-" "Typhoeus - https://github.com/typhoeus/typhoeus" 10.17.128.250:33931 x_forwarded_for:"10.17.128.254" vcap_request_id:8b5845e9-0380-4a1f-7577-6162547ad283 response_time:0.031032076 app_id:a15abe61-9666-4b87-bc15-37ca534098b3 2015-06-24T16:04:56.96-0500 [App/0] OUT [WEB] 2015/06/24 21:04:56 Attempting to deliver message to johndoe@xyz.com 2015-06-24T16:04:56.96-0500 [App/0] OUT [WEB] 2015/06/24 21:04:56 Failed to deliver message due to SMTP error: x509: certificate is valid for mail.xyz.com, not relaymail.xyz.com 2015-06-24T16:04:56.96-0500 [App/0] OUT [WEB] 2015/06/24 21:04:56 Message failed to send, retrying at: Jun 24, 2015 at 9:05pm (UTC) 2015-06-24T16:04:56.96-0500 [App/0] OUT [METRIC] {"kind":"counter","payload":{"name":"notifications.worker.retry"}}

From the above logs, it appears that the smtp is looking for a valid SSL certificate, however from the env variable for notifications app 'VERIFY_SSL' is set to false. Checking the env variables for 'notifications' app :

-> cf env notifications

--snip--
ser-Provided:
CC_HOST: https://api.system.abc.xyz.com
DATABASE_URL: mysql://f0665babee1b5a1e6a88:f3d36a4a0826f05893af@10.0.1.46:3306/notifications
DOMAIN: system.abc.xyz.com
ENCRYPTION_KEY: a1a777d2ea66476fe99d
GOBBLE_MIGRATIONS_DIR: $HOME/gobble/migrations
ROOT_PATH: $HOME
SENDER: google@gmail.com
SMTP_AUTH_MECHANISM: none
SMTP_CRAMMD5_SECRET:
SMTP_HOST: smtp.sendgrid.net
SMTP_PASS: abc.xyz
SMTP_PORT: 587
SMTP_TLS: true
SMTP_USER: krashpanic
UAA_CLIENT_ID: notifications
UAA_CLIENT_SECRET: fbb72bce255561f08930
UAA_HOST: https://uaa.system.abc.xyz.com
VERIFY_SSL: false

Cause
This is a bug in the notification app where the SMTP client ignores the "VERIFY_SSL" set to "false" environment variable and enforces having a valid SSL certificate. 

Resolution

This is a known issue in PCF v1.4, the fix is planned to available in PCF v1.5.1 release.

If you are adding new users via the app-manager/console, the workaround is to manually send the user registration link. The other fix is to provide a valid SSL certificate on the SMTP server.

Comments

Powered by Zendesk