Pivotal Knowledge Base

Follow

ERT smoke tests error: "[Fail] Loggregator: cf logs linux [BeforeEach] can see app messages in the logs"

Environment

Pivotal Elastic Runtime: 1.10.x

Symptom

While running the Elastic Runtime deployment, the smoke tests errand times out while waiting for the CF logs output to appear from the newly started SMOKES app instance. 

Error Message

Summarizing 1 Failure:
[Fail] Loggregator: cf logs linux [BeforeEach] can see app messages in the logs
/var/vcap/packages/smoke-tests/src/github.com/cloudfoundry/cf-smoke-tests/smoke/logging/loggregator_test.go:32
Ran 1 of 2 Specs in 38.784 seconds
FAIL! -- 0 Passed | 1 Failed | 0 Pending | 1 Skipped --- FAIL: TestSmokeTests (38.78s)
FAIL
Ginkgo ran 1 suite in 41.901955019s
Test Suite Failed
Smoke Tests Complete; exit status: 1
[stderr]
+ command -v go
+ [[ -z /var/vcap/packages/smoke-tests ]]
+ export PATH=/var/vcap/packages/smoke-tests/bin:/var/vcap/packages/smoke-tests/bin:/var/vcap/packages/cli/bin:/var/vcap/data/packages/golang1.7/e220a4c112df4a98699b4c2f1c608ad25eb5f548/bin:/usr/sbin:/usr/bin:/sbin:/bin
+ PATH=/var/vcap/packages/smoke-tests/bin:/var/vcap/packages/smoke-tests/bin:/var/vcap/packages/cli/bin:/var/vcap/data/packages/golang1.7/e220a4c112df4a98699b4c2f1c608ad25eb5f548/bin:/usr/sbin:/usr/bin:/sbin:/bin
+ command -v ginkgo
+ ginkgo -r --succinct -slowSpecThreshold=300
Errand 'smoke-tests' completed with error (exit code 1)

Cause 

The app instance is dependant on the ruby_buildpack but it is takes too long to download it from the blobstore, however, the errand continues with its steps and eventually fails after it reports error during step 3

1. cf start app SMOKES-1-APP-XX

2. cf app SMOKES-1-APP-XX --guid

3. cf logs SMOKES-1-APP-XX --recent

Failure in Spec Setup (BeforeEach) [316.265 seconds]
Loggregator: cf logs linux [BeforeEach] can see app messages in the logs
/var/vcap/packages/smoke-tests/src/github.com/cloudfoundry/cf-smoke-tests/smoke/logging/loggregator_test.go:43 class="wysiwyg-text-align-left">Timed out after 300.000s.
Expected process to exit. It did not. class="wysiwyg-text-align-left">/var/vcap/packages/smoke-tests/src/github.com/cloudfoundry/cf-smoke-tests/smoke/logging/loggregator_test.go:33<4. cf delete SMOKES-1-APP-XX -f -r

4. cf delete SMOKES-1-APP-XX -f -r

Resolution

Follow the steps to resolve this issue:

  1. Run the push-apps-manager errand (confirm it also failed for a timeout)
  2. Confirm the blobstore is external to Pivotal Cloud Foundry (S3)
  3. Investigate possible slowness in retrieval from the blobstore
  4. One option could be to use S3 Browser to test the retrieval of a buildpack from the blobstore

Once you resolve the slow retrieval issue, the smoke tests should pass.

 

Comments

Powered by Zendesk