CI: capture test output as artifact#433
Merged
Merged
Conversation
CircleCI log output is truncated, hiding the actual `ERROR in (...)` line when flaky tests fail. Save the full tee'd log under target/ and upload it via store_artifacts so we can download the unmunged output and pinpoint the failing test.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Currently CircleCI's job-log view is truncated, so when a flaky test errors we only see the printed exception traces from
maybe-log-exception(which are caught — red herrings) and never the actualERROR in (test-name)line that produced the1 errorscount.Tee the test output to
target/test-output.logandstore_artifactsit. UsesPIPESTATUSso a tee'd failure still fails the step, andwhen: alwaysso the artifact uploads even on failure. Next time the suggest-libspecs flake fires, we can download the log and grep for the real failing test.