Skip to content

Generate test failure output on abnormal termination - #1011

Merged
Kenny Kerr (kennykerr) merged 2 commits into
masterfrom
user/dfields/report-fatal
Sep 8, 2021
Merged

Generate test failure output on abnormal termination#1011
Kenny Kerr (kennykerr) merged 2 commits into
masterfrom
user/dfields/report-fatal

Conversation

@dfields-msft

Copy link
Copy Markdown
Contributor

Fixes #1010 by calling Catch2's reportFatal() in the std::terminate handler so that failure logging is generated before the process exits.

@kennykerr

Copy link
Copy Markdown
Collaborator

Thanks David! Why don't you add the noexcept back to the guid constructor that I removed here:

https://github.com/microsoft/cppwinrt/pull/1009/files

That will verify whether the CI build fails visibly or silently. Once that has been confirmed, just remove the noexcept again and it should be green again.

@kennykerr

Copy link
Copy Markdown
Collaborator

The log captures the failure but it doesn't fail the build itself.

image

@dfields-msft

Copy link
Copy Markdown
Contributor Author

It's worse than that - even normal test failures don't cause the pipeline to fail (see my latest commit in this PR and the corresponding run). I've sent email to the team to discuss.

@dfields-msft

Copy link
Copy Markdown
Contributor Author

Added some changes to reroute output to stderr when the test executable does not exit cleanly, and validated that this fails the pipeline with the noexcept in place as expected:
image

@kennykerr Kenny Kerr (kennykerr) left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@dfields-msft

Copy link
Copy Markdown
Contributor Author

Logged #1012 on an unrelated test failure that passed on rerun (also passed on both x86 runs); this may have been masked for some time due to the lack of failure reporting in the build pipeline until now.

@kennykerr
Kenny Kerr (kennykerr) merged commit 3aa0339 into master Sep 8, 2021
@kennykerr
Kenny Kerr (kennykerr) deleted the user/dfields/report-fatal branch September 8, 2021 00:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Unit tests that terminate abnormally are not considered as failures

2 participants