Skip to content

Flex Consumption graceful-shutdown e2e (E18) depends on host shutdown log strings; find a more reliable signal #11864

Description

Summary

The Flex Consumption graceful-shutdown e2e test (E18) asserts that the .NET Functions Host ran its graceful-shutdown / telemetry-flush chain by grepping for specific host shutdown log strings. This coupling is fragile: if the host ever changes those log messages, the match will no longer succeed and the test will start failing (or need manual updating) even though the behavior it validates is still correct.

This came out of review of Forward SIGTERM to Functions Host process group on shutdown, which fixes intermittent tail-telemetry loss on instance recycle by forwarding SIGTERM to the host so its ApplicationStopping -> StopAsync/Dispose -> App Insights/OTel flush chain runs.

Where the coupling lives

The test lives in the AAPT-Antares-Functions-Docker repo, but the strings it depends on are owned by this host repo:

  • images/flexconsumption/e2e-test/e2e-harness.sh (test_E18_sigterm_forwarded_to_host), regex:
    Stopping host\.\.\.|Host shutdown completed\.|Host state changed from Stopping to Stopped
    
  • The same list is mirrored in prose at images/flexconsumption/components/appserver/cv2/graceful-shutdown-design.md.

If the host renames/reformats any of these shutdown log messages, the e2e assertion silently stops matching.

Severity

Low / non-blocking. The failure mode is safe: a changed log string makes the marker count fail to increase, so E18 fails loudly rather than producing a false pass. So this is about brittleness and maintenance, not a correctness gap.

Asks

  1. Evaluate more reliable signals for "the host shut down gracefully and flushed telemetry" that don't depend on exact log text, e.g.:
    • a stable framework-level marker (Microsoft.Hosting.Lifetime "Application is shutting down..."), or
    • the host process exit code (graceful SIGTERM shutdown exits 0 vs SIGKILL 137), or
    • a documented/contractual shutdown log line the host commits to keeping stable.
  2. In the meantime, keep a regression test in place so the graceful-shutdown/flush-on-SIGTERM behavior can't regress, and treat any of the strings above as a known dependency (update the e2e regex + design doc together if host shutdown logging changes).

References

  • PR tracked in internal issue link

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions