Skip to content

chore: reduce flakiness for test api equivalence spec#6955

Merged
PeterSchafer merged 1 commit into
mainfrom
chore/reduce_flakiness
Jun 29, 2026
Merged

chore: reduce flakiness for test api equivalence spec#6955
PeterSchafer merged 1 commit into
mainfrom
chore/reduce_flakiness

Conversation

@CatalinSnyk

Copy link
Copy Markdown
Contributor

Pull Request Submission Checklist

  • Follows CONTRIBUTING guidelines
  • Commit messages
    are release-note ready, emphasizing
    what was changed, not how.
  • Includes detailed description of changes
  • Contains risk assessment (Low | Medium | High)
  • Highlights breaking API changes (if applicable)
  • Links to automated tests covering new functionality
  • Includes manual testing instructions (if necessary)
  • Updates relevant GitBook documentation (PR link: ___)
  • Includes product update to be announced in the next stable release notes

What does this PR do?

Swap out the "localhost" for the IPv4 address to reduce the flakiness for the test-api-equivalence acceptance tests.

Where should the reviewer start?

  • test/jest/acceptance/snyk-test/unified-test-api-equivalence.spec.ts

How should this be manually tested?

  • Wait for CI to pass - the changes should be aligned with some other acceptance tests that make use of this pattern.

What's the product update that needs to be communicated to CLI users?

N/A - CI fix only.

@CatalinSnyk CatalinSnyk marked this pull request as ready for review June 26, 2026 17:04
@CatalinSnyk CatalinSnyk requested a review from a team as a code owner June 26, 2026 17:04
@snyk-io

snyk-io Bot commented Jun 26, 2026

Copy link
Copy Markdown

Snyk checks have passed. No issues have been found so far.

Status Scan Engine Critical High Medium Low Total (0)
Open Source Security 0 0 0 0 0 issues
Licenses 0 0 0 0 0 issues
Code Security 0 0 0 0 0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

@snyk-pr-review-bot

This comment has been minimized.

@@ -27,10 +30,11 @@ describe('snyk test — unified test API equivalence (FF off vs on)', () => {
beforeAll((done) => {
const port = getServerPort(process);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: why not exchange the deprecated method as well and use getAvailableServerPort()?

...process.env,
SNYK_API: 'http://localhost:' + port + baseApi,
SNYK_HOST: 'http://localhost:' + port,
SNYK_API: `http://${fakeServerIp}:${port}${baseApi}`,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: Is there any explanation why this change has an effect on flakiness?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general it is the right move to use this pattern but for other reasons then flakiness in my understanding.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it seems he's applying the pattern in other tests, so sounds correct.
For the flakiness, it might be the timeout increase to fix it:
Image

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was writing a comment here, it turned to be the same as this

For the flakiness, it might be the timeout increase to fix it

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, I added the timeout ;)

@PeterSchafer PeterSchafer force-pushed the chore/reduce_flakiness branch from 3a11961 to 5f57ac5 Compare June 26, 2026 18:22
@snyk-pr-review-bot

This comment has been minimized.

@PeterSchafer PeterSchafer force-pushed the chore/reduce_flakiness branch from 5f57ac5 to fbbf420 Compare June 26, 2026 18:57
@PeterSchafer PeterSchafer force-pushed the chore/reduce_flakiness branch from fbbf420 to a13259c Compare June 26, 2026 18:59
@snyk-pr-review-bot

This comment has been minimized.

@snyk-pr-review-bot

Copy link
Copy Markdown

PR Reviewer Guide 🔍

🧪 PR contains tests
🔒 No security concerns identified
⚡ No major issues detected
📚 Repository Context Analyzed

This review considered 3 relevant code sections from 3 files (average relevance: 1.00)

🤖 Repository instructions applied (from AGENTS.md)

@PeterSchafer PeterSchafer merged commit 96a13f7 into main Jun 29, 2026
10 checks passed
@PeterSchafer PeterSchafer deleted the chore/reduce_flakiness branch June 29, 2026 07:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants