Skip to content

Conversation

@jerm-dro
Copy link
Contributor

@jerm-dro jerm-dro commented Dec 12, 2025

Summary

This PR attempts to deflake:

  • test/e2e/thv-operator/virtualmcp/virtualmcp_auth_discovery_test.go
  • test/e2e/thv-operator/virtualmcp/virtualmcp_external_auth_test.go

Changes

  • Wait for backend servers to be fully deleted between steps
  • Configure readiness checks for backend pods that check if the pod is ready to accept traffic.
  • Add some logging for flakes like this:
VirtualMCPServer Unauthenticated Backend Auth when using unauthenticated backend auth [It] should discover, validate, and successfully use unauthenticated backend auth
/home/runner/work/toolhive/toolhive/test/e2e/thv-operator/virtualmcp/virtualmcp_external_auth_test.go:136

  [FAILED] Unexpected error:
      <*transport.Error | 0xc000706610>: 
      transport error: failed to send request: failed to send request: Post "http://localhost:30018/mcp": EOF
      {
          Err: <*fmt.wrapError | 0xc000710120>{
              msg: "failed to send request: failed to send request: Post \"http://localhost:30018/mcp\": EOF",
              err: <*fmt.wrapError | 0xc000710100>{
                  msg: "failed to send request: Post \"http://localhost:30018/mcp\": EOF",
                  err: <*url.Error | 0xc0007d0b10>{
                      Op: "Post",
                      URL: "http://localhost:30018/mcp",
                      Err: <*errors.errorString | 0x2598d00>{s: "EOF"},
                  },
              },
          },
      }
  occurred
  In [It] at: /home/runner/work/toolhive/toolhive/test/e2e/thv-operator/virtualmcp/virtualmcp_external_auth_test.go:209 @ 12/12/25 21:12:44.956

Which don't make a lot of sense, because they're immediately preceded by successful use of the same mcpClient.

Signed-off-by: Jeremy Drouillard <jeremy@stacklok.com>
@github-actions github-actions bot added the size/XS Extra small PR: < 100 lines changed label Dec 12, 2025
@codecov
Copy link

codecov bot commented Dec 12, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 56.44%. Comparing base (cc991e2) to head (541aa71).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #3024   +/-   ##
=======================================
  Coverage   56.43%   56.44%           
=======================================
  Files         333      333           
  Lines       33001    33001           
=======================================
+ Hits        18625    18626    +1     
+ Misses      12795    12793    -2     
- Partials     1581     1582    +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Signed-off-by: Jeremy Drouillard <jeremy@stacklok.com>
@github-actions github-actions bot added size/XS Extra small PR: < 100 lines changed and removed size/XS Extra small PR: < 100 lines changed labels Dec 12, 2025
Signed-off-by: Jeremy Drouillard <jeremy@stacklok.com>
@github-actions github-actions bot added size/XS Extra small PR: < 100 lines changed and removed size/XS Extra small PR: < 100 lines changed labels Dec 12, 2025
Signed-off-by: Jeremy Drouillard <jeremy@stacklok.com>
@github-actions github-actions bot added size/XS Extra small PR: < 100 lines changed and removed size/XS Extra small PR: < 100 lines changed labels Dec 12, 2025
@jerm-dro jerm-dro changed the title Maybe deflake tests Attempt to Deflake vMCP Tests Dec 12, 2025
Signed-off-by: Jeremy Drouillard <jeremy@stacklok.com>
@github-actions github-actions bot added size/XS Extra small PR: < 100 lines changed and removed size/XS Extra small PR: < 100 lines changed labels Dec 12, 2025
@jerm-dro jerm-dro marked this pull request as ready for review December 12, 2025 22:19
@jerm-dro jerm-dro merged commit 1310d3d into main Dec 12, 2025
33 checks passed
@jerm-dro jerm-dro deleted the jerm/maybe-deflake-tests branch December 12, 2025 23:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/XS Extra small PR: < 100 lines changed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants