Automate coverage reporting #205
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
WalkthroughAdds two GitHub Actions workflows: one to generate and archive LCOV coverage artifacts on pushes/PRs, and another to publish the artifact to gluesql.github.io and comment on PRs with a link. The publish workflow can be triggered by the coverage workflow’s completion or manually. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
actor Dev as Developer
participant GH as GitHub Actions
participant Repo as gluesql/glues
participant Cov as Coverage WF
participant Art as Artifact Store
participant Pages as gluesql.github.io (gh-pages)
Dev->>Repo: Push / PR to main
GH->>Cov: Trigger "Coverage"
Cov->>Repo: Checkout
Cov->>Cov: cargo llvm-cov (workspace) -> lcov.info
Cov->>Cov: Compress lcov.info -> lcov.info.xz (timestamp)
Cov->>Art: Upload artifact (coverage)
alt push to main on gluesql/glues
Cov->>Pages: Clone gh-pages, commit lcov.info.xz under coverage/glues/main
Cov->>Pages: Push with GitHub App token
end
sequenceDiagram
autonumber
participant GH as GitHub Actions
participant Pub as Publish Coverage WF
participant Art as Artifact Store
participant Pages as gluesql.github.io (gh-pages)
participant PR as Pull Request
GH->>Pub: Trigger on workflow_run(Completion of Coverage) or manual_dispatch
Pub->>Art: Download artifact (default: coverage)
Pub->>Pub: Determine RUN_ID, PR_NUMBER, COMMIT_SHA, TIMESTAMP
Pub->>Pages: Clone gh-pages, add file under coverage/glues/pr/<PR>/<TIMESTAMP>.<SHA>.lcov.info.xz
Pub->>Pages: Commit & push via GitHub App token
Pub->>PR: Create/Update comment with report URL (gluesql.org/coverage/?path=...)
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Poem
Pre-merge checks and finishing touches✅ Passed checks (3 passed)
✨ Finishing touches🧪 Generate unit tests
Tip 👮 Agentic pre-merge checks are now available in preview!Pro plan users can now enable pre-merge checks in their settings to enforce checklists before merging PRs.
Please see the documentation for more information. Example: reviews:
pre_merge_checks:
custom_checks:
- name: "Undocumented Breaking Changes"
mode: "warning"
instructions: |
Pass/fail criteria: All breaking changes to public APIs, CLI flags, environment variables, configuration keys, database schemas, or HTTP/GraphQL endpoints must be documented in the "Breaking Change" section of the PR description and in CHANGELOG.md. Exclude purely internal or private changes (e.g., code not exported from package entry points or explicitly marked as internal).Please share your feedback with us on this Discord post. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://waps.l3s.uni-hannover.de/live/im_/https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (8)
.github/workflows/coverage.yml (4)
54-65: Avoid push races to gh-pages (rebase before commit).Add a rebase pull before staging to reduce non-fast-forward failures.
- name: Publish coverage to gluesql.github.io if: github.repository == 'gluesql/glues' && github.event_name == 'push' && github.ref == 'refs/heads/main' run: | git clone https://github.com/gluesql/gluesql.github.io.git cd gluesql.github.io git checkout gh-pages + git pull --rebase origin gh-pages mkdir -p coverage/glues/main cp ../coverage/lcov.info.xz coverage/glues/main/${TIMESTAMP}.${COMMIT_SHA}.lcov.info.xz git add coverage/glues/main/${TIMESTAMP}.${COMMIT_SHA}.lcov.info.xz git commit -m "Coverage: main@${COMMIT_SHA}" git push https://x-access-token:${{ steps.app-token.outputs.token }}@github.com/gluesql/gluesql.github.io.git
12-17: Add job-level concurrency to auto-cancel superseded runs.Prevents redundant coverage runs per ref/PR.
jobs: coverage: + concurrency: + group: coverage-${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }} + cancel-in-progress: true runs-on: ubuntu-latest env: COMMIT_SHA: ${{ github.event.pull_request.head.sha || github.sha }}
35-39: Compression level nit: speed vs size.
-9eis expensive for marginal gains. Consider-6(default) or-3to save CI minutes.- xz -T0 -9e -c coverage/lcov.info > coverage/lcov.info.xz + xz -T0 -6 -c coverage/lcov.info > coverage/lcov.info.xz
30-34: Cache Cargo artifacts to cut CI time.Add a simple cargo target/registry cache.
- name: Generate coverage report run: | mkdir -p coverage cargo llvm-cov --workspace --lcov --output-path coverage/lcov.info + - name: Cache Cargo registry and target + uses: actions/cache@v4 + with: + path: | + ~/.cargo/registry + ~/.cargo/git + target + key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }} + restore-keys: | + ${{ runner.os }}-cargo-.github/workflows/publish-coverage.yml (4)
28-36: Add concurrency to prevent clobbering when multiple runs publish.Scope by PR number for auto-cancel on newer runs.
jobs: publish: + concurrency: + group: publish-coverage-${{ github.event_name == 'workflow_dispatch' && inputs.pr_number || github.event.workflow_run.pull_requests[0].number || github.run_id }} + cancel-in-progress: true
64-71: Guard against missing PR array in workflow_run payload.Rare but possible; add a sanity check fallback to avoid empty
PR_NUMBER.- name: Set variables for workflow_run if: github.event_name == 'workflow_run' run: | - echo "RUN_ID=${{ github.event.workflow_run.id }}" >> $GITHUB_ENV - echo "COMMIT_SHA=${{ github.event.workflow_run.head_sha }}" >> $GITHUB_ENV - echo "PR_NUMBER=${{ github.event.workflow_run.pull_requests[0].number }}" >> $GITHUB_ENV + echo "RUN_ID=${{ github.event.workflow_run.id }}" >> $GITHUB_ENV + echo "COMMIT_SHA=${{ github.event.workflow_run.head_sha }}" >> $GITHUB_ENV + echo "PR_NUMBER=${{ github.event.workflow_run.pull_requests && github.event.workflow_run.pull_requests[0] && github.event.workflow_run.pull_requests[0].number || '' }}" >> $GITHUB_ENV echo "ARTIFACT_NAME=coverage" >> $GITHUB_ENVIf you prefer stricter handling, fail fast when
PR_NUMBERis empty and print the event JSON for debugging.
83-94: Add a lightweight push retry to handle concurrent updates.Prevents transient failures when multiple jobs commit to gh-pages.
- name: Publish coverage to gluesql.github.io run: | git clone https://github.com/gluesql/gluesql.github.io.git cd gluesql.github.io git checkout gh-pages git pull --rebase origin gh-pages mkdir -p coverage/glues/pr/${PR_NUMBER} cp ../coverage/lcov.info.xz coverage/glues/pr/${PR_NUMBER}/${TIMESTAMP}.${COMMIT_SHA}.lcov.info.xz git add coverage/glues/pr/${PR_NUMBER}/${TIMESTAMP}.${COMMIT_SHA}.lcov.info.xz git commit -m "Coverage: PR#${PR_NUMBER}@${COMMIT_SHA}" - git push https://x-access-token:${{ steps.app-token.outputs.token }}@github.com/gluesql/gluesql.github.io.git + for i in 1 2 3; do + if git push https://x-access-token:${{ steps.app-token.outputs.token }}@github.com/gluesql/gluesql.github.io.git; then + break + fi + git pull --rebase origin gh-pages + sleep $((i*2)) + done
95-134: Make PR comment idempotent with a stable hidden marker.Use a hidden HTML marker to find/update the bot comment reliably.
- const body = [ - '### Glues Coverage Report', + const marker = '<!-- glues-coverage-report -->'; + const body = [ + marker, + '### Glues Coverage Report', '', `- **Commit:** \`${process.env.COMMIT_SHA}\``, `- **Timestamp:** \`${process.env.TIMESTAMP}\``, `- **Report:** [View report](${url})` ].join('\n'); ... - const comment = comments.find(c => c.body.startsWith('### Glues Coverage Report')); + const comment = comments.find(c => c.body.includes('<!-- glues-coverage-report -->'));
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
.github/workflows/coverage.yml(1 hunks).github/workflows/publish-coverage.yml(1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: coverage
Comment on lines
+18
to
+19
| - uses: actions/checkout@v4 | ||
|
|
There was a problem hiding this comment.
Pin third-party Actions to commit SHAs (supply‑chain hardening).
Pin actions/checkout@v4, taiki-e/install-action@v2, and actions/upload-artifact@v4 to immutable commit SHAs.
Example (replace COMMIT_SHA placeholders with real SHAs):
- - uses: actions/checkout@v4
+ - uses: actions/checkout@v4@COMMIT_SHA
- - name: Install cargo-llvm-cov
- uses: taiki-e/install-action@v2
+ - name: Install cargo-llvm-cov
+ uses: taiki-e/install-action@v2@COMMIT_SHA
- - name: Upload coverage artifact
- uses: actions/upload-artifact@v4
+ - name: Upload coverage artifact
+ uses: actions/upload-artifact@v4@COMMIT_SHAAlso applies to: 25-29, 66-70
🤖 Prompt for AI Agents
.github/workflows/coverage.yml around lines 18-19 (also apply same change at
25-29 and 66-70): the workflow uses third‑party GitHub Actions by tag (e.g.,
actions/checkout@v4, taiki-e/install-action@v2, actions/upload-artifact@v4)
which should be pinned to immutable commit SHAs; replace each action tag with
the corresponding full commit SHA reference (e.g.,
actions/checkout@<COMMIT_SHA>) across the specified line ranges, updating any
uses steps to reference the exact SHA, and verify the SHAs are current by
checking the action repos and using the latest stable commit SHA for each
action.
Comment on lines
+46
to
+55
| - name: Create GitHub App token | ||
| id: app-token | ||
| uses: actions/create-github-app-token@v1 | ||
| with: | ||
| app-id: ${{ secrets.COVERAGE_APP_ID }} | ||
| installation-id: ${{ secrets.COVERAGE_APP_INSTALLATION_ID }} | ||
| private-key: ${{ secrets.COVERAGE_APP_PRIVATE_KEY }} | ||
| owner: gluesql | ||
| repositories: glues, gluesql.github.io | ||
|
|
There was a problem hiding this comment.
Pin third-party Actions to commit SHAs (supply‑chain hardening).
Pin actions/create-github-app-token@v1, actions/download-artifact@v4, and actions/github-script@v7 to immutable SHAs.
- - name: Create GitHub App token
- uses: actions/create-github-app-token@v1
+ - name: Create GitHub App token
+ uses: actions/create-github-app-token@v1@COMMIT_SHA
...
- - name: Download coverage artifact
- uses: actions/download-artifact@v4
+ - name: Download coverage artifact
+ uses: actions/download-artifact@v4@COMMIT_SHA
...
- - name: Comment coverage link on PR
- uses: actions/github-script@v7
+ - name: Comment coverage link on PR
+ uses: actions/github-script@v7@COMMIT_SHAAlso applies to: 72-79, 95-97
🤖 Prompt for AI Agents
.github/workflows/publish-coverage.yml lines 46-55 (and also update same pattern
at 72-79 and 95-97): currently third‑party actions are referenced by floating
tags (e.g. actions/create-github-app-token@v1); replace those tag references
with the corresponding immutable commit SHAs for each action (pin to the
specific full commit SHA) to harden the supply chain. For each uses: entry
(actions/create-github-app-token, actions/download-artifact,
actions/github-script) look up the latest tested commit SHA in the upstream
action repo, replace the @vX tag with @<full-commit-sha>, and ensure the
workflow still passes CI after the change.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Testing
Summary by CodeRabbit
New Features
Chores