chore(ci): migrate compatibility indexes from git branch to GitHub Releases (0.18) by LHT129 · Pull Request #1756 · antgroup/vsag

LHT129 · 2026-03-25T09:41:58Z

Summary

Backport of #1692 and #1708 to 0.18 branch.

Migrate compatibility test index files from the old_version_index git branch to GitHub Releases for improved CI performance and maintainability.

Changes

Modified generate_old_version_index.yml to upload indexes to Release instead of creating PR
Modified check_compatibility.yml to download indexes from Release instead of git clone
Removed git clone of old_version_index branch (961 MB)
Added gh release download/upload commands for faster CI execution
Add --prerelease flag when creating new release
Improved shell script safety with bash arrays

Expected Improvements

Metric	Before	After	Improvement
CI time	6-10 min	3.5-6.5 min	40-50% faster
Clone size	961 MB	~300 MB	70% smaller
Repository growth	Continuous	None	Eliminated

Related Issues

…leases - Modify generate_old_version_index.yml to upload indexes to Release - Modify check_compatibility.yml to download indexes from Release - Remove git clone of old_version_index branch (961 MB) - Use gh release download/upload for faster CI execution - Add --prerelease flag when creating new release - Improve shell script safety with bash arrays Backport of antgroup#1692 and antgroup#1708 to 0.18 branch Signed-off-by: LHT129 <tianlan.lht@antgroup.com>

gemini-code-assist · 2026-03-25T09:42:04Z

Note

Gemini is unable to generate a summary for this pull request due to the file types involved not being currently supported.

Copilot

Pull request overview

Migrates compatibility test index artifacts from a dedicated git branch (old_version_index) to GitHub Release assets to reduce CI clone size and speed up compatibility workflows on the 0.18 branch.

Changes:

Generate workflow now uses gh release create/upload instead of opening PRs to an index branch.
Compatibility check workflow now downloads index assets from a Release instead of cloning a heavy branch.
Adds GitHub CLI installation steps inside CI containers to support Release asset operations.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File	Description
.github/workflows/generate_old_version_index.yml	Replaces index-branch checkout + PR creation with Release asset download/upload flow.
.github/workflows/check_compatibility.yml	Replaces `old_version_index` branch clone with Release asset download into `/tmp`.

Copilot · 2026-03-25T09:49:42Z

.github/workflows/check_compatibility.yml

+          gh release download compatibility-indexes \
+            --repo antgroup/vsag \
+            --dir /tmp \
+            --pattern "*.index" \
+            --pattern "*.bin" \
+            --pattern "*.json" \
+            || echo "Warning: Failed to download compatibility indexes"


The gh release download failure is intentionally ignored, so the workflow may proceed to run compatibility checks with missing/partial assets and produce misleading results (false failures or false passes depending on script behavior). Consider failing the step when download fails, or explicitly validating the presence of required assets in /tmp and exiting non-zero if they are missing.

Suggested change

gh release download compatibility-indexes \

--repo antgroup/vsag \

--dir /tmp \

--pattern "*.index" \

--pattern "*.bin" \

--pattern "*.json" \

|| echo "Warning: Failed to download compatibility indexes"

if ! gh release download compatibility-indexes \

--repo antgroup/vsag \

--dir /tmp \

--pattern "*.index" \

--pattern "*.bin" \

--pattern "*.json"; then

echo "Warning: Failed to download compatibility indexes"

exit 1

fi

Copilot · 2026-03-25T09:49:43Z

.github/workflows/generate_old_version_index.yml

+          release_error=$(gh release view compatibility-indexes --repo antgroup/vsag 2>&1)
+          if [ $? -eq 0 ]; then
+            echo "Uploading to existing release..."
+            if ! gh release upload compatibility-indexes "${files[@]}" --repo antgroup/vsag --clobber; then
+              echo "Error: Failed to upload files to release"
+              exit 1
+            fi
+          else
+            if echo "$release_error" | grep -qi "404\|not found\|notfound"; then
+              echo "Creating new release..."
+              if ! gh release create compatibility-indexes "${files[@]}" \
+                --repo antgroup/vsag \
+                --title "VSAG Compatibility Test Indexes" \
+                --notes "Index files for backward compatibility testing" \
+                --prerelease; then
+                echo "Error: Failed to create release"
+                exit 1
+              fi
+            else
+              echo "Error checking release existence: $release_error"


There’s a TOCTOU race between gh release view and gh release create: two concurrent runs can both observe “not found” and then one create will fail because the release was created by the other run. Make the logic idempotent by handling the “already exists” failure from gh release create and then retrying with gh release upload, or by attempting upload first and creating only if the upload indicates the release/tag is missing.

Suggested change

release_error=$(gh release view compatibility-indexes --repo antgroup/vsag 2>&1)

if [ $? -eq 0 ]; then

echo "Uploading to existing release..."

if ! gh release upload compatibility-indexes "${files[@]}" --repo antgroup/vsag --clobber; then

echo "Error: Failed to upload files to release"

exit 1

fi

else

if echo "$release_error" | grep -qi "404\|not found\|notfound"; then

echo "Creating new release..."

if ! gh release create compatibility-indexes "${files[@]}" \

--repo antgroup/vsag \

--title "VSAG Compatibility Test Indexes" \

--notes "Index files for backward compatibility testing" \

--prerelease; then

echo "Error: Failed to create release"

exit 1

fi

else

echo "Error checking release existence: $release_error"

echo "Attempting to upload assets to release 'compatibility-indexes'..."

upload_output=$(gh release upload compatibility-indexes "${files[@]}" --repo antgroup/vsag --clobber 2>&1)

upload_status=$?

if [ $upload_status -eq 0 ]; then

echo "Successfully uploaded ${#files[@]} file(s) to existing release"

else

if echo "$upload_output" | grep -qi "404\|not found\|notfound"; then

echo "Release 'compatibility-indexes' not found, creating new release..."

create_output=$(gh release create compatibility-indexes "${files[@]}" \

--repo antgroup/vsag \

--title "VSAG Compatibility Test Indexes" \

--notes "Index files for backward compatibility testing" \

--prerelease 2>&1)

create_status=$?

if [ $create_status -eq 0 ]; then

echo "Successfully created release and uploaded ${#files[@]} file(s)"

else

if echo "$create_output" | grep -qi "already exists\|exists"; then

echo "Release already exists (detected race), retrying upload..."

if ! gh release upload compatibility-indexes "${files[@]}" --repo antgroup/vsag --clobber; then

echo "Error: Failed to upload files to existing release after create race"

echo "$create_output"

exit 1

fi

echo "Successfully uploaded ${#files[@]} file(s) to existing release after race"

else

echo "Error: Failed to create release: $create_output"

exit 1

fi

fi

else

echo "Error: Failed to upload files to release: $upload_output"

Copilot · 2026-03-25T09:49:43Z

.github/workflows/generate_old_version_index.yml

+      - name: Install GitHub CLI
+        run: |
+          (type -p wget >/dev/null || (apt update && apt-get install wget -y)) \
+          && mkdir -p -m 755 /etc/apt/keyrings \
+          && wget -qO- https://cli.github.com/packages/githubcli-archive-keyring.gpg | tee /etc/apt/keyrings/githubcli.gpg >/dev/null \
+          && chmod go+r /etc/apt/keyrings/githubcli.gpg \
+          && echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/githubcli.gpg] https://cli.github.com/packages stable main" | tee /etc/apt/sources.list.d/github-cli.list >/dev/null \
+          && apt update \
+          && apt install gh -y


This workflow adds a new external apt repository and imports its signing key at runtime without any explicit verification (key fingerprint / checksum), which increases supply-chain risk in CI. Prefer using a pinned/verified installation method (e.g., downloading a specific gh release artifact with checksum verification, or a vetted setup action) to reduce the attack surface and improve reproducibility.

Suggested change

- name: Install GitHub CLI

run: |

(type -p wget >/dev/null || (apt update && apt-get install wget -y)) \

&& mkdir -p -m 755 /etc/apt/keyrings \

&& wget -qO- https://cli.github.com/packages/githubcli-archive-keyring.gpg | tee /etc/apt/keyrings/githubcli.gpg >/dev/null \

&& chmod go+r /etc/apt/keyrings/githubcli.gpg \

&& echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/githubcli.gpg] https://cli.github.com/packages stable main" | tee /etc/apt/sources.list.d/github-cli.list >/dev/null \

&& apt update \

&& apt install gh -y

- name: Setup GitHub CLI

uses: cli/cli@v2

with:

# Pin to a specific GitHub CLI version for reproducibility

version: '2.63.1'

Copilot · 2026-03-25T09:49:43Z

.github/workflows/check_compatibility.yml

+      - name: Install GitHub CLI
        run: |
-          git clone -b old_version_index https://github.com/antgroup/vsag.git
-          cp -r ./vsag/* /tmp/
+          (type -p wget >/dev/null || (apt update && apt-get install wget -y)) \
+          && mkdir -p -m 755 /etc/apt/keyrings \
+          && wget -qO- https://cli.github.com/packages/githubcli-archive-keyring.gpg | tee /etc/apt/keyrings/githubcli.gpg >/dev/null \
+          && chmod go+r /etc/apt/keyrings/githubcli.gpg \
+          && echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/githubcli.gpg] https://cli.github.com/packages stable main" | tee /etc/apt/sources.list.d/github-cli.list >/dev/null \
+          && apt update \
+          && apt install gh -y


The GitHub CLI installation block is duplicated across multiple workflows in this PR. To reduce drift and simplify future changes, consider extracting it into a reusable composite action (e.g., .github/actions/setup-gh) or a shared workflow (workflow_call) so updates (version pinning, repo URL changes, hardening) only need to be made once.

wxyucs

lgtm

codecov · 2026-03-25T11:31:28Z

Codecov Report

✅ All modified and coverable lines are covered by tests.

@@            Coverage Diff             @@
##             0.18    #1756      +/-   ##
==========================================
+ Coverage   90.73%   91.15%   +0.41%     
==========================================
  Files         328      328              
  Lines       19327    19327              
==========================================
+ Hits        17537    17618      +81     
+ Misses       1790     1709      -81

Flag	Coverage Δ
cpp	`91.15% <ø> (+0.41%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
common	`85.81% <ø> (ø)`
datacell	`93.37% <ø> (+0.18%)`	⬆️
index	`90.77% <ø> (+0.46%)`	⬆️
simd	`100.00% <ø> (ø)`

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3da8349...f9a6258. Read the comment docs.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

inabao

LGTM

LHT129 requested a review from wxyucs as a code owner March 25, 2026 09:41

Copilot AI review requested due to automatic review settings March 25, 2026 09:41

pull-request-size bot added the size/L label Mar 25, 2026

Copilot AI reviewed Mar 25, 2026

View reviewed changes

Copilot started reviewing on behalf of LHT129 March 25, 2026 11:20 View session

wxyucs approved these changes Mar 25, 2026

View reviewed changes

inabao approved these changes Mar 26, 2026

View reviewed changes

LHT129 merged commit 06129c5 into antgroup:0.18 Mar 26, 2026
36 of 38 checks passed

LHT129 mentioned this pull request Mar 30, 2026

chore(ci): migrate compatibility indexes from git branch to GitHub Releases (0.15) #1775

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore(ci): migrate compatibility indexes from git branch to GitHub Releases (0.18)#1756

chore(ci): migrate compatibility indexes from git branch to GitHub Releases (0.18)#1756
LHT129 merged 1 commit intoantgroup:0.18from
LHT129:0.18-cp-1692

LHT129 commented Mar 25, 2026

Uh oh!

gemini-code-assist bot commented Mar 25, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Mar 25, 2026

Uh oh!

Copilot AI Mar 25, 2026

Uh oh!

Copilot AI Mar 25, 2026

Uh oh!

Copilot AI Mar 25, 2026

Uh oh!

wxyucs left a comment

Uh oh!

codecov bot commented Mar 25, 2026 •

edited

Loading

Uh oh!

inabao left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

-          release_error=$(gh release view compatibility-indexes --repo antgroup/vsag 2>&1)
-          if [ $? -eq 0 ]; then
-            echo "Uploading to existing release..."
-            if ! gh release upload compatibility-indexes "${files[@]}" --repo antgroup/vsag --clobber; then
-              echo "Error: Failed to upload files to release"
-              exit 1
-            fi
-          else
-            if echo "$release_error" | grep -qi "404\|not found\|notfound"; then
-              echo "Creating new release..."
-              if ! gh release create compatibility-indexes "${files[@]}" \
-                --repo antgroup/vsag \
-                --title "VSAG Compatibility Test Indexes" \
-                --notes "Index files for backward compatibility testing" \
-                --prerelease; then
-                echo "Error: Failed to create release"
-                exit 1
-              fi
-            else
-              echo "Error checking release existence: $release_error"
+          echo "Attempting to upload assets to release 'compatibility-indexes'..."
+          upload_output=$(gh release upload compatibility-indexes "${files[@]}" --repo antgroup/vsag --clobber 2>&1)
+          upload_status=$?
+          if [ $upload_status -eq 0 ]; then
+            echo "Successfully uploaded ${#files[@]} file(s) to existing release"
+          else
+            if echo "$upload_output" | grep -qi "404\|not found\|notfound"; then
+              echo "Release 'compatibility-indexes' not found, creating new release..."
+              create_output=$(gh release create compatibility-indexes "${files[@]}" \
+                --repo antgroup/vsag \
+                --title "VSAG Compatibility Test Indexes" \
+                --notes "Index files for backward compatibility testing" \
+                --prerelease 2>&1)
+              create_status=$?
+              if [ $create_status -eq 0 ]; then
+                echo "Successfully created release and uploaded ${#files[@]} file(s)"
+              else
+                if echo "$create_output" | grep -qi "already exists\|exists"; then
+                  echo "Release already exists (detected race), retrying upload..."
+                  if ! gh release upload compatibility-indexes "${files[@]}" --repo antgroup/vsag --clobber; then
+                    echo "Error: Failed to upload files to existing release after create race"
+                    echo "$create_output"
+                    exit 1
+                  fi
+                  echo "Successfully uploaded ${#files[@]} file(s) to existing release after race"
+                else
+                  echo "Error: Failed to create release: $create_output"
+                  exit 1
+                fi
+              fi
+            else
+              echo "Error: Failed to upload files to release: $upload_output"

Conversation

LHT129 commented Mar 25, 2026

Summary

Changes

Expected Improvements

Related Issues

Uh oh!

gemini-code-assist bot commented Mar 25, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

wxyucs left a comment

Choose a reason for hiding this comment

Uh oh!

codecov bot commented Mar 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

inabao left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

codecov bot commented Mar 25, 2026 •

edited

Loading