WIP: Implement package fetch retries #30171

Draft

joshua-holmes wants to merge 10 commits from joshua-holmes/zig:master into master

pull from: joshua-holmes/zig:master

merge into: ziglang:master

ziglang:master

ziglang:elfv2-dyn

ziglang:powerpc64le

ziglang:jobserver

ziglang:threadtheft

ziglang:io-threaded-no-queue

ziglang:0.15.x

ziglang:Io.net

ziglang:comptime-allocator

ziglang:restricted-function-pointers

ziglang:cli

ziglang:wasm-linker-writer

ziglang:wrangle-writer-buffering

ziglang:sha1-stream

ziglang:async-await-demo

ziglang:fixes

ziglang:0.14.x

ziglang:ast-node-methods

ziglang:spork8

ziglang:macos-debug-info

ziglang:make-vs-configure

ziglang:fuzz-macos

ziglang:main

ziglang:sans-aro

ziglang:ArrayList-reserve

ziglang:incr-bug

ziglang:llvm-ir-nosanitize-metadata

ziglang:ci-tarballs

ziglang:ci-scripts

ziglang:threadpool

ziglang:0.12.x

ziglang:new-pkg-hash

ziglang:json-diagnostics

ziglang:more-doctests

ziglang:rework-comptime-mutation

ziglang:0.11.x

ziglang:ci-perf-comment

ziglang:stage2-async

ziglang:0.10.x

ziglang:autofix

ziglang:0.9.x

ziglang:aro

ziglang:hcs

ziglang:0.8.x

ziglang:0.7.x

Conversation 2 Commits 10 Files changed 4 +693 -128

joshua-holmes commented 2025-12-12 00:51:37 +01:00

First-time contributor

Copy link

The original PR is in GitHub.

Overview

Like in GitHub issue #17472, the package manager does not retry fetching when communication to the server fails in a way that could possibly be resolved by retrying the fetch.

Solution

• Default retries is 3 (total of 4 attempts)
  • For some HTTP failures (500+, 429, 404), try parsing the "Retry-After" header, which could be a number of seconds or an HTTP date string referring to a date in the future.
    • I used the MDN Retry-After docs and MDN Date docs to inform me on how to parse Retry-After. If parsing fails, we skip reading it.
    • Adheres to our 10 sec max, so we will never wait longer than that, even if "Retry-After" tells us to, just like in Cargo.
    • If "Retry-After" is set to a point in time in the past, we ignore the header and calculate our delay (with jitter) as normal.
  • If we don't use "Retry-After", first delay is a randomized value between 500ms and 1500ms
  • Subsequent delays are linear backoff 3000ms, 6000ms, etc up to a max of 10 seconds. Note that Cargo uses 3500 and 6500, etc instead. I just felt this was simpler.
• Classification of spurious (retryable) errors can be shown in the code linked here. This is a combination of what Cargo and uv thinks is spurious, plus my common sense. HTTP status codes 500+, 429, and 404 are considered spurious. Others are not.
• Extended lib/std/time/epoch.zig, which was necessary to convert date to epoch seconds

Testing

I unit tested quite a bit of the new code, and the tests pass. I also tested this locally with and without network failures, while logging before and after the delays to see that the number matched with real delay. I also tested this with a local endpoint that includes a "Retry-After" header. All works really well.

That being said, I have rebased a few times since doing all that testing and have only lightly tested my code since the rebases. Trying to keep my PR up to date is time consuming and testing it every time is more time consuming. But so far my light tests have worked and I'm happy to perform my thorough tests again at any time. I am doubtful the rebases have introduced a bug, though.

The [original PR is in GitHub](https://github.com/ziglang/zig/pull/25120). # Overview Like in [GitHub issue #17472](https://github.com/ziglang/zig/issues/17472), the package manager does not retry fetching when communication to the server fails in a way that could possibly be resolved by retrying the fetch. # Solution - Default retries is 3 (total of 4 attempts) - For some HTTP failures (500+, 429, 404), try parsing the "Retry-After" header, which could be a number of seconds or an HTTP date string referring to a date in the future. - I used the [MDN Retry-After docs](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Retry-After) and [MDN Date docs](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Date) to inform me on how to parse Retry-After. If parsing fails, we skip reading it. - Adheres to our 10 sec max, so we will never wait longer than that, even if "Retry-After" tells us to, just like in Cargo. - If "Retry-After" is set to a point in time in the past, we ignore the header and calculate our delay (with jitter) as normal. - If we don't use "Retry-After", first delay is a randomized value between 500ms and 1500ms - Subsequent delays are linear backoff 3000ms, 6000ms, etc up to a max of 10 seconds. Note that Cargo uses 3500 and 6500, etc instead. I just felt this was simpler. - Classification of spurious (retryable) errors can be shown in the [code linked here](https://github.com/ziglang/zig/blob/6b3f0c411ee1c3530cbb552e9e2a8ab4647fd949/src/Package/Fetch.zig#L384). This is a combination of what Cargo and uv thinks is spurious, plus my common sense. HTTP status codes 500+, 429, and 404 are considered spurious. Others are not. - Extended `lib/std/time/epoch.zig`, which was necessary to convert date to epoch seconds # Testing I unit tested quite a bit of the new code, and the tests pass. I also tested this locally with and without network failures, while logging before and after the delays to see that the number matched with real delay. I also tested this with a local endpoint that includes a "Retry-After" header. All works really well. That being said, I have rebased a few times since doing all that testing and have only lightly tested my code since the rebases. Trying to keep my PR up to date is time consuming and testing it every time is more time consuming. But so far my light tests have worked and I'm happy to perform my thorough tests again at any time. I am doubtful the rebases have introduced a bug, though.

joshua-holmes added 11 commits 2025-12-12 00:51:39 +01:00

README: migrated to codeberg a493e7923e

Implement package fetch retries ... f75210e099

There are some locations where memory is being freed that wasn't being
freed before. This is because, in the case of the user possibly
configuring `Fetch` to retry many times, I don't want the allocator to
allocate without freeing each time it attempts to refetch the network
data, resulting in a memory leak.

Use u16 instead of u32 for retry_count 25b40ef743

Always retry when status code is not "ok" 09677ba84e

Set default retries to 3 (total of 4 attempts) 9fc8d09237

Correct error return type of Client.connect() ... 7b1e1cc104

`ConnectTcpError` is a subset of `ConnectError`, but the 4 additional
errors that `ConnectError` comes with:
```zig
error{
    UnsupportedUriScheme,
    UriMissingHost,
    UriHostTooLong,
    CertificateBundleLoadFailure,
}
```
...are never returned by `Client.connect()`. So we are communicating
that this method could return 1 of those 4 types, when in reality, it
never can. This commit fixes this communication error.

Use error types considered spurious, implement jitter a4b8de28d9

Extend std.time.epoch to get std.Io.Timestamp from datetime 49ba654985

Implement the use of "Retry-After" HTTP header to set retry delay 575887ec1b

Add Retry unit tests deaac30e64

Write tests for "Retry-After" header parser 226176018e

joshua-holmes force-pushed master from 226176018e to 66b7e1a20e 2025-12-12 00:56:23 +01:00 Compare

joshua-holmes changed title from master to WIP: master 2025-12-12 00:57:02 +01:00

joshua-holmes force-pushed master from 66b7e1a20e to acc55ae3b6 2025-12-12 00:59:12 +01:00 Compare

joshua-holmes changed title from WIP: master to master 2025-12-12 01:05:53 +01:00

alexrp commented 2025-12-12 01:08:38 +01:00

Owner

Copy link

Note that master on Codeberg and GitHub have diverged; a493e7923e is not on Codeberg. So you will need to rebase this PR on Codeberg master.

Note that `master` on Codeberg and GitHub have diverged; a493e7923e72594e9c364ad55f32c7c43cdd02e9 is not on Codeberg. So you will need to rebase this PR on Codeberg `master`.

joshua-holmes changed title from master to Implement package fetch retries 2025-12-12 01:08:53 +01:00

joshua-holmes commented 2025-12-12 01:09:49 +01:00

Author

First-time contributor

Copy link

@alexrp thanks for letting me know, I have already done so.

@alexrp thanks for letting me know, I have already done so.

joshua-holmes force-pushed master from acc55ae3b6 to 07eb2a2f16 2025-12-12 01:12:21 +01:00 Compare

joshua-holmes force-pushed master from 07eb2a2f16 to 021910f65d 2025-12-12 01:16:22 +01:00 Compare

joshua-holmes force-pushed master from 021910f65d to 8ad6fbdb01 2025-12-12 01:17:33 +01:00 Compare

andrewrk requested review from andrewrk 2025-12-12 09:31:04 +01:00

joshua-holmes changed title from Implement package fetch retries to WIP: Implement package fetch retries 2025-12-14 21:33:26 +01:00

joshua-holmes force-pushed master from 8ad6fbdb01 to 3384e8c22f 2025-12-14 23:30:00 +01:00 Compare

joshua-holmes force-pushed master from 3384e8c22f to d137fd040b 2025-12-15 01:02:21 +01:00 Compare

joshua-holmes force-pushed master from d137fd040b to ee2fa2b971 2025-12-15 01:08:07 +01:00 Compare

joshua-holmes force-pushed master from ee2fa2b971 to 0b6222945f 2025-12-15 01:09:44 +01:00 Compare

joshua-holmes force-pushed master from 0b6222945f to 1abdae7541 2025-12-15 04:05:40 +01:00 Compare

joshua-holmes force-pushed master from 1abdae7541 to 4cacf87dcf 2025-12-24 16:05:33 +01:00 Compare

Some checks failed

Hide all checks

ci / riscv64-linux-debug (pull_request) Has been skipped

Details

ci / riscv64-linux-release (pull_request) Has been skipped

Details

ci / x86_64-windows-release (pull_request) Failing after 29m34s

Required

Details

ci / x86_64-freebsd-release (pull_request) Successful in 33m33s

Required

Details

ci / x86_64-freebsd-debug (pull_request) Successful in 42m47s

Required

Details

ci / aarch64-macos-release (pull_request) Successful in 43m27s

Required

Details

ci / x86_64-windows-debug (pull_request) Failing after 45m2s

Required

Details

ci / x86_64-linux-debug (pull_request) Successful in 54m20s

Required

Details

ci / x86_64-linux-debug-llvm (pull_request) Successful in 1h15m27s

Required

Details

ci / aarch64-macos-debug (pull_request) Successful in 1h25m35s

Required

Details

ci / aarch64-linux-release (pull_request) Successful in 1h25m30s

Required

Details

ci / s390x-linux-release (pull_request) Successful in 1h27m49s

Required

Details

ci / x86_64-linux-release (pull_request) Failing after 1h48m57s

Required

Details

ci / loongarch64-linux-release (pull_request) Successful in 1h49m16s

Required

Details

ci / s390x-linux-debug (pull_request) Successful in 2h20m37s

Required

Details

ci / aarch64-linux-debug (pull_request) Successful in 2h27m50s

Required

Details

ci / loongarch64-linux-debug (pull_request) Successful in 2h35m58s

Required

Details

This pull request has changes conflicting with the target branch.

• src/Package/Fetch.zig

View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u master:joshua-holmes-master

git switch joshua-holmes-master

Sign in to join this conversation.

Reviewers

No reviewers

andrewrk

Labels

Clear labels   

abi/f32

  

abi/ilp32

  

abi/n32

  

abi/sf

  

abi/x32

  

accepted

This proposal is planned.

  

arch/1750a

  

arch/21k

  

arch/6502

  

arch/a29k

  

arch/aarch64

  

arch/alpha

  

arch/amdgcn

  

arch/arc

  

arch/arc32

  

arch/arc64

  

arch/arm

  

arch/avr

  

arch/avr32

  

arch/bfin

  

arch/bpf

  

arch/clipper

  

arch/colossus

  

arch/cr16

  

arch/cris

  

arch/csky

  

arch/dlx

  

arch/dsp16xx

  

arch/elxsi

  

arch/epiphany

  

arch/fr30

  

arch/frv

  

arch/h8300

  

arch/h8500

  

arch/hexagon

  

arch/hppa

  

arch/hppa64

  

arch/i370

  

arch/i860

  

arch/i960

  

arch/ia64

  

arch/ip2k

  

arch/kalimba

  

arch/kvx

  

arch/lanai

  

arch/lm32

  

arch/loongarch32

  

arch/loongarch64

  

arch/m32r

  

arch/m68k

  

arch/m88k

  

arch/maxq

  

arch/mcore

  

arch/metag

  

arch/microblaze

  

arch/mips

  

arch/mips64

  

arch/mmix

  

arch/mn10200

  

arch/mn10300

  

arch/moxie

  

arch/mrisc32

  

arch/msp430

  

arch/nds32

  

arch/nios2

  

arch/ns32k

  

arch/nvptx

  

arch/or1k

  

arch/pdp10

  

arch/pdp11

  

arch/pj

  

arch/powerpc

  

arch/powerpc64

  

arch/propeller

  

arch/riscv32

  

arch/riscv64

  

arch/rl78

  

arch/rx

  

arch/s390

  

arch/s390x

  

arch/sh

  

arch/sh64

  

arch/sparc

  

arch/sparc64

  

arch/spirv

  

arch/spu

  

arch/st200

  

arch/starcore

  

arch/tilegx

  

arch/tilepro

  

arch/tricore

  

arch/ts

  

arch/v850

  

arch/vax

  

arch/vc4

  

arch/ve

  

arch/wasm

  

arch/we32k

  

arch/x86

  

arch/x86_16

  

arch/x86_64

  

arch/xcore

  

arch/xgate

  

arch/xstormy16

  

arch/xtensa

  

autodoc

The web application for interactive documentation and generation of its assets.

  

backend/c

The C backend outputs C source code.

  

backend/llvm

The LLVM backend outputs an LLVM bitcode module.

  

backend/self-hosted

The self-hosted backends produce machine code directly.

  

binutils

Zig's included binary utilities: zig ar, zig dlltool, zig lib, zig ranlib, zig objcopy, and zig rc.

  

breaking

Implementing this issue could cause existing code to no longer compile or have different behavior.

  

build system

The Zig build system - zig build, std.Build, the build runner, and package management.

  

debug info

An issue related to debug information (e.g. DWARF) produced by the Zig compiler.

  

docs

An issue with documentation, e.g. the language reference or standard library doc comments.

  

error message

This issue points out an error message that is unhelpful and should be improved.

  

frontend

Tokenization, parsing, AstGen, ZonGen, Sema, Legalize, and Liveness.

  

fuzzing

An issue related to Zig's integrated fuzz testing.

  

incremental

Reuse of internal compiler state for faster compilation.

  

lib/c

This issue relates to Zig's libc implementation and/or vendored libcs.

  

lib/compiler-rt

This issue relates to Zig's compiler-rt library.

  

lib/cxx

This issue relates to Zig's vendored libc++ and/or libc++abi.

  

lib/std

This issue relates to Zig's standard library.

  

lib/tsan

This issue relates to Zig's vendored libtsan.

  

lib/ubsan-rt

This issue relates to Zig's ubsan-rt library.

  

lib/unwind

This issue relates to Zig's vendored libunwind.

  

linking

Zig's integrated object file and incremental linker.

  

miscompilation

The compiler reports success but produces semantically incorrect code.

  

os/aix

  

os/android

  

os/bridgeos

  

os/contiki

  

os/dragonfly

  

os/driverkit

  

os/emscripten

  

os/freebsd

  

os/fuchsia

  

os/haiku

  

os/hermit

  

os/hurd

  

os/illumos

  

os/ios

  

os/kfreebsd

  

os/linux

  

os/maccatalyst

  

os/macos

  

os/managarm

  

os/netbsd

  

os/ohos

  

os/openbsd

  

os/plan9

  

os/redox

  

os/rtems

  

os/serenity

  

os/solaris

  

os/tvos

  

os/uefi

  

os/visionos

  

os/wali

  

os/wasi

  

os/watchos

  

os/windows

  

os/zos

  

proposal

This issue suggests modifications. If it also has the "accepted" label then it is planned.

  

release notes

This issue or pull request should be mentioned in the release notes.

  

testing

This issue is related to testing the compiler, standard library, or other parts of Zig.

  

tier system

This issue tracks the support tier for a target.

  

zig cc

Zig as a drop-in C-family compiler.

  

zig fmt

The Zig source code formatter.

  

bounty

https://ziglang.org/news/announcing-donor-bounties

  

bug

Observed behavior contradicts documented or intended behavior.

  

contributor-friendly

This issue is limited in scope and/or knowledge of project internals.

  

downstream

An issue with a third-party project that uses this project.

  

enhancement

Solving this issue will likely involve adding new logic or components to the codebase.

  

infra

An issue related to project infrastructure, e.g. continuous integration.

  

optimization

A task to improve performance and/or resource usage.

  

question

No questions on the issue tracker; use a community space instead.

  

regression

A bug that did not occur in a previous version.

  

upstream

An issue with a third-party project that this project uses.

No labels

abi/f32

abi/ilp32

abi/n32

abi/sf

abi/x32

accepted

arch/1750a

arch/21k

arch/6502

arch/a29k

arch/aarch64

arch/alpha

arch/amdgcn

arch/arc

arch/arc32

arch/arc64

arch/arm

arch/avr

arch/avr32

arch/bfin

arch/bpf

arch/clipper

arch/colossus

arch/cr16

arch/cris

arch/csky

arch/dlx

arch/dsp16xx

arch/elxsi

arch/epiphany

arch/fr30

arch/frv

arch/h8300

arch/h8500

arch/hexagon

arch/hppa

arch/hppa64

arch/i370

arch/i860

arch/i960

arch/ia64

arch/ip2k

arch/kalimba

arch/kvx

arch/lanai

arch/lm32

arch/loongarch32

arch/loongarch64

arch/m32r

arch/m68k

arch/m88k

arch/maxq

arch/mcore

arch/metag

arch/microblaze

arch/mips

arch/mips64

arch/mmix

arch/mn10200

arch/mn10300

arch/moxie

arch/mrisc32

arch/msp430

arch/nds32

arch/nios2

arch/ns32k

arch/nvptx

arch/or1k

arch/pdp10

arch/pdp11

arch/pj

arch/powerpc

arch/powerpc64

arch/propeller

arch/riscv32

arch/riscv64

arch/rl78

arch/rx

arch/s390

arch/s390x

arch/sh

arch/sh64

arch/sparc

arch/sparc64

arch/spirv

arch/spu

arch/st200

arch/starcore

arch/tilegx

arch/tilepro

arch/tricore

arch/ts

arch/v850

arch/vax

arch/vc4

arch/ve

arch/wasm

arch/we32k

arch/x86

arch/x86_16

arch/x86_64

arch/xcore

arch/xgate

arch/xstormy16

arch/xtensa

autodoc

backend/c

backend/llvm

backend/self-hosted

binutils

breaking

build system

debug info

docs

error message

frontend

fuzzing

incremental

lib/c

lib/compiler-rt

lib/cxx

lib/std

lib/tsan

lib/ubsan-rt

lib/unwind

linking

miscompilation

os/aix

os/android

os/bridgeos

os/contiki

os/dragonfly

os/driverkit

os/emscripten

os/freebsd

os/fuchsia

os/haiku

os/hermit

os/hurd

os/illumos

os/ios

os/kfreebsd

os/linux

os/maccatalyst

os/macos

os/managarm

os/netbsd

os/ohos

os/openbsd

os/plan9

os/redox

os/rtems

os/serenity

os/solaris

os/tvos

os/uefi

os/visionos

os/wali

os/wasi

os/watchos

os/windows

os/zos

proposal

release notes

testing

tier system

zig cc

zig fmt

bounty

bug

contributor-friendly

downstream

enhancement

infra

optimization

question

regression

upstream

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

2 participants

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

ziglang/zig!30171

No description provided.