Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Sep 16, 2025

The Dead Link Checker workflow was failing with 429 (Too Many Requests) errors from HashiCorp domains that rate-limit automated link checking tools like lychee.

Problem

The workflow was encountering rate limiting on these URLs:

  • https://learn.hashicorp.com/tutorials/terraform/install-cli (in aws, azure, gcp READMEs)
  • https://developer.hashicorp.com/vault/install (in aws, azure, gcp READMEs)
  • https://www.vaultproject.io/downloads (in main README)

Solution

Updated URLs to current HashiCorp documentation structure:

  • Changed https://learn.hashicorp.com/tutorials/terraform/install-cli to https://developer.hashicorp.com/terraform/install
  • Changed https://www.vaultproject.io/downloads to https://developer.hashicorp.com/vault/install

Added ignore patterns to .lycheeignore:

  • https://developer.hashicorp.com/*
  • https://learn.hashicorp.com/*
  • https://www.vaultproject.io/*

This dual approach ensures current documentation links are used while preventing future rate limiting issues from HashiCorp domains.

Files Modified

  • aws/README.md - Updated Terraform installation URL
  • azure/README.md - Updated Terraform installation URL
  • gcp/README.md - Updated Terraform installation URL
  • README.md - Updated Vault installation URL
  • .lycheeignore - Added HashiCorp domain patterns

The deadlink checker workflow should now pass without 429 rate limiting errors.

Fixes #2240.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • 1password.com
    • Triggering command: ./lychee --exclude-all-private --exclude-path src/main/resources/templates/about.html --exclude-path .lycheeignore README.md (dns block)
  • alphasec.io
    • Triggering command: ./lychee --exclude-all-private --exclude-path src/main/resources/templates/about.html --exclude-path .lycheeignore README.md (dns block)
  • aws.amazon.com
    • Triggering command: ./lychee --exclude-all-private --exclude-path src/main/resources/templates/about.html --exclude-path .lycheeignore README.md (dns block)
  • bestpractices.coreinfrastructure.org
    • Triggering command: ./lychee --exclude-all-private --exclude-path src/main/resources/templates/about.html --exclude-path .lycheeignore README.md (dns block)
  • blog.gitguardian.com
    • Triggering command: ./lychee --exclude-all-private --exclude-path src/main/resources/templates/about.html --exclude-path .lycheeignore README.md (dns block)
  • canarytokens.org
    • Triggering command: ./lychee --exclude-all-private --exclude-path src/main/resources/templates/about.html --exclude-path .lycheeignore README.md (dns block)
  • dev.to
    • Triggering command: ./lychee --exclude-all-private --exclude-path src/main/resources/templates/about.html --exclude-path .lycheeignore README.md (dns block)
  • helm.sh
  • img.shields.io
    • Triggering command: ./lychee --exclude-all-private --exclude-path src/main/resources/templates/about.html --exclude-path .lycheeignore README.md (dns block)
  • kubernetes.io
    • Triggering command: ./lychee --exclude-all-private --exclude-path src/main/resources/templates/about.html --exclude-path .lycheeignore README.md (dns block)
  • minikube.sigs.k8s.io
    • Triggering command: ./lychee --exclude-all-private --exclude-path src/main/resources/templates/about.html --exclude-path .lycheeignore README.md (dns block)
  • owasp.org
    • Triggering command: ./lychee --exclude-all-private --exclude-path src/main/resources/templates/about.html --exclude-path .lycheeignore README.md (dns block)
  • owasp.slack.com
    • Triggering command: ./lychee --exclude-all-private --exclude-path src/main/resources/templates/about.html --exclude-path .lycheeignore README.md (dns block)
  • owaspsamm.org
    • Triggering command: ./lychee --exclude-all-private --exclude-path src/main/resources/templates/about.html --exclude-path .lycheeignore README.md (dns block)
  • railway.com
    • Triggering command: ./lychee --exclude-all-private --exclude-path src/main/resources/templates/about.html --exclude-path .lycheeignore README.md (dns block)
  • render.com
    • Triggering command: ./lychee --exclude-all-private --exclude-path src/main/resources/templates/about.html --exclude-path .lycheeignore README.md (dns block)
  • stedolan.github.io
    • Triggering command: ./lychee --exclude-all-private --exclude-path src/main/resources/templates/about.html --exclude-path .lycheeignore README.md (dns block)
  • www.baeldung.com
    • Triggering command: ./lychee --exclude-all-private --exclude-path src/main/resources/templates/about.html --exclude-path .lycheeignore README.md (dns block)
  • www.gitguardian.com
    • Triggering command: ./lychee --exclude-all-private --exclude-path src/main/resources/templates/about.html --exclude-path .lycheeignore README.md (dns block)
  • www.github.com
    • Triggering command: ./lychee --exclude-all-private --exclude-path src/main/resources/templates/about.html --exclude-path .lycheeignore README.md (dns block)
  • www.jetbrains.com
    • Triggering command: ./lychee --exclude-all-private --exclude-path src/main/resources/templates/about.html --exclude-path .lycheeignore README.md (dns block)
  • www.linkedin.com
    • Triggering command: ./lychee --exclude-all-private --exclude-path src/main/resources/templates/about.html --exclude-path .lycheeignore README.md (dns block)
  • www.opencre.org
    • Triggering command: ./lychee --exclude-all-private --exclude-path src/main/resources/templates/about.html --exclude-path .lycheeignore README.md (dns block)
  • www.wrongsecrets.com
    • Triggering command: ./lychee --exclude-all-private --exclude-path src/main/resources/templates/about.html --exclude-path .lycheeignore README.md (dns block)

If you need me to access, download, or install something from one of these locations, you can either:


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot AI and others added 2 commits September 16, 2025 07:25
…terns

Co-authored-by: commjoen <1457214+commjoen@users.noreply.github.com>
Copilot AI changed the title [WIP] Deadlink checker is erroring Fix deadlink checker by updating HashiCorp URLs and adding ignore patterns Sep 16, 2025
Copilot AI requested a review from commjoen September 16, 2025 07:27
@commjoen commjoen marked this pull request as ready for review September 16, 2025 08:23
@commjoen commjoen requested a review from bendehaan as a code owner September 16, 2025 08:23
@commjoen commjoen merged commit 17422bf into master Sep 16, 2025
33 checks passed
@commjoen commjoen deleted the copilot/fix-2240 branch September 16, 2025 08:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Deadlink checker is erroring

2 participants