Skip to content
This repository was archived by the owner on Dec 19, 2022. It is now read-only.
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: gocardless/github-archive
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: master
Choose a base ref
...
head repository: zph/github-archive
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: master
Choose a head ref
Checking mergeability… Don’t worry, you can still create the pull request.
  • 16 commits
  • 889 files changed
  • 1 contributor

Commits on Jul 7, 2016

  1. Use HTTPS for git clone and add MD5 hashing

    This commit switches from git clone using ssh to git clone using https.
    
    With SSH, the box must have ssh pub/priv keys for github private repos.
    With HTTPS, we can reuse the GITHUB_ACCESS_TOKEN used for API calls.
    
    It also changes the structure of the final file's key on S3 so that
    the `/` characters cause folder-like behavior in S3 clients. And we
    append a MD5 hash of the final tar.gz for validation.
    zph committed Jul 7, 2016
    Configuration menu
    Copy the full SHA
    cfce6ca View commit details
    Browse the repository at this point in the history
  2. Move to gb vendor style project

    zph committed Jul 7, 2016
    Configuration menu
    Copy the full SHA
    fce2760 View commit details
    Browse the repository at this point in the history
  3. Update README

    zph committed Jul 7, 2016
    Configuration menu
    Copy the full SHA
    ea22324 View commit details
    Browse the repository at this point in the history
  4. Remove double dashes in README

    zph committed Jul 7, 2016
    Configuration menu
    Copy the full SHA
    339147e View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    ce44c02 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    60cdd39 View commit details
    Browse the repository at this point in the history
  7. Add random sleeps before git clone and for 1 retry

    On Heroku dynos we saw `resource unavailable errors` and 128 exit codes
    for some clones.
    
    I suspect that all 50 clones spinning up at nearly same millisecond is
    causing resource issues.
    zph committed Jul 7, 2016
    Configuration menu
    Copy the full SHA
    b43ddf1 View commit details
    Browse the repository at this point in the history
  8. Use settable ENV variable for GO_WORKERS count

    This will help scale the activity on boxes with limited resources
    
    The retry and backoff from prior commit was insufficient for ensuring
    stable cloning behavior on Heroku dynos.
    zph committed Jul 7, 2016
    Configuration menu
    Copy the full SHA
    265dc20 View commit details
    Browse the repository at this point in the history

Commits on Jul 9, 2016

  1. Add mongo-archiver

    Heroku doesn't provide a persistent filestructure, so this commit
    moves the mongo->s3 behavior into a streaming process that pipes
    output from mongodump-> s3upload.
    
    Conceptually it's `mongodump --stdout | s3upload`.
    zph committed Jul 9, 2016
    Configuration menu
    Copy the full SHA
    78c1d06 View commit details
    Browse the repository at this point in the history

Commits on Jul 10, 2016

  1. Use pseudo_uuid for filename

    Files are overwriting each other on S3 when done hourly. Using
    pseudo_uuid will fix it.
    zph committed Jul 10, 2016
    Configuration menu
    Copy the full SHA
    5a2ffcd View commit details
    Browse the repository at this point in the history

Commits on Apr 8, 2017

  1. Remove vendor folder

    zph committed Apr 8, 2017
    Configuration menu
    Copy the full SHA
    054f4fd View commit details
    Browse the repository at this point in the history
  2. Upgrade to using glide

    zph committed Apr 8, 2017
    Configuration menu
    Copy the full SHA
    4d27231 View commit details
    Browse the repository at this point in the history
  3. Update Makefile for glide

    zph committed Apr 8, 2017
    Configuration menu
    Copy the full SHA
    8a49218 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    ab913f1 View commit details
    Browse the repository at this point in the history

Commits on Apr 11, 2017

  1. Add notes to readme re buildpack

    zph committed Apr 11, 2017
    Configuration menu
    Copy the full SHA
    7651073 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    ffbcd5f View commit details
    Browse the repository at this point in the history
Loading