Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: GregorUT/vgchartzScrape
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: master
Choose a base ref
...
head repository: ashaheedq/vgchartzScrape
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: master
Choose a head ref
Checking mergeability… Don’t worry, you can still create the pull request.
  • 18 commits
  • 5 files changed
  • 1 contributor

Commits on Apr 7, 2019

  1. added headers, sleep, genre, try:except for handling timeout, in proc… 

    …ess of chaning from urllib to requests
    @ashaheedq
    ashaheedq committed Apr 7, 2019
    Configuration menu
    Copy the full SHA
    7448af2 View commit details
    Browse the repository at this point in the history

  2. added headers, sleep, genre, try:except for handling timeout, in proc… 

    …ess of chaning from urllib to requests
    @ashaheedq
    ashaheedq committed Apr 7, 2019
    Configuration menu
    Copy the full SHA
    68f0e1e View commit details
    Browse the repository at this point in the history

Commits on Apr 9, 2019

  1. making progress

    @ashaheedq
    ashaheedq committed Apr 9, 2019
    Configuration menu
    Copy the full SHA
    e81ab33 View commit details
    Browse the repository at this point in the history

  2. multiprocessing works, after second thougt will change to threading

    @ashaheedq
    ashaheedq committed Apr 9, 2019
    Configuration menu
    Copy the full SHA
    29d6aad View commit details
    Browse the repository at this point in the history

  3. fixed couple of things

    @ashaheedq
    ashaheedq committed Apr 9, 2019
    Configuration menu
    Copy the full SHA
    dd2df6f View commit details
    Browse the repository at this point in the history

Commits on Apr 10, 2019

  1. finished

    @ashaheedq
    ashaheedq committed Apr 10, 2019
    Configuration menu
    Copy the full SHA
    cdd36c4 View commit details
    Browse the repository at this point in the history

  2. README Update!

    @ashaheedq
    ashaheedq committed Apr 10, 2019
    Configuration menu
    Copy the full SHA
    9e206fc View commit details
    Browse the repository at this point in the history

  3. fixed a small bug that made the script runs longer

    @ashaheedq
    ashaheedq committed Apr 10, 2019
    Configuration menu
    Copy the full SHA
    79cba81 View commit details
    Browse the repository at this point in the history

  4. Re-enabled proxies, very FAST!

    @ashaheedq
    ashaheedq committed Apr 10, 2019
    Configuration menu
    Copy the full SHA
    a672625 View commit details
    Browse the repository at this point in the history

  5. added couple of additional exceptions and proxies to parse pages. Now… 

    … the crawled data get saved before raising error
    @ashaheedq
    ashaheedq committed Apr 10, 2019
    Configuration menu
    Copy the full SHA
    43f5e89 View commit details
    Browse the repository at this point in the history

  6. updated readme

    @ashaheedq
    ashaheedq committed Apr 10, 2019
    Configuration menu
    Copy the full SHA
    c0d4e32 View commit details
    Browse the repository at this point in the history

  7. updated readme

    @ashaheedq
    ashaheedq committed Apr 10, 2019
    Configuration menu
    Copy the full SHA
    48060d2 View commit details
    Browse the repository at this point in the history

  8. added option to continue after error

    @ashaheedq
    ashaheedq committed Apr 10, 2019
    Configuration menu
    Copy the full SHA
    69076d7 View commit details
    Browse the repository at this point in the history

  9. good that I caught it before running the script

    @ashaheedq
    ashaheedq committed Apr 10, 2019
    Configuration menu
    Copy the full SHA
    1b553af View commit details
    Browse the repository at this point in the history

Commits on Apr 11, 2019

  1. few updates and bug fixes

    @ashaheedq
    ashaheedq committed Apr 11, 2019
    Configuration menu
    Copy the full SHA
    7c20f33 View commit details
    Browse the repository at this point in the history

  2. changed how to calculate num_workers, and minor updates

    @ashaheedq
    ashaheedq committed Apr 11, 2019
    Configuration menu
    Copy the full SHA
    67819a5 View commit details
    Browse the repository at this point in the history

Commits on Apr 12, 2019

  1. added a cleaner version without the print statements

    @ashaheedq
    ashaheedq committed Apr 12, 2019
    Configuration menu
    Copy the full SHA
    b7c12e4 View commit details
    Browse the repository at this point in the history

Commits on Apr 13, 2019

  1. bug fixes and few changes

    @ashaheedq
    ashaheedq committed Apr 13, 2019
    Configuration menu
    Copy the full SHA
    debad0f View commit details
    Browse the repository at this point in the history
Loading