Skip to content

septian-putra/linkedin

 
 

Repository files navigation

Linkedin Scraping

built with Selenium built with Python3

Scraping software aimed to visit as more linkedin's user pages as possible :-D, the objective is to gain visibility with your account: since LinkedIn notifies the issued User when someone visits his page.

Uses: Scrapy, Selenium web driver, Chromium headless, docker and python3.

Install

Docker allows very easy and fast run without any pain and tears.

0. Preparations

Install docker from the official website https://www.docker.com/

Install VNC viewer if you do not have one. For ubuntu, go for vinagre:

sudo apt-get update
sudo apt-get install vinagre

Then connect to localhost:5900, password: secret

1. Set your linkedin login and password

Open conf.py and fill the quotes with your credentials.

2. Run and build containers with docker-compose

First you need to open your terminal, move to the root folder (usually with the cd command) of the project and then type:

docker-compose up -d --build

3. See what your bot can do right now

Run your VNC viewer, and type address and port localhost:5900. The password is secret.

4. Stop the scraper

Use your terminal again, type in the same window:

docker-compose down
Test:

Create the selenium server:

docker run --name selenium -p 4444:4444 -p 5900:5900 --publish-all --shm-size="128M" selenium/standalone-chrome-debug
virtualenvs -p python .venv
source .venv/bin/activate
pip install -r requirements.txt

python -m unittest test.py

Stop and delete selenium server:

docker stop $(docker ps -aq --filter name=selenium)

docker rm $(docker ps -aq --filter name=selenium)

About

Linkedin Scraper using Selenium Web Driver, Chromium headless, Docker and Scrapy

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 97.5%
  • Shell 2.5%