Scapy powered spider crawling newest 50 themes titles from stackoverflow.com

Installing scrapy from pip(latest version)

pip install scrapy

clone this repo, cd to working dir and start crawling!

scrapy crawl stack

Enjoy results

stack/items.py items fields defenitions

stack/pipelines.py pipelines for parsing. Not using in current version since script is very simple and more like proof of concept

stack/setting.py Main settings file more info

stack/spiders/stack_spider.py spider for crawling. Main logic here