simonhlm/crawler
Folders and files
| Name | Name | Last commit date | ||
|---|---|---|---|---|
Repository files navigation
""" This is a crawler written by python v3.6 """ For the setup: Before you step up the program, you should ensure you are in python 3 ( or later) and has installed below packages 1, requests 2, beautifulsoup 3, Sqlalchmey and then, follow up below ones to setup it: 1, Run the orm.py firstly to create the tables in sqlite. 2, You can run the lkong_crawler.py to fetch the records 3, Run the test.py to check the records that you stored in the sqlite # Basic introduce to the main files: lkong_crawler.py : this is a specifiled crawler to lkong.net. Now it is in progress orm.py: This is the ORM file & setup of tables. craw.py : this is the class for the crawler - draft one . only implmenet the basic function.