- event: these are available options
resttouristMoviesTechNewsSportsNewsithometaiwan_attractionstaiwan_restaurantyahoosportyahoomoviepoliticsart
num: please don't query num bigger than20, some type of event json didn't prepare for that much.
example : 140.120.13.243:12435/api/?event=SportsNews&num=20
- run Django
- . venv/bin/activate
- cd user_interest_api_server/
- nohup python3 manage.py runserver 0.0.0.0:8000 &
- run Crawler
- nohup python3 manage.py scrapy_background.py &
- nohup python3 yahoo_sport.py &
- nohup python3 comprehensiveCrawler.py &
index.html provide a web service by loading json
When you start to look up those data.
you need to run scrapy command first to get those json file.
- scrapy:
scrapy crawl Attractions -o attractions.json -t jsonfrom tripadvisorscrapy crawl restaurant -o restaurant.json -t jsonfrom tripadvisorscrapy crawl taiwan_attractions -o taiwan_attractions.json -t jsonfrom taiwan tripadvisorscrapy crawl taiwan_restaurant -o taiwan_restaurant.json -t jsonfrom taiwan restaurantscrapy crawl ithome -o ithome.json -t jsonfrom ithomescrapy crawl yahoomovie -o yahoomovie.json -t jsonfrom movie board of yahoo- use
python scrapy_background.pyto activate
- use
scrapy crawl artemperor -o art.json -t jsonfrom movie board of yahoo- use
python scrapy_background.pyto activate
- use
scrapy crawl politics -o politics.json -t jsonfrom movie board of yahoo- use
python scrapy_background.pyto activate
- use
- seperate crawlers:
comprehensiveCrawler.pyis a web crawler which can get the information from ESPN, CNET and IMDB, and organize them into JSON files for sports news, tech news and movie inforamtion respectively. The JSON files will be created in ./result/.python yahoo_sport.pyfrom sport board of yahoo
proportion: the proportion of 20 product categories.- the categories sequence is shown below.
tissuenotebooklodgingfragrancesportswearmakeuphealthorganicfoodwatchunderweargirlshoespregnantappliancescampingbagbookvideostationeryreligionanime
num: the number of displays.- please don't input num bigger than
200. - currently there are only 10 data in each category.
example : 140.120.13.243:12435/momoapi/?proportion=[1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1]&num=100