my8100's repos on GitHub
Python · 3181 人关注
scrapydweb
Web app for Scrapyd cluster management, Scrapy log analysis & visualization, Auto packaging, Timer tasks, Monitor & Alert, and Mobile UI. DEMO :point_right:
421 人关注
files
Docs and files for ScrapydWeb, Scrapyd, Scrapy, and other projects
Python · 122 人关注
scrapyd-cluster-on-heroku
Set up free and scalable Scrapyd cluster for distributed web-crawling with just a few clicks. DEMO :point_right:
Python · 89 人关注
logparser
A tool for parsing Scrapy log files periodically and incrementally, extending the HTTP JSON API of Scrapyd.
Python · 10 人关注
scrapyd
[PR #326] Native support for basic auth :lock: `pip install -U git+https://github.com/scrapy/scrapyd.git`, then add `username = yourusername` and `password = yourpassword` in the scrapyd.conf file. DOCS :point_right:
7 人关注
awesome-scrapy
A curated list of awesome packages, articles, and other cool resources from the Scrapy community.
Makefile · 6 人关注
awesome-python-cn
Python资源大全中文版,包括:Web框架、网络爬虫、模板引擎、数据库、数据可视化、图片处理等,由伯乐在线持续更新。
Python · 6 人关注
notes
Keep on reading
5 人关注
awesome-crawler
A collection of awesome web crawler,spider in different languages
Makefile · 3 人关注
awesome-web-scraping
List of libraries, tools and APIs for web scraping and data processing.
1 人关注
awesome-flask
A curated list of awesome Flask resources and plugins
Python · 1 人关注
awesome-python
A curated list of awesome Python frameworks, libraries, software and resources
Python · 1 人关注
scrapy-redis
Redis-based components for Scrapy.
Python · 0 人关注
queuelib
Collection of persistent (disk-based) queues