gnemoug/distribute_crawler: 使用scrapy,redis, mongodb,graphite实现的一个分布式网络爬虫,底层存储mongodb集群,分布式使用redis实现,爬虫状态显示使用graphite实现

3234
STARS
369
WATCHERS
1631
FORKS
26
ISSUES

distribute_crawler's Language Statistics

gnemoug's Other Repos

Star history of distribute_crawler
Issue history of distribute_crawler

distribute_crawler Recent Issues

Issue Title State Comments Created Date Updated Date Closed Date