lonpo/crawler4j
Open Source Web Crawler for Java
使用scrapy,redis, mongodb,graphite实现的一个分布式网络爬虫,底层存储mongodb集群,分布式使用redis实现,爬虫状态显示使用graphite实现
Public repository record indexed from GitHub. Explore verified star velocity metrics, source code repositories, and curated developer tool directories across the GitHubRepo ecosystem.
Open Source Web Crawler for Java
C# Crawler 多线程爬虫程序,支持正则表达式过滤、关键字过滤、正文内容识别等等
Agent for twemproxy to work with a redis sentinel (master-slave) stack
javascript-cloud9