linkerlin/paoding-rose
Public repository.
使用scrapy,redis, mongodb,graphite实现的一个分布式网络爬虫,底层存储mongodb集群,分布式使用redis实现,爬虫状态显示使用graphite实现
This repository is cataloged as part of our automated global GitHub synchronization. Full telemetry, velocity snapshots, and code summaries are scheduled for continuous enrichment.
Public repository.
Public repository.
This plugin crawls web sites and then stores them into Elasticsearch.
Distributed web crawler backed by Elasticsearch