# leigaosearch/distribute_crawler

> 使用scrapy,redis, mongodb,graphite实现的一个分布式网络爬虫,底层存储mongodb集群,分布式使用redis实现,爬虫状态显示使用graphite实现

- **Canonical URL**: https://githubrepo.cloud/repo/leigaosearch/distribute_crawler
- **GitHub Repository**: https://github.com/leigaosearch/distribute_crawler
- **Maintainer / Creator**: https://github.com/leigaosearch
- **Discovered On**: Sep 27, 2026

---
*Indexed by GitHubRepo (https://githubrepo.cloud). Real-time open-source discovery & telemetry.*