chermong/distribute_crawler
使用scrapy,redis, mongodb,graphite实现的一个分布式网络爬虫,底层存储mongodb集群,分布式使用redis实现,爬虫状态显示使用graphite实现
Developer Profile
chermong
Discovered public repositories for chermong in the GitHub catalog.
chermong/sina_reptile
获取新浪微博1000w用户的基本信息和每个爬取用户最近发表的50条微博,使用python编写，多进程爬取，将数据存储在了mongodb中
chermong/uliweb
Simple and easy use python web framework
chermong/open-taobao
make use taobao open api more flexible
chermong/ComPerformance
一个使用django开发的学生综合成绩管理平台
chermong/redis
Redis is an in-memory database that persists on disk. The data model is key-value, but many different kind of values are supported: Strings, Lists, Sets, Sorted Sets, Hashes