shyalyn/distribute_crawler
使用scrapy,redis, mongodb,graphite实现的一个分布式网络爬虫,底层存储mongodb集群,分布式使用redis实现,爬虫状态显示使用graphite实现
DI ( Dependency Injection: Setter, Constructor, Method), AOP ( Aspect Oriented Programming ), Events support, xml, yaml, and annotations (including some JSR 250 and JSR 330, like @Configuration and @Bean ala java configuration) , lightweight, simple, and quick MVC ( Model View Controller ), syslog, tcp client and server, with non blocking sockets, custom error, signal, and exception handling through events. Needs PHP 5.3, very similar to seasar, spring ( java ) . Can be deployed as a PHAR file.
This repository is cataloged as part of our automated global GitHub synchronization. Full telemetry, velocity snapshots, and code summaries are scheduled for continuous enrichment.
使用scrapy,redis, mongodb,graphite实现的一个分布式网络爬虫,底层存储mongodb集群,分布式使用redis实现,爬虫状态显示使用graphite实现
Scrapy, a fast high-level screen scraping and web crawling framework for Python.
Yii Framework project structure boilerplate for basic requirements
A generic admin theme built with Bootstrap 3 free for both personal and commercial use.