robhammond/scrapy-elasticsearch
A scrapy pipeline which send items to Elastic Search server
Manage & check proxies using Mojolicious & MongoDB
This repository is cataloged as part of our automated global GitHub synchronization. Full telemetry, velocity snapshots, and code summaries are scheduled for continuous enrichment.
A scrapy pipeline which send items to Elastic Search server
Very simple Mojolicious Lite front-end to ElasticSearch
Use scrapy with mongodb to store the request queues (FIFO or LIFO)
A middleware to use random user agent in Scrapy crawler.