redhotpenguin/walker
An efficient, scalable, open source crawler leveraging Go and Cassandra
Public repository discovered through GitHub real-time crawl.
This repository is cataloged as part of our automated global GitHub synchronization. Full telemetry, velocity snapshots, and code summaries are scheduled for continuous enrichment.
An efficient, scalable, open source crawler leveraging Go and Cassandra
Inline::Java
Old MacDonald had a farm
Perl port of the original Ruby library. It's intended to work using the same backend to share tasks and be able to manage the system using ruby's resque-server webapp. Resque is a Redis-backed library for creating background jobs, placing them on multiple queues, and processing them later.