sunchen009/frontera
A flexible frontier for web crawlers
The goal of this project is to make XMPP really simple to use for developers. This module takes away all of the XML and works by hooking to events which are passed between client and server using a transport in JSON. In the example code we use socket.io, but there is no reason this can not be replaced with engine.io, or implement your own transport and pass in as a connection.
Public repository record indexed from GitHub. Explore verified star velocity metrics, source code repositories, and curated developer tool directories across the GitHubRepo ecosystem.
A flexible frontier for web crawlers
Lightweight, scriptable browser as a service with an HTTP API
Scrapy, a fast high-level web crawling & scraping framework for Python.
The Polipo caching HTTP proxy