Built something? We create video reels & spotlights for GitHub projects.Promote your project →
Catalog / stpettersens / CrawlerBot
Public GitHub Catalog Discovered Oct 7, 2026

stpettersens / CrawlerBot

:baby_symbol: A robots.txt-compliant website crawler written in Python.

View repository on GitHub ↗ View creator profile Browse directory

About this discovery

Public repository record indexed from GitHub. Explore verified star velocity metrics, source code repositories, and curated developer tool directories across the GitHubRepo ecosystem.

#35727895GitHub System ID
stpettersensOrganization / User
PublicVisibility
ActiveCatalog Status

More from stpettersens

↗

stpettersens/love2js

:space_invader: Utility to convert a Love2D game to a JavaScript-based HTML5 game.

Discovered
↗

stpettersens/Coyote

:wolf: A simple web server for serving static content with supported MIME types and headers configured via XML.

Discovered
FOR MAINTAINERS

Built something? Put it in front of millions of developers.

We make a short reel about your project and post it across YouTube, Instagram, Threads, and X. Send a link, we do the rest.

SPONSORED Sponsored Partner