Top 11 open source projects in Headless Browsers & Scraping. Ranked by stars, activity, and recorded community growth.
🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev
An AI-powered research assistant that performs iterative, deep research on any topic by combining search engines, web scraping, and large language models. The goal of this repo is to provide the simplest implementation of a deep research agent - e.g. an agent that can refine its research direction overtime and deep dive into a topic.
Ruby gem for web scraping purposes. It scrapes a given URL, and returns you its title, meta description, meta keywords, links, images...
📘 Get all Warframe items directly from Warframe's API. No more messy wikia scraping.
A simple to learn and use, yet powerful web scraping toolkit!
this shows how to use github actions to do periodic data scraping
Spider is a Web spidering library for Ruby. It handles the robots.txt, scraping, collecting, and looping so that you can just handle the data.
A web-scraping interface to various Google Services.
python twitter library which gets by API & html scraping
A webpage that generates an ordered list of Heroes episodes and comics by scraping Wikipedia