Create agents that monitor and act on your behalf. Your agents are standing by!

Web Scraper in Go, similar to BeautifulSoup

Creating Scrapy scrapers via the Django admin interface

Take the hassle out of web scraping

Suck the InTernet

a class that uses scraped proxies to make http GET/POST requests (Python requests)

Command line tool to download and extract data from HTML/XML pages or JSON-APIs, using CSS, XPath 3.0, XQuery 3.0, JSONiq or pattern matching. It can also create new or transformed XML/HTML/JSON documents.

🥫 The simple, fast, and modern web scraping library

An R web crawler and scraper

Be nice on the web

Open Source web scraping API. Falkor turns web pages into queryable JSON

LinkedIn enumeration tool to extract valid employee names from an organization through search engine scraping

Guide, reference and cheatsheet on web scraping using rvest, httr and Rselenium.

📲 Bot to help solve HQ trivia

An extensible API for breaking captchas

Extract price and indicator data from TradingView charts to create ML datasets

An exploration of New York Times crossword answers from 1994-2017, i.e. the Will Shortz era.

Scrapes g4g and creates PDF

Github stargazers information gathering tool

A php crawler that finds emails on the internets

🎬 A Crunchyroll show/season ripper

Code for the second edition Web Scraping with Python book by Packt Publications

🔍 Data pipeline for crawling PDFs from the Web and transforming their contents into structured data using AWS textract. Built with AWS CDK + TypeScript

Chemical Information from the Web

DotnetCrawler is a straightforward, lightweight web crawling/scrapying library for Entity Framework Core output based on dotnet core. This library designed like other strong crawler libraries like WebMagic and Scrapy but for enabling extandable your custom requirements. Medium link : https://medium.com/@mehmetozkaya/creating-custom-web-crawler-with-dotnet-core-using-entity-framework-core-ec8d23f0ca7c

A tkinter GUI collating various data

Perceptual image hashing for Node.js

extract videos from youtube in audio format using webscraping techniques 🎶

This repository contains data and code used to get and clean data from https://github.com/CSSEGISandData/COVID-19 and https://www.worldometers.info/coronavirus/

operating systems three easy pieces by Rezmi