The Wayback Machine - https://web.archive.org/web/20201130063244/https://github.com/topics/spider
Skip to content
#

spider

Star

Here are 2,166 public repositories matching this topic...

Language: All
Filter by language
All 2,166 Python 1,207 JavaScript 247 Java 158 Go 125 HTML 79 PHP 69 C# 34 Jupyter Notebook 34 TypeScript 22 CSS 15
Sort: Best match
Sort options
Best match Most stars Fewest stars Most forks Fewest forks Recently updated Least recently updated

kangvcar / InfoSpider

Star

INFO-SPIDER 是一个集众多数据源于一身的爬虫工具箱🧰，旨在安全快捷的帮助用户拿回自己的数据，工具代码开源，流程透明。支持数据源包括GitHub、QQ邮箱、网易邮箱、阿里邮箱、新浪邮箱、Hotmail邮箱、Outlook邮箱、京东、淘宝、支付宝、中国移动、中国联通、中国电信、知乎、哔哩哔哩、网易云音乐、QQ好友、QQ群、生成朋友圈相册、浏览器浏览历史、12306、博客园、CSDN博客、开源中国博客、简书。
chrome automation spider selenium python3 crawl tkinter outlook wxpython csdn hotmail
  • Updated Nov 29, 2020
  • Python

my8100 / scrapydweb

Star
Open

User Guide | Q&A | 用户指南 | 问答

78
LWsmile
LWsmile commented Nov 27, 2018

linux：HTTPConnectionPool(host='192.168.0.24', port=6801): Max retries exceeded with url: /listprojects.json (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7f0a78b2d828>: Failed to establish a new connection: [Errno 111] Connection refused',))
windows：HTTPConnectionPool(host='localhost', port=6801): Max retries exceeded with url: /jobs (Caused by Ne

Read more
good first issue question

JayBizzle / Crawler-Detect

Star
Open

Potential bots

3
JayBizzle
JayBizzle commented Jan 21, 2020
  • Filestack
  • Google-Ads-Overview Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.118 Safari/537.36
  • Google-Ads-Overview Mozilla/5.0 (Linux; U; Android 6.0.1; generic) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.118 Version/4.0 Mobile Safari/537.36
  • Google-Ads-Overview Mozilla/5.0 (Linux; U; Android 2.3.4; generic) AppleW
Read more
good first issue

Improve this page

Add a description, image, and links to the spider topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the spider topic, visit your repo's landing page and select "manage topics."

Learn more

You can’t perform that action at this time.