SpiderClub

Repositories

• weibospider

  ⚡ A distributed crawler for weibo, building with celery and requests.

  python3 data-analysis weibo sina distributed-crawler weibospider
  Python MIT 1,257 4,475 11 (2 issues need help) 2 Updated Jul 11, 2020

• haipproxy

  💖 High available distributed ip proxy pool, powerd by Scrapy and Redis

  redis crawler spider scheduler distributed scrapy high-availability
  Python MIT 851 4,604 31 1 Updated Nov 1, 2019

• smart_login

  各大网站登陆方式，有的是通过selenium登录，有的是通过抓包直接模拟登录（精力原因，目前不再继续维护）
  Python MIT 385 996 14 (1 issue needs help) 0 Updated Apr 9, 2018