MacOS下设置Python爬虫代理池

https://github.com/jhao104/proxy_pool

https://github.com/jhao104/proxy_pool/issues/473

https://proxy-pool.readthedocs.io/zh/latest/user/how_to_run.html

如果已配置好运行环境, 具备运行条件, 可以通过 proxyPool.py 启动. proxyPool.py 是项目的CLI入口. 完整程序包含两部份: schedule 调度程序和 server API服务, 调度程序负责采集和验证代理, API服务提供代理服务HTTP接口.

通过命令行程序分别启动调度程序和API服务:

1
2
3
4
5
# 启动调度程序
$ python proxyPool.py schedule

# 启动webApi服务
$ python proxyPool.py server

把redis 启动弄好,修改HOST 为 127.0.0.1

![](/Users/m0nk3y/Library/Application Support/typora-user-images/image-20210308230707733.png)

Author: m0nk3y
Link: https://hack-for.fun/a1df.html
Copyright Notice: All articles in this blog are licensed under CC BY-NC-SA 4.0 unless stating additionally.