scrapy实战 常见错误

Last updated on September 15, 2024 pm

🧙 Questions

☄️ Ideas

常见错误

ModuleNotFoundError: No module named 'protego'
解决方案

删掉 d:\tools\python\lib\site-packages\protego 重新安装

pip install protego
爬虫数据
2021-05-26 17:31:17 [scrapy.downloadermiddlewares.robotstxt] DEBUG: Forbidden by robots.txt: <GET https://www.baidu.com/>
# 将settings.py 
ROBOTSTXT_OBEYFalse
scrapy
scrapy实战 常见错误
https://ispong.isxcode.com/pytorch/scrapy/scrapy实战 常见错误/
Author
ispong
Posted on
May 26, 2021
Licensed under