感觉自己写得很复杂
果然自己太低了
已更新ip格式
#Python3.6中需要添加http://proxy _ pool={ ' http ' : ' 127.0.0.1:8080 '，' https':'127.0.133608080} proxies=random.choice (proxy _ pool ) ) proxies={ ' http ' : ' 127.0.0.133608080 ' }否则加proxy _ pool=' https ' : ' https://127.0.0.133608080 '，弄错1:8080 '的proxies=random.choice (proxy _ pool ) ) proxies={ 创建用户代理池#1.1以获取用户池agents=[ ' Mozilla/5.0 (windows nt 10.0； WOW64 ) appleWebKit/537.36(khtml，like Gecko ) chrome/78.0.3904.108 safari/537.36 ' Mozilla/5.0 ) Windowsnt10 WOW64 like gecko (chrome/78.0.3904.108 safari/537.36 ' ) 1.2报头信息def ua ) )封装用于存储：a=random.choice ) agents (headers )的函数' referer ' : ' http://pic.yeses returnheaders#2.2.1IP创建IP池# 2.1 ' 39.106.123.219:3128 '，' 117.185.17.17:80 '， ] #2.2封装函数将本地IP映射到代理ipdefippools (: b=random.choice ) IPS ) print(b ) proxy=request.proxy handler (b ) request.httphandler (request.install _ opener (opener ) )是全局变量#3.URL规则defURL ) I ) :URL='http://PIC.yesky 4 .图片forIinrange(2， 13 ) : ippools ) #每次可以与ip thisurl=url(i ) thisheaders=ua ) ) rep=交换的headers=this headers (resp=request.urll ) 网页源代码内容text=resp.read ).decode('utf-8 '， () txt=re.compile(pat ).find all (text ) URLthatURL=txt )0) # 获得print ) thatURL ) localsfile=) f3360 () )的'第' str(I ) ' ysdwk '.jpg ' request.URL retrieve (that URL，filename )
这里是引用
3359 blog.csdn.net/douzhenwen/article/details/109660728