Python3爬虫之urllib携带cookie爬取网页的方法
作者:haeasringnar 时间:2022-07-01 14:19:02
如下所示:
import urllib.request
import urllib.parse
url = 'https://weibo.cn/5273088553/info'
#正常的方式进行访问
# headers = {
# 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.84 Safari/537.36'
# }
# 携带cookie进行访问
headers = {
'GET https':'//weibo.cn/5273088553/info HTTP/1.1',
'Host':' weibo.cn',
'Connection':' keep-alive',
'Upgrade-Insecure-Requests':' 1',
'User-Agent':' Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.84 Safari/537.36',
'Accept':' text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8',
# 'Referer: https':'//weibo.cn/',
'Accept-Language':' zh-CN,zh;q=0.9',
'Cookie':' _T_WM=c1913301844388de10cba9d0bb7bbf1e; SUB=_2A253Wy_dDeRhGeNM7FER-CbJzj-IHXVUp7GVrDV6PUJbkdANLXPdkW1NSesPJZ6v1GA5MyW2HEUb9ytQW3NYy19U; SUHB=0bt8SpepeGz439; SCF=Aua-HpSw5-z78-02NmUv8CTwXZCMN4XJ91qYSHkDXH4W9W0fCBpEI6Hy5E6vObeDqTXtfqobcD2D32r0O_5jSRk.; SSOLoginState=1516199821',
}
request = urllib.request.Request(url=url,headers=headers)
response = urllib.request.urlopen(request)
#输出所有
# print(response.read().decode('gbk'))
#将内容写入文件中
with open('weibo.html','wb') as fp:
fp.write(response.read())
来源:https://blog.csdn.net/haeasringnar/article/details/79090971
标签:Python3,urllib,cookie
![](/images/zang.png)
![](/images/jiucuo.png)
猜你喜欢
详解PHP变量传值赋值和引用赋值变量销毁
2023-10-25 00:27:31
![](https://img.aspxhome.com/file/2023/0/55720_0s.png)
一个ASP站内搜索的实例源代码
2007-09-21 12:13:00
SQL对时间处理的语句小结
2011-12-01 07:53:04
Oracle中查询本月星期5的所有日期列表的语句
2012-07-11 16:13:21
谈非线性任务流程的窗口打开方式
2008-08-28 12:47:00
![](https://img.aspxhome.com/file/UploadPic/20088/28/2008828125458515s.jpg)
python脚本框架webpy模板赋值实现
2022-07-06 10:46:54
![](https://img.aspxhome.com/file/2023/4/70024_0s.png)
PHP中使用addslashes函数转义的安全性原理分析
2023-08-16 10:31:22
用户体验设计答疑对话(半吊子和纯外行)
2010-11-30 21:27:00
![](https://img.aspxhome.com/file/UploadPic/201011/30/p1030884-13s.jpg)
Golang实现文件传输功能
2023-08-05 13:50:31
![](https://img.aspxhome.com/file/2023/8/90478_0s.jpg)
成为一个顶级设计师的第一准则
2008-04-18 10:29:00
![](https://img.aspxhome.com/file/UploadPic/20084/18/2008418103351248s.jpg)
python读取和保存图片5种方法对比
2022-05-27 23:54:32
MySQL配置文件my.cnf中文版
2011-09-30 11:06:15
用于WebKit的CSS诀窍[译]
2009-03-11 20:03:00
![](https://img.aspxhome.com/file/UploadPic/20093/11/01-94s.jpg)
Web设计色彩速查表
2009-12-21 16:24:00
python语言元素知识点详解
2023-07-30 03:33:08
SQL查询入门(中篇)
2011-09-30 11:15:09
html元素input使用方法
2007-12-06 13:02:00
php调用快递鸟接口实例代码
2023-11-17 13:48:49
![](https://img.aspxhome.com/file/2023/3/55743_0s.png)
SQL语句练习实例之六 人事系统中的缺勤(休假)统计
2011-11-03 17:15:55
Oracle与MySQL删除字段时对索引和约束的处理
2008-12-26 16:41:00