Python利用requests模块下载图片实例代码
作者:傲娇的草履虫 时间:2023-11-18 16:10:13
本文主要介绍的是关于Python利用requests模块下载图片的相关,下面话不多说了,来一起看看详细的介绍吧
MySQL中事先保存好爬取到的图片链接地址。
然后使用多线程把图片下载到本地。
示例代码:
# coding: utf-8
import MySQLdb
import requests
import os
import re
from threading import Thread
import datetime
header = {'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) '
'Chrome/63.0.3239.132 Safari/537.36'}
file_path = 'F:\\mlu2'
if not os.path.exists(file_path):
os.mkdir(file_path)
class Spider(object):
def __init__(self, file_path, header):
self.file_path = file_path
self.header = header
@staticmethod
def timer(func):
def time_count(*args):
start_time = datetime.datetime.now()
func(*args)
end_time = datetime.datetime.now()
day = (end_time - start_time).days
times = (end_time - start_time).seconds
hour = times / 3600
h = times % 3600
minute = h / 60
m = h % 60
second = m
print "爬取完成"
print "一共用时%s天%s时%s分%s秒" % (day, hour, minute, second)
return time_count
def get_link(self):
conn = MySQLdb.connect(host='localhost',
port=3306,
user='root',
passwd='729814',
db='mlu',
charset='utf8')
cur = conn.cursor()
sql = 'select image from msg limit 100' # image为事先爬取存到MySQL的图片链接地址
cur.execute(sql)
img_link = cur.fetchall()
return img_link
def download(self, link):
filename = re.findall(r'.*/(.+)', link)[0]
try:
pic = requests.get(link, headers=self.header)
if pic.status_code == 200:
with open(os.path.join(self.file_path)+os.sep+filename, 'wb') as fp:
fp.write(pic.content)
fp.close()
print "下载完成"
except Exception as e:
print e
@timer
def run_main(self):
threads = []
links = self.get_link()
for link in links:
img = str(link[0])
t = Thread(target=self.download, args=[img])
t.start()
threads.append(t)
for t in threads:
t.join()
spider = Spider(file_path, header)
spider.run_main()
来源:https://www.cnblogs.com/delav/p/9398825.html
标签:python,requests,下载


猜你喜欢
实例操作MySQL短链接
2024-01-16 00:07:42

用python + openpyxl处理excel2007文档思路以及心得
2022-10-14 04:56:56
MySQL的Query Cache图文详解
2024-01-28 01:18:42

MySQL数据库索引order by排序精讲
2024-01-17 09:33:43

Django连接MQTT的示例代码
2022-10-04 22:40:39
首页访问感受提升三步曲
2007-12-13 20:36:00

django框架中间件原理与用法详解
2022-12-15 03:25:37

从pandas一个单元格的字符串中提取字符串方式
2022-10-14 21:24:35
REPAIR TABLE语法介绍——MySQL数据库
2012-01-05 19:08:59
彻底解决ewebeditor网站后台不能上传图片的方法
2023-07-09 04:09:01
python celery分布式任务队列的使用详解
2021-04-22 18:17:56

pandas库中 DataFrame的用法小结
2021-08-11 00:54:01

python方法如何实现字符串反转
2022-10-26 22:07:03

Pycharm运行程序时,控制台输出PyDev console:starting问题
2022-09-27 14:26:32

oracle 常见等待事件及处理方法
2009-04-24 12:01:00
Python编程实现线性回归和批量梯度下降法代码实例
2021-10-13 07:33:27

基于python进行桶排序与基数排序的总结
2023-06-13 17:32:33
python for循环remove同一个list过程解析
2023-03-20 22:07:48
Pandas中GroupBy具体用法详解
2023-08-10 04:16:42
基于go-cqhttp与Flask搭建定制机器人项目实战示例
2024-04-26 17:30:32
