pytorch + visdom CNN处理自建图片数据集的方法
作者:泛泛之素 时间:2023-05-03 06:33:16
环境
系统:win10
cpu:i7-6700HQ
gpu:gtx965m
python : 3.6
pytorch :0.3
数据下载
来源自Sasank Chilamkurthy 的教程; 数据:下载链接。
下载后解压放到项目根目录:
数据集为用来分类 蚂蚁和蜜蜂。有大约120个训练图像,每个类有75个验证图像。
数据导入
可以使用 torchvision.datasets.ImageFolder(root,transforms) 模块 可以将 图片转换为 tensor。
先定义transform:
ata_transforms = {
'train': transforms.Compose([
# 随机切成224x224 大小图片 统一图片格式
transforms.RandomResizedCrop(224),
# 图像翻转
transforms.RandomHorizontalFlip(),
# totensor 归一化(0,255) >> (0,1) normalize channel=(channel-mean)/std
transforms.ToTensor(),
transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
]),
"val" : transforms.Compose([
# 图片大小缩放 统一图片格式
transforms.Resize(256),
# 以中心裁剪
transforms.CenterCrop(224),
transforms.ToTensor(),
transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
])
}
导入,加载数据:
data_dir = './hymenoptera_data'
# trans data
image_datasets = {x: datasets.ImageFolder(os.path.join(data_dir, x), data_transforms[x]) for x in ['train', 'val']}
# load data
data_loaders = {x: DataLoader(image_datasets[x], batch_size=BATCH_SIZE, shuffle=True) for x in ['train', 'val']}
data_sizes = {x: len(image_datasets[x]) for x in ['train', 'val']}
class_names = image_datasets['train'].classes
print(data_sizes, class_names)
{'train': 244, 'val': 153} ['ants', 'bees']
训练集 244图片 , 测试集153图片 。
可视化部分图片看看,由于visdom支持tensor输入 ,不用换成numpy,直接用tensor计算即可 :
inputs, classes = next(iter(data_loaders['val']))
out = torchvision.utils.make_grid(inputs)
inp = torch.transpose(out, 0, 2)
mean = torch.FloatTensor([0.485, 0.456, 0.406])
std = torch.FloatTensor([0.229, 0.224, 0.225])
inp = std * inp + mean
inp = torch.transpose(inp, 0, 2)
viz.images(inp)
创建CNN
net 根据上一篇的处理cifar10的改了一下规格:
class CNN(nn.Module):
def __init__(self, in_dim, n_class):
super(CNN, self).__init__()
self.cnn = nn.Sequential(
nn.BatchNorm2d(in_dim),
nn.ReLU(True),
nn.Conv2d(in_dim, 16, 7), # 224 >> 218
nn.BatchNorm2d(16),
nn.ReLU(inplace=True),
nn.MaxPool2d(2, 2), # 218 >> 109
nn.ReLU(True),
nn.Conv2d(16, 32, 5), # 105
nn.BatchNorm2d(32),
nn.ReLU(True),
nn.Conv2d(32, 64, 5), # 101
nn.BatchNorm2d(64),
nn.ReLU(True),
nn.Conv2d(64, 64, 3, 1, 1),
nn.BatchNorm2d(64),
nn.ReLU(True),
nn.MaxPool2d(2, 2), # 101 >> 50
nn.Conv2d(64, 128, 3, 1, 1), #
nn.BatchNorm2d(128),
nn.ReLU(True),
nn.MaxPool2d(3), # 50 >> 16
)
self.fc = nn.Sequential(
nn.Linear(128*16*16, 120),
nn.BatchNorm1d(120),
nn.ReLU(True),
nn.Linear(120, n_class))
def forward(self, x):
out = self.cnn(x)
out = self.fc(out.view(-1, 128*16*16))
return out
# 输入3层rgb ,输出 分类 2
model = CNN(3, 2)
loss,优化函数:
line = viz.line(Y=np.arange(10))
loss_f = nn.CrossEntropyLoss()
optimizer = optim.SGD(model.parameters(), lr=LR, momentum=0.9)
scheduler = optim.lr_scheduler.StepLR(optimizer, step_size=7, gamma=0.1)
参数:
BATCH_SIZE = 4
LR = 0.001
EPOCHS = 10
运行 10个 epoch 看看:
[9/10] train_loss:0.650|train_acc:0.639|test_loss:0.621|test_acc0.706
[10/10] train_loss:0.645|train_acc:0.627|test_loss:0.654|test_acc0.686
Training complete in 1m 16s
Best val Acc: 0.712418
运行 20个看看:
[19/20] train_loss:0.592|train_acc:0.701|test_loss:0.563|test_acc0.712
[20/20] train_loss:0.564|train_acc:0.721|test_loss:0.571|test_acc0.706
Training complete in 2m 30s
Best val Acc: 0.745098
准确率比较低:只有74.5%
我们使用models 里的 resnet18 运行 10个epoch:
model = torchvision.models.resnet18(True)
num_ftrs = model.fc.in_features
model.fc = nn.Linear(num_ftrs, 2)
[9/10] train_loss:0.621|train_acc:0.652|test_loss:0.588|test_acc0.667
[10/10] train_loss:0.610|train_acc:0.680|test_loss:0.561|test_acc0.667
Training complete in 1m 24s
Best val Acc: 0.686275
效果也很一般,想要短时间内就训练出效果很好的models,我们可以下载训练好的state,在此基础上训练:
model = torchvision.models.resnet18(pretrained=True)
num_ftrs = model.fc.in_features
model.fc = nn.Linear(num_ftrs, 2)
[9/10] train_loss:0.308|train_acc:0.877|test_loss:0.160|test_acc0.941
[10/10] train_loss:0.267|train_acc:0.885|test_loss:0.148|test_acc0.954
Training complete in 1m 25s
Best val Acc: 0.954248
10个epoch直接的到95%的准确率。
示例代码:https://github.com/ffzs/ml_pytorch/blob/master/ml_pytorch_hymenoptera
来源:https://blog.csdn.net/tonydz0523/article/details/79075077
![](/images/zang.png)
![](/images/jiucuo.png)
猜你喜欢
Python模拟登陆淘宝并统计淘宝消费情况的代码实例分享
![](https://img.aspxhome.com/file/2023/8/86218_0s.gif)
全面了解django的缓存机制及使用方法
![](https://img.aspxhome.com/file/2023/2/85352_0s.png)
python tensorflow基于cnn实现手写数字识别
一个简单的像素画小工具
Python实现多线程的两种方式分析
详解在Anaconda环境下Python安装pydot与graphviz的方法
![](https://img.aspxhome.com/file/2023/7/101657_0s.png)
在win64上使用bypy进行百度网盘文件上传功能
![](https://img.aspxhome.com/file/2023/9/124799_0s.jpg)
Python编写简单的HTML页面合并脚本
pytorch创建tensor函数详情
GOLang单元测试用法详解
![](https://img.aspxhome.com/file/2023/1/128691_0s.png)
在ASP.NET 2.0中操作数据之三十六:在DataList里编辑和删除数据概述
![](https://img.aspxhome.com/file/2023/1/129811_0s.png)
PHP下常用正则表达式整理
PHP PDOStatement::errorInfo讲解
PyTorch实现MNIST数据集手写数字识别详情
Django后台获取前端post上传的文件方法
![](https://img.aspxhome.com/file/2023/9/80539_0s.jpg)
DW表格应用之细线框的制作
![](https://img.aspxhome.com/file/UploadPic/20082/3/20082319946365s.gif)
python 如何读、写、解析CSV文件
![](https://img.aspxhome.com/file/2023/7/97627_0s.jpg)
如何用Python对数学函数进行求值、求偏导
详解Visual Studio中Git的简单使用
![](https://img.aspxhome.com/file/2023/5/132405_0s.png)
OpenCV图像处理之七种常用图像几何变换
![](https://img.aspxhome.com/file/2023/7/66427_0s.jpg)