pytorch中的named_parameters()和parameters()
pytorch named_parameters()和parameters()
nn.Module
nn.Module里面关于参数有两个很重要的属性named_parameters()和parameters(),前者给出网络层的名字和参数的迭代器,而后者仅仅是参数的迭代器。
import torchvision.models as models model = models.resnet18() for param in model.named_parameters(): print(param[0]) ''' conv1.weight bn1.weight bn1.bias layer1.0.conv1.weight layer1.0.bn1.weight layer1.0.bn1.bias layer1.0.conv2.weight layer1.0.bn2.weight layer1.0.bn2.bias layer1.1.conv1.weight layer1.1.bn1.weight layer1.1.bn1.bias layer1.1.conv2.weight layer1.1.bn2.weight layer1.1.bn2.bias layer2.0.conv1.weight layer2.0.bn1.weight layer2.0.bn1.bias layer2.0.conv2.weight layer2.0.bn2.weight layer2.0.bn2.bias layer2.0.downsample.0.weight layer2.0.downsample.1.weight layer2.0.downsample.1.bias layer2.1.conv1.weight layer2.1.bn1.weight layer2.1.bn1.bias layer2.1.conv2.weight layer2.1.bn2.weight layer2.1.bn2.bias layer3.0.conv1.weight layer3.0.bn1.weight layer3.0.bn1.bias layer3.0.conv2.weight layer3.0.bn2.weight layer3.0.bn2.bias layer3.0.downsample.0.weight layer3.0.downsample.1.weight layer3.0.downsample.1.bias layer3.1.conv1.weight layer3.1.bn1.weight layer3.1.bn1.bias layer3.1.conv2.weight layer3.1.bn2.weight layer3.1.bn2.bias layer4.0.conv1.weight layer4.0.bn1.weight layer4.0.bn1.bias layer4.0.conv2.weight layer4.0.bn2.weight layer4.0.bn2.bias layer4.0.downsample.0.weight layer4.0.downsample.1.weight layer4.0.downsample.1.bias layer4.1.conv1.weight layer4.1.bn1.weight layer4.1.bn1.bias layer4.1.conv2.weight layer4.1.bn2.weight layer4.1.bn2.bias fc.weight fc.bias '''
模型参数:named_parameters()、parameters()、state_dict()区别
torch中存在3个功能极其类似的方法,它们分别是model.parameters()、model.named_parameters()、model.state_dict(),
下面就具体来说说这三个函数的差异:
一、model.parameters()和model.named_parameters()差别
- named_parameters()返回的list中,每个元组(与list相似,只是数据不可修改)打包了2个内容,分别是
layer-name
和layer-param
(网络层的名字和参数的迭代器); - parameters()只有后者layer-param(参数的迭代器)
1、model.named_parameters()里的网络层名字
import torchvision.models as models model = models.resnet18() for param_tuple in model.named_parameters(): name, param = param_tuple print("name = ", name) print("-" * 100)
打印结果:
name = conv1.weight
----------------------------------------------------------------------------------------------------
name = bn1.weight
----------------------------------------------------------------------------------------------------
name = bn1.bias
----------------------------------------------------------------------------------------------------
name = layer1.0.conv1.weight
----------------------------------------------------------------------------------------------------
name = layer1.0.bn1.weight
----------------------------------------------------------------------------------------------------
name = layer1.0.bn1.bias
----------------------------------------------------------------------------------------------------
name = layer1.0.conv2.weight
----------------------------------------------------------------------------------------------------
name = layer1.0.bn2.weight
----------------------------------------------------------------------------------------------------
name = layer1.0.bn2.bias
----------------------------------------------------------------------------------------------------
name = layer1.1.conv1.weight
----------------------------------------------------------------------------------------------------
name = layer1.1.bn1.weight
----------------------------------------------------------------------------------------------------
name = layer1.1.bn1.bias
----------------------------------------------------------------------------------------------------
name = layer1.1.conv2.weight
----------------------------------------------------------------------------------------------------
name = layer1.1.bn2.weight
----------------------------------------------------------------------------------------------------
name = layer1.1.bn2.bias
----------------------------------------------------------------------------------------------------
name = layer2.0.conv1.weight
----------------------------------------------------------------------------------------------------
name = layer2.0.bn1.weight
----------------------------------------------------------------------------------------------------
name = layer2.0.bn1.bias
----------------------------------------------------------------------------------------------------
name = layer2.0.conv2.weight
----------------------------------------------------------------------------------------------------
name = layer2.0.bn2.weight
----------------------------------------------------------------------------------------------------
name = layer2.0.bn2.bias
----------------------------------------------------------------------------------------------------
name = layer2.0.downsample.0.weight
----------------------------------------------------------------------------------------------------
name = layer2.0.downsample.1.weight
----------------------------------------------------------------------------------------------------
name = layer2.0.downsample.1.bias
----------------------------------------------------------------------------------------------------
name = layer2.1.conv1.weight
----------------------------------------------------------------------------------------------------
name = layer2.1.bn1.weight
----------------------------------------------------------------------------------------------------
name = layer2.1.bn1.bias
----------------------------------------------------------------------------------------------------
name = layer2.1.conv2.weight
----------------------------------------------------------------------------------------------------
name = layer2.1.bn2.weight
----------------------------------------------------------------------------------------------------
name = layer2.1.bn2.bias
----------------------------------------------------------------------------------------------------
name = layer3.0.conv1.weight
----------------------------------------------------------------------------------------------------
name = layer3.0.bn1.weight
----------------------------------------------------------------------------------------------------
name = layer3.0.bn1.bias
----------------------------------------------------------------------------------------------------
name = layer3.0.conv2.weight
----------------------------------------------------------------------------------------------------
name = layer3.0.bn2.weight
----------------------------------------------------------------------------------------------------
name = layer3.0.bn2.bias
----------------------------------------------------------------------------------------------------
name = layer3.0.downsample.0.weight
----------------------------------------------------------------------------------------------------
name = layer3.0.downsample.1.weight
----------------------------------------------------------------------------------------------------
name = layer3.0.downsample.1.bias
----------------------------------------------------------------------------------------------------
name = layer3.1.conv1.weight
----------------------------------------------------------------------------------------------------
name = layer3.1.bn1.weight
----------------------------------------------------------------------------------------------------
name = layer3.1.bn1.bias
----------------------------------------------------------------------------------------------------
name = layer3.1.conv2.weight
----------------------------------------------------------------------------------------------------
name = layer3.1.bn2.weight
----------------------------------------------------------------------------------------------------
name = layer3.1.bn2.bias
----------------------------------------------------------------------------------------------------
name = layer4.0.conv1.weight
----------------------------------------------------------------------------------------------------
name = layer4.0.bn1.weight
----------------------------------------------------------------------------------------------------
name = layer4.0.bn1.bias
----------------------------------------------------------------------------------------------------
name = layer4.0.conv2.weight
----------------------------------------------------------------------------------------------------
name = layer4.0.bn2.weight
----------------------------------------------------------------------------------------------------
name = layer4.0.bn2.bias
----------------------------------------------------------------------------------------------------
name = layer4.0.downsample.0.weight
----------------------------------------------------------------------------------------------------
name = layer4.0.downsample.1.weight
----------------------------------------------------------------------------------------------------
name = layer4.0.downsample.1.bias
----------------------------------------------------------------------------------------------------
name = layer4.1.conv1.weight
----------------------------------------------------------------------------------------------------
name = layer4.1.bn1.weight
----------------------------------------------------------------------------------------------------
name = layer4.1.bn1.bias
----------------------------------------------------------------------------------------------------
name = layer4.1.conv2.weight
----------------------------------------------------------------------------------------------------
name = layer4.1.bn2.weight
----------------------------------------------------------------------------------------------------
name = layer4.1.bn2.bias
----------------------------------------------------------------------------------------------------
name = fc.weight
----------------------------------------------------------------------------------------------------
name = fc.bias
----------------------------------------------------------------------------------------------------Process finished with exit code 0
2、model.named_parameters()里的网络层名字、参数
import torchvision.models as models model = models.resnet18() for param_tuple in model.named_parameters(): name, param = param_tuple print("name = ", name) print("-" * 100) print("param_tuple = ", param_tuple) print("*" * 200)
打印结果:
C:\Program_Files_AI\Anaconda3531\python.exe C:/Users/Admin/OneDrive/WorkSpace_AI/0-基于知识库的智能问答系统-华控智加/01-意图识别/test.py
name = conv1.weight
----------------------------------------------------------------------------------------------------
param_tuple = ('conv1.weight', Parameter containing:
tensor([[[[-1.4115e-05, 2.9187e-02, 2.9325e-03, ..., -4.2247e-02,
1.7490e-02, -4.5253e-02],
[-2.4594e-02, -3.0836e-02, 3.8604e-02, ..., 3.5473e-02,
-4.7046e-03, -2.9440e-02],
[ 2.4811e-02, 1.2679e-02, 1.0070e-02, ..., -8.3476e-03,
1.7960e-02, -1.7406e-02],
...,
[-1.3021e-02, 2.9023e-02, -6.1800e-02, ..., -5.2802e-02,
-4.7817e-02, -2.2377e-02],
[-3.8513e-03, -1.0603e-02, -3.9712e-02, ..., 5.1941e-03,
8.2868e-03, -8.3469e-03],
[ 3.8993e-03, 3.2017e-02, -3.6292e-02, ..., -2.0210e-02,
-4.0358e-02, 1.7709e-02]],[[-1.0894e-03, 1.5720e-02, 7.0129e-03, ..., -1.2024e-02,
1.8644e-02, 1.7892e-02],
[-2.3866e-02, 9.1136e-03, 3.5243e-02, ..., -1.6756e-02,
1.4441e-03, 4.7943e-02],
[-2.0514e-03, 4.3022e-02, 2.6358e-02, ..., -2.3662e-02,
-7.8241e-04, 1.0167e-02],
...[[-4.6689e-02, -1.1407e-03, 1.8674e-02, ..., 1.2649e-03,
-2.9532e-02, 6.4535e-04],
[ 1.4171e-03, -1.9274e-02, -8.6811e-03, ..., 2.4428e-02,
6.9516e-03, 4.3715e-02],
[ 1.9982e-02, 1.3124e-02, 9.1508e-03, ..., 2.5405e-02,
-1.3132e-02, 4.0835e-02],
...,
[-3.4174e-03, 1.8623e-02, -1.4386e-02, ..., 1.0627e-03,
-5.1297e-04, 2.2055e-02],
[ 2.7333e-02, 2.4858e-02, -5.4305e-02, ..., -1.2139e-02,
1.7735e-03, -3.4184e-03],
[ 1.1412e-03, 1.5794e-02, -2.0699e-02, ..., -1.7846e-02,
3.7425e-02, -1.6059e-02]]],
...,
[[[-2.7389e-02, -3.8327e-02, -2.9043e-02, ..., -7.6396e-03,
-1.6519e-02, 3.9659e-02],
[ 2.8740e-03, -1.0621e-02, -9.2430e-03, ..., 2.2581e-02,
5.1526e-03, -2.0006e-02],
[ 1.3575e-02, 1.5290e-02, -1.7260e-02, ..., 6.3830e-03,
-1.9759e-02, 1.5501e-02],
...,
[ 1.6091e-02, 2.4038e-02, 2.4507e-02, ..., -4.5613e-02,
-3.6233e-02, 2.1632e-02],
[-1.1573e-02, -3.6514e-02, 4.1576e-02, ..., 1.8090e-02,
-2.3350e-02, -8.7074e-03],
[-1.5837e-02, -3.1353e-02, 1.8726e-02, ..., 9.3698e-03,
3.0781e-02, 1.0976e-02]],[[-2.7063e-02, 8.7158e-03, 2.7193e-03, ..., -1.6670e-03,
-4.3033e-03, 7.2011e-04],
[ 2.7870e-03, 1.4264e-02, -5.0581e-02, ..., 2.5463e-02,
7.6864e-03, -4.9655e-02],
[ 2.6030e-03, 2.5918e-02, 2.9615e-02, ..., 3.0676e-02,
-2.7723e-02, -7.3628e-03],
...,
[ 2.5969e-02, -1.4247e-02, 1.2516e-02, ..., 5.9602e-03,
-3.2843e-02, 3.5822e-02],
[ 1.2845e-02, -2.0035e-02, 9.9398e-04, ..., -3.1800e-02,
5.7984e-03, 2.8756e-02],
[ 2.3458e-02, 3.8193e-02, -2.3754e-03, ..., -1.3867e-02,
8.0831e-03, -3.2438e-02]],
...[[-9.9291e-03, -5.6023e-03, -1.7064e-02, ..., 8.8544e-03,
-5.8145e-03, 2.3248e-02],
[ 1.2148e-02, -1.0730e-02, -1.2682e-02, ..., 9.4389e-03,
1.2149e-02, 3.8613e-03],
[ 3.5913e-02, -5.2048e-04, -8.7133e-02, ..., -2.0969e-03,
-5.4117e-03, 5.4637e-05],
...,
[ 4.0351e-03, -1.3189e-02, 3.1229e-02, ..., 3.2340e-02,
-2.8351e-02, 1.0634e-02],
[ 2.6041e-02, -3.0633e-04, -1.2732e-02, ..., 2.9417e-02,
-7.3859e-03, 1.7207e-02],
[ 6.9960e-04, 3.8486e-03, 1.0397e-02, ..., 1.4535e-03,
-3.6449e-02, 3.4848e-02]]]], requires_grad=True))
********************************************************************************************************************************************************************************************************
name = bn1.weight
----------------------------------------------------------------------------------------------------
param_tuple = ('bn1.weight', Parameter containing:
tensor([1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1., 1., 1., 1., 1., 1.], requires_grad=True))
********************************************************************************************************************************************************************************************************
name = bn1.bias
----------------------------------------------------------------------------------------------------
param_tuple = ('bn1.bias', Parameter containing:
tensor([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
requires_grad=True))
****************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************
name = layer1.0.conv2.weight
----------------------------------------------------------------------------------------------------
param_tuple = ('layer1.0.conv2.weight', Parameter containing:
tensor([[[[-8.6159e-02, 1.8507e-04, 5.4006e-03],
[-6.3063e-03, 3.9225e-03, -6.3141e-02],
[-7.0145e-02, -3.9266e-02, 1.9724e-03]],[[ 4.6454e-02, 2.1519e-02, 5.3696e-02],
[ 1.1086e-02, 1.6269e-01, -7.0579e-02],
[-1.1220e-01, -4.9811e-02, -7.5515e-02]],[[ 5.5275e-02, -8.2407e-02, -8.9807e-02],
[ 5.8418e-02, 4.4029e-02, 3.0584e-03],
[ 5.2371e-02, -1.5983e-02, 5.1494e-02]],...,
[[ 7.3441e-02, 4.5401e-02, -1.9175e-02],
[-6.2500e-02, -8.5905e-03, -7.4856e-02],
[-1.6170e-02, 3.7529e-02, -5.1231e-02]],[[ 7.7501e-04, -5.7506e-02, 1.8422e-01],
[ 2.4594e-02, 1.7378e-02, 4.0000e-02],
[-8.6796e-02, -6.0548e-02, 2.6795e-02]],[[ 5.3264e-02, -8.6190e-02, 4.2443e-02],
[-6.8029e-03, -1.6581e-02, 7.8568e-02],
[ 3.2037e-02, -7.3002e-02, 4.9353e-02]]]], requires_grad=True))
...,
********************************************************************************************************************************************************************************************************
name = layer4.1.bn2.bias
----------------------------------------------------------------------------------------------------
param_tuple = ('layer4.1.bn2.bias', Parameter containing:
tensor([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
...,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0.], requires_grad=True))
********************************************************************************************************************************************************************************************************
name = fc.weight
----------------------------------------------------------------------------------------------------
param_tuple = ('fc.weight', Parameter containing:
tensor([[-0.0125, 0.0437, -0.0014, ..., -0.0230, 0.0280, 0.0249],
[-0.0105, 0.0242, 0.0291, ..., 0.0153, 0.0366, -0.0236],
[-0.0315, 0.0306, -0.0216, ..., 0.0387, 0.0403, 0.0056],
...,
[-0.0068, -0.0222, -0.0027, ..., -0.0243, 0.0260, 0.0065],
[ 0.0213, 0.0167, -0.0379, ..., -0.0140, 0.0037, -0.0372],
[ 0.0180, 0.0101, -0.0341, ..., -0.0295, -0.0146, 0.0416]],
requires_grad=True))
********************************************************************************************************************************************************************************************************
name = fc.bias
----------------------------------------------------------------------------------------------------
param_tuple = ('fc.bias', Parameter containing:
tensor([ 3.5711e-02, 3.2682e-02, 7.5932e-03, -3.1623e-02, -9.6316e-03,
-2.4051e-02, -1.0393e-02, 2.3210e-02, -3.6044e-02, 2.3099e-02,
-3.5723e-02, -3.9482e-02, 4.8526e-03, -3.2688e-02, 3.7720e-03,
-2.2014e-02, -4.0935e-02, 4.0533e-02, -4.1172e-02, 3.9513e-02,
-3.0332e-02, 3.2777e-02, 1.3342e-02, 2.3394e-02, 8.2328e-03,
1.3757e-02, -1.7578e-02, -2.7165e-02, 3.8495e-03, -3.2116e-02,
7.9903e-03, 9.9640e-04, -8.3106e-03, 2.5033e-02, -3.0446e-02,
-1.8282e-02, -3.8420e-03, -8.6129e-03, -4.2712e-03, 1.7169e-02,
...,
-5.3570e-05, -3.7353e-02, -9.8633e-03, -9.1069e-03, 3.2688e-02,
2.2457e-02, 7.6379e-03, -3.6287e-02, -1.0444e-02, 2.1669e-02,
2.5270e-02, -4.3881e-02, 2.1960e-02, 2.6293e-02, -3.5049e-02,
-2.0074e-02, -9.7686e-03, -2.3766e-02, -5.0265e-03, -2.1095e-02,
2.0981e-02, -3.5132e-02, 8.6407e-03, 1.8453e-02, 2.4282e-02,
3.8392e-02, -1.7470e-02, 3.6958e-02, -3.7590e-02, -4.1951e-02,
-1.8246e-02, 9.0818e-03, 3.8774e-02, 7.3408e-03, 1.7728e-02,
3.5547e-02, -7.2857e-03, -2.7015e-02, -8.6983e-03, -2.3785e-02],
requires_grad=True))
********************************************************************************************************************************************************************************************************Process finished with exit code 0
二、model.named_parameters()和model.state_dict()差别
它们的差异主要体现在3方面:
- 返回值类型不同
- 存储的模型参数的种类不同
- 返回的值的require_grad属性不同
named_parameters() | state_dict() |
---|---|
将layer_name : layer_param的键值信息打包成一个元祖然后再存到list当中 | 将layer_name : layer_param的键值信息存储为dict形式 |
只保存可学习、可被更新的参数,model.buffer()中的参数不包含在model.named_parameters()中 | 存储的是该model中包含的所有layer中的所有参数 |
require_grad属性都是True | 存储的模型参数tensor的require_grad属性都是False |
为何model.parameters()迭代出来的所有参数的require_grad属性都是True,因为它们在被创建时,默认的require_grad就是True。
这也符合逻辑,即,使用nn.Parameter()创建的变量是模型参数,本就是要参与学习和更新的
总结
以上为个人经验,希望能给大家一个参考,也希望大家多多支持脚本之家。
相关文章
Python中json.load()和json.loads()有哪些区别
json.loads()用于解析一个有效的JSON字符串并将其转换为Python字典,json.load——()用于从一个文件读取JSON类型的数据,然后转转换成Python字典,本文讲解下python中两者的使用2021-06-06在django项目中导出数据到excel文件并实现下载的功能
这篇文章主要介绍了在django项目中导出数据到excel文件并实现下载的功能,具有很好的参考价值,希望对大家有所帮助。一起跟随小编过来看看吧2020-03-03
最新评论