国产一区日韩精品,成人国产,91视频8mav

在用pdb debug的時候，有時候需要看一下特定layer的權(quán)重以及相應(yīng)的梯度信息，如何查看呢？

1. 首先把你的模型打印出來，像這樣

pytorch查看模型weight與grad方式

2. 然后觀察到model下面有module的key，module下面有features的key， features下面有(0)的key，這樣就可以直接打印出weight了，在pdb debug界面輸入p model.module.features[0].weight，就可以看到weight，輸入 p model.module.features[0].weight.grad就可以查看梯度信息

pytorch查看模型weight與grad方式

補充知識：查看Pytorch網(wǎng)絡(luò)的各層輸出(feature map)、權(quán)重(weight)、偏置(bias)

BatchNorm2d參數(shù)量

									torch.nn.BatchNorm2d(num_features, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

									# 卷積層中卷積核的數(shù)量C 

									num_features – C from an expected input of size (N, C, H, W)

									>>> import torch

									>>> m = torch.nn.BatchNorm2d(100)

									>>> m.weight.shape

									torch.Size([100])

									>>> m.numel()

									AttributeError: 'BatchNorm2d' object has no attribute 'numel'

									>>> m.weight.numel()

									100

									>>> m.parameters().numel()

									Traceback (most recent call last):

									 File "<stdin>", line 1, in <module>

									AttributeError: 'generator' object has no attribute 'numel'

									>>> [p.numel() for p in m.parameters()]

									[100, 100]

linear層

									>>> import torch

									>>> m1 = torch.nn.Linear(100,10)

									# 參數(shù)數(shù)量= (輸入神經(jīng)元+1)*輸出神經(jīng)元

									>>> m1.weight.shape

									torch.Size([10, 100])

									>>> m1.bias.shape

									torch.Size([10])

									>>> m1.bias.numel()

									10

									>>> m1.weight.numel()

									1000

									>>> m11 = list(m1.parameters())

									>>> m11[0].shape

									# weight

									torch.Size([10, 100])

									>>> m11[1].shape

									# bias

									torch.Size([10])

weight and bias

									# Method 1 查看Parameters的方式多樣化，直接訪問即可

									model = alexnet(pretrained=True).to(device)

									conv1_weight = model.features[0].weight# Method 2 

									# 這種方式還適合你想自己參考一個預(yù)訓(xùn)練模型寫一個網(wǎng)絡(luò)，各層的參數(shù)不變，但網(wǎng)絡(luò)結(jié)構(gòu)上表述有所不同

									# 這樣你就可以把param迭代出來，賦給你的網(wǎng)絡(luò)對應(yīng)層，避免直接load不能匹配的問題！

									for layer,param in model.state_dict().items(): # param is weight or bias(Tensor) 

									 print layer,param

feature map

由于pytorch是動態(tài)網(wǎng)絡(luò)，不存儲計算數(shù)據(jù)，查看各層輸出的特征圖并不是很方便！分下面兩種情況討論：

1、你想查看的層是獨立的,那么你在forward時用變量接收并返回即可！！

									class Net(nn.Module):

									  def __init__(self):

									    self.conv1 = nn.Conv2d(1, 1, 3)

									    self.conv2 = nn.Conv2d(1, 1, 3)

									    self.conv3 = nn.Conv2d(1, 1, 3)  def forward(self, x):

									    out1 = F.relu(self.conv1(x))

									    out2 = F.relu(self.conv2(out1))

									    out3 = F.relu(self.conv3(out2))

									    return out1, out2, out3

2、你的想看的層在nn.Sequential()順序容器中，這個麻煩些，主要有以下幾種思路：

									# Method 1 巧用nn.Module.children()

									# 在模型實例化之后，利用nn.Module.children()刪除你查看的那層的后面層

									import torch

									import torch.nn as nn

									from torchvision import modelsmodel = models.alexnet(pretrained=True)# remove last fully-connected layer

									new_classifier = nn.Sequential(*list(model.classifier.children())[:-1])

									model.classifier = new_classifier

									# Third convolutional layer

									new_features = nn.Sequential(*list(model.features.children())[:5])

									model.features = new_features

									# Method 2 巧用hook,推薦使用這種方式，不用改變原有模型

									# torch.nn.Module.register_forward_hook(hook)

									# hook(module, input, output) -> Nonemodel = models.alexnet(pretrained=True)

									# 定義

									def hook (module,input,output):

									  print output.size()

									# 注冊

									handle = model.features[0].register_forward_hook(hook)

									# 刪除句柄

									handle.remove()# torch.nn.Module.register_backward_hook(hook)

									# hook(module, grad_input, grad_output) -> Tensor or None

									model = alexnet(pretrained=True).to(device)

									outputs = []

									def hook (module,input,output):

									  outputs.append(output)

									  print len(outputs)handle = model.features[0].register_backward_hook(hook)