моя точность нейронной сети и тестовые потери застряли при построении модели на HAR .Модель была построена с использованием только переменных временных рядов (экспертные функции не использовались)
Я перепробовал много курсов обучения и оптимизаторов, даже тогда, с 1 по 200 годы, они зависали с точностью 18%
Используемая архитектура была от здесь И я не мог понять, действительно ли архитектура плохая или в моем коде есть ошибка.
class Net(nn.Module):
def __init__(self):
super().__init__()
self.model = nn.ModuleDict({
'lstm': nn.LSTM(
input_size=64,
hidden_size=64,batch_first=True
),
'linear': nn.Linear(
in_features=128,
out_features=6),
'dropout': nn.Dropout(0.7),
'lstm2': nn.LSTM(
input_size=64,
hidden_size=128,batch_first=True),
'linear2': nn.Linear(
in_features=9,
out_features=64)})
def forward(self,x):
x = x.permute(1,0,2)
x = x.contiguous().view(-1,9)
x = self.model['linear2'](x)
x = x.contiguous().view(-1,128,64)
x, _ = self.model['lstm'](x)
x = self.model['dropout'](x)
x, _ = self.model['lstm2'](x)
x = self.model['dropout'](x)
x = self.model['linear'](x[:,-1,:])
return x
net=Net().to(device)
def train( model, device, train_loader, optimizer, epoch,):
model.train()
train_loss=0
for batch_idx, (data, target) in enumerate(train_loader):
data, target = data.to(device), target.view(-1).to(device)
optimizer.zero_grad()
output = model(data.float())
train_loss+=F.cross_entropy(output, target).item()
loss = F.cross_entropy(output, target)
loss.backward()
optimizer.step()
train_loss /= np.ceil(len(train_loader.dataset)/bs)
print('Train Epoch: {} \tLoss per Batch: {:.6f}'.format(epoch,train_loss))
def test( model, device, test_loader):
model.eval()
test_loss = 0
correct = 0
with torch.no_grad():
for batch_idx,(data, target) in enumerate(test_loader):
data, target = data.to(device), target.to(device)
output = model(data.float())
test_loss += F.cross_entropy(output, target).item() # sum up batch loss
pred = output.argmax(dim=1, keepdim=True) # get the index of the max log-probability
correct += pred.eq(target.view_as(pred)).sum().item()
test_loss /= np.ceil(len(test_loader.dataset)/bs)
print('\nTest set: Loss per Batch: {:.4f}, Accuracy: {}/{} ({:.0f}%)\n'.format(
test_loss, correct, len(test_loader.dataset),
100. * correct / len(test_loader.dataset)))
import torch.optim as optim
optimizer = optim.Adam(net.parameters(), lr=.01)
Используя Адама с lr = .01, я получил следующие результаты:
потери на партию и точность набора тестов оставались постоянными в течение всего периода обучения 70эпох
Train Epoch: 0 Loss per Batch: 1.788564
Test set: Loss per Batch: 1.7989, Accuracy: 533/2947 (18%)
Train Epoch: 1 Loss per Batch: 1.786124
Test set: Loss per Batch: 1.7926, Accuracy: 537/2947 (18%)
Train Epoch: 2 Loss per Batch: 1.785584
Test set: Loss per Batch: 1.7947, Accuracy: 537/2947 (18%)
Train Epoch: 3 Loss per Batch: 1.785016
Test set: Loss per Batch: 1.8117, Accuracy: 537/2947 (18%)
Train Epoch: 4 Loss per Batch: 1.786215
Test set: Loss per Batch: 1.7921, Accuracy: 537/2947 (18%)
Train Epoch: 5 Loss per Batch: 1.785138
Test set: Loss per Batch: 1.7939, Accuracy: 537/2947 (18%)
Train Epoch: 6 Loss per Batch: 1.784805
Test set: Loss per Batch: 1.7932, Accuracy: 537/2947 (18%)
Train Epoch: 7 Loss per Batch: 1.784633
Test set: Loss per Batch: 1.7932, Accuracy: 537/2947 (18%)
Train Epoch: 8 Loss per Batch: 1.784855
Test set: Loss per Batch: 1.7938, Accuracy: 537/2947 (18%)
Train Epoch: 9 Loss per Batch: 1.785028
Test set: Loss per Batch: 1.7936, Accuracy: 537/2947 (18%)
Train Epoch: 10 Loss per Batch: 1.784683
Test set: Loss per Batch: 1.7930, Accuracy: 537/2947 (18%)
Train Epoch: 11 Loss per Batch: 1.785211
Test set: Loss per Batch: 1.7930, Accuracy: 527/2947 (18%)
Train Epoch: 12 Loss per Batch: 1.784648
Test set: Loss per Batch: 1.7912, Accuracy: 514/2947 (17%)
Train Epoch: 13 Loss per Batch: 1.784625
Test set: Loss per Batch: 1.7909, Accuracy: 537/2947 (18%)
Train Epoch: 14 Loss per Batch: 1.785082
Test set: Loss per Batch: 1.7921, Accuracy: 537/2947 (18%)
Train Epoch: 15 Loss per Batch: 1.785162
Test set: Loss per Batch: 1.7931, Accuracy: 509/2947 (17%)
Train Epoch: 16 Loss per Batch: 1.784732
Test set: Loss per Batch: 1.7932, Accuracy: 532/2947 (18%)
Train Epoch: 17 Loss per Batch: 1.784819
Test set: Loss per Batch: 1.7949, Accuracy: 537/2947 (18%)
Train Epoch: 18 Loss per Batch: 1.785135
Test set: Loss per Batch: 1.7927, Accuracy: 537/2947 (18%)
Train Epoch: 19 Loss per Batch: 1.784840
Test set: Loss per Batch: 1.7933, Accuracy: 532/2947 (18%)
Train Epoch: 20 Loss per Batch: 1.784750
Test set: Loss per Batch: 1.7939, Accuracy: 532/2947 (18%)
Train Epoch: 21 Loss per Batch: 1.784810
Test set: Loss per Batch: 1.7940, Accuracy: 517/2947 (18%)
Train Epoch: 22 Loss per Batch: 1.784939
Test set: Loss per Batch: 1.7872, Accuracy: 537/2947 (18%)
Train Epoch: 23 Loss per Batch: 1.785099
Test set: Loss per Batch: 1.7926, Accuracy: 537/2947 (18%)
Train Epoch: 24 Loss per Batch: 1.784571
Test set: Loss per Batch: 1.7918, Accuracy: 537/2947 (18%)
Train Epoch: 25 Loss per Batch: 1.784607
Test set: Loss per Batch: 1.7922, Accuracy: 532/2947 (18%)
Train Epoch: 26 Loss per Batch: 1.784505
Test set: Loss per Batch: 1.7902, Accuracy: 514/2947 (17%)
Train Epoch: 27 Loss per Batch: 1.785079
Test set: Loss per Batch: 1.7932, Accuracy: 532/2947 (18%)
Train Epoch: 28 Loss per Batch: 1.784903
Test set: Loss per Batch: 1.7933, Accuracy: 532/2947 (18%)
Train Epoch: 29 Loss per Batch: 1.784797
Test set: Loss per Batch: 1.7938, Accuracy: 537/2947 (18%)
Train Epoch: 30 Loss per Batch: 1.784668
Test set: Loss per Batch: 1.7940, Accuracy: 537/2947 (18%)
Train Epoch: 31 Loss per Batch: 1.784762
Test set: Loss per Batch: 1.7934, Accuracy: 537/2947 (18%)
Train Epoch: 32 Loss per Batch: 1.784556
Test set: Loss per Batch: 1.7947, Accuracy: 532/2947 (18%)
Train Epoch: 33 Loss per Batch: 1.784983
Test set: Loss per Batch: 1.7935, Accuracy: 532/2947 (18%)
Train Epoch: 34 Loss per Batch: 1.785202
Test set: Loss per Batch: 1.7945, Accuracy: 532/2947 (18%)
Train Epoch: 35 Loss per Batch: 1.785080
Test set: Loss per Batch: 1.7921, Accuracy: 532/2947 (18%)
Train Epoch: 36 Loss per Batch: 1.784834
Test set: Loss per Batch: 1.7925, Accuracy: 532/2947 (18%)
Train Epoch: 37 Loss per Batch: 1.784711
Test set: Loss per Batch: 1.7938, Accuracy: 532/2947 (18%)
Train Epoch: 38 Loss per Batch: 1.784832
Test set: Loss per Batch: 1.7922, Accuracy: 537/2947 (18%)
Train Epoch: 39 Loss per Batch: 1.784580
Test set: Loss per Batch: 1.7931, Accuracy: 532/2947 (18%)
Train Epoch: 40 Loss per Batch: 1.784810
Test set: Loss per Batch: 1.7928, Accuracy: 537/2947 (18%)
Train Epoch: 41 Loss per Batch: 1.784549
Test set: Loss per Batch: 1.7934, Accuracy: 532/2947 (18%)
Train Epoch: 42 Loss per Batch: 1.784458
Test set: Loss per Batch: 1.7920, Accuracy: 537/2947 (18%)
Train Epoch: 43 Loss per Batch: 1.785136
Test set: Loss per Batch: 1.7974, Accuracy: 537/2947 (18%)
Train Epoch: 44 Loss per Batch: 1.784797
Test set: Loss per Batch: 1.7762, Accuracy: 548/2947 (19%)
Train Epoch: 45 Loss per Batch: 1.785898
Test set: Loss per Batch: 1.7944, Accuracy: 532/2947 (18%)
Train Epoch: 46 Loss per Batch: 1.784899
Test set: Loss per Batch: 1.7951, Accuracy: 537/2947 (18%)
Train Epoch: 47 Loss per Batch: 1.784710
Test set: Loss per Batch: 1.7934, Accuracy: 537/2947 (18%)
Train Epoch: 48 Loss per Batch: 1.785024
Test set: Loss per Batch: 1.7946, Accuracy: 532/2947 (18%)
Train Epoch: 49 Loss per Batch: 1.784930
Test set: Loss per Batch: 1.7926, Accuracy: 537/2947 (18%)
Train Epoch: 50 Loss per Batch: 1.784564
Test set: Loss per Batch: 1.7913, Accuracy: 532/2947 (18%)
Train Epoch: 51 Loss per Batch: 1.784940
Test set: Loss per Batch: 1.7936, Accuracy: 532/2947 (18%)
Train Epoch: 52 Loss per Batch: 1.785000
Test set: Loss per Batch: 1.7921, Accuracy: 537/2947 (18%)
Train Epoch: 53 Loss per Batch: 1.784505
Test set: Loss per Batch: 1.7947, Accuracy: 532/2947 (18%)
Train Epoch: 54 Loss per Batch: 1.784886
Test set: Loss per Batch: 1.7914, Accuracy: 532/2947 (18%)
Train Epoch: 55 Loss per Batch: 1.784894
Test set: Loss per Batch: 1.7920, Accuracy: 537/2947 (18%)
Train Epoch: 56 Loss per Batch: 1.784917
Test set: Loss per Batch: 1.7920, Accuracy: 537/2947 (18%)
Train Epoch: 57 Loss per Batch: 1.784813
Test set: Loss per Batch: 1.7941, Accuracy: 537/2947 (18%)
Train Epoch: 58 Loss per Batch: 1.784905
Test set: Loss per Batch: 1.7933, Accuracy: 537/2947 (18%)
Train Epoch: 59 Loss per Batch: 1.784620
Test set: Loss per Batch: 1.7914, Accuracy: 537/2947 (18%)
Train Epoch: 60 Loss per Batch: 1.784722
Test set: Loss per Batch: 1.7934, Accuracy: 532/2947 (18%)
Train Epoch: 61 Loss per Batch: 1.784789
Test set: Loss per Batch: 1.7929, Accuracy: 537/2947 (18%)
Train Epoch: 62 Loss per Batch: 1.784626
Test set: Loss per Batch: 1.7933, Accuracy: 537/2947 (18%)
Train Epoch: 63 Loss per Batch: 1.784967
Test set: Loss per Batch: 1.7932, Accuracy: 537/2947 (18%)
Train Epoch: 64 Loss per Batch: 1.784695
Test set: Loss per Batch: 1.7918, Accuracy: 537/2947 (18%)
Train Epoch: 65 Loss per Batch: 1.784872
Test set: Loss per Batch: 1.7939, Accuracy: 537/2947 (18%)
Train Epoch: 66 Loss per Batch: 1.784757
Test set: Loss per Batch: 1.7925, Accuracy: 537/2947 (18%)
Train Epoch: 67 Loss per Batch: 1.784619
Test set: Loss per Batch: 1.7912, Accuracy: 537/2947 (18%)
Train Epoch: 68 Loss per Batch: 1.784919
Test set: Loss per Batch: 1.7927, Accuracy: 537/2947 (18%)
Train Epoch: 69 Loss per Batch: 1.784839
Test set: Loss per Batch: 1.7927, Accuracy: 537/2947 (18%)
Train Epoch: 70 Loss per Batch: 1.784806
Test set: Loss per Batch: 1.7937, Accuracy: 537/2947 (18%)
Train Epoch: 71 Loss per Batch: 1.784729
Test set: Loss per Batch: 1.7910, Accuracy: 537/2947 (18%)
Train Epoch: 72 Loss per Batch: 1.784705
Test set: Loss per Batch: 1.7937, Accuracy: 537/2947 (18%)
Train Epoch: 73 Loss per Batch: 1.785041
Test set: Loss per Batch: 1.7936, Accuracy: 537/2947 (18%)
Train Epoch: 74 Loss per Batch: 1.784568
Test set: Loss per Batch: 1.7919, Accuracy: 537/2947 (18%)
Train Epoch: 75 Loss per Batch: 1.784705
Test set: Loss per Batch: 1.7946, Accuracy: 537/2947 (18%)
Train Epoch: 76 Loss per Batch: 1.784397
Test set: Loss per Batch: 1.7896, Accuracy: 537/2947 (18%)
Train Epoch: 77 Loss per Batch: 1.785445
Test set: Loss per Batch: 1.7932, Accuracy: 532/2947 (18%)
Train Epoch: 78 Loss per Batch: 1.785322
Test set: Loss per Batch: 1.7935, Accuracy: 532/2947 (18%)
Train Epoch: 79 Loss per Batch: 1.784972
Test set: Loss per Batch: 1.7929, Accuracy: 537/2947 (18%)
Train Epoch: 80 Loss per Batch: 1.785531
Test set: Loss per Batch: 1.7935, Accuracy: 532/2947 (18%)
Train Epoch: 81 Loss per Batch: 1.784897
Test set: Loss per Batch: 1.7930, Accuracy: 537/2947 (18%)
Train Epoch: 82 Loss per Batch: 1.784814
Test set: Loss per Batch: 1.7935, Accuracy: 536/2947 (18%)
Train Epoch: 83 Loss per Batch: 1.784614
Test set: Loss per Batch: 1.7960, Accuracy: 532/2947 (18%)
Train Epoch: 84 Loss per Batch: 1.785118
Test set: Loss per Batch: 1.7930, Accuracy: 537/2947 (18%)
Train Epoch: 85 Loss per Batch: 1.785127
Test set: Loss per Batch: 1.7919, Accuracy: 532/2947 (18%)
Train Epoch: 86 Loss per Batch: 1.784900
Test set: Loss per Batch: 1.7934, Accuracy: 537/2947 (18%)
Train Epoch: 87 Loss per Batch: 1.784775
Test set: Loss per Batch: 1.7954, Accuracy: 537/2947 (18%)
Train Epoch: 88 Loss per Batch: 1.784831
Test set: Loss per Batch: 1.7931, Accuracy: 537/2947 (18%)
Train Epoch: 89 Loss per Batch: 1.784919
Test set: Loss per Batch: 1.7937, Accuracy: 537/2947 (18%)
Train Epoch: 90 Loss per Batch: 1.784544
Test set: Loss per Batch: 1.7935, Accuracy: 532/2947 (18%)
Train Epoch: 91 Loss per Batch: 1.784984
Test set: Loss per Batch: 1.7938, Accuracy: 537/2947 (18%)
Train Epoch: 92 Loss per Batch: 1.784681
Test set: Loss per Batch: 1.7919, Accuracy: 537/2947 (18%)
Train Epoch: 93 Loss per Batch: 1.784792
Test set: Loss per Batch: 1.7928, Accuracy: 537/2947 (18%)
Train Epoch: 94 Loss per Batch: 1.785022
Test set: Loss per Batch: 1.7918, Accuracy: 532/2947 (18%)
Train Epoch: 95 Loss per Batch: 1.785049
Test set: Loss per Batch: 1.7926, Accuracy: 537/2947 (18%)
Train Epoch: 96 Loss per Batch: 1.784717
Test set: Loss per Batch: 1.7946, Accuracy: 537/2947 (18%)
Train Epoch: 97 Loss per Batch: 1.784922
Test set: Loss per Batch: 1.7912, Accuracy: 537/2947 (18%)
Train Epoch: 98 Loss per Batch: 1.784669
Test set: Loss per Batch: 1.7917, Accuracy: 532/2947 (18%)
Train Epoch: 99 Loss per Batch: 1.784664
Test set: Loss per Batch: 1.7927, Accuracy: 537/2947 (18%)
Train Epoch: 100 Loss per Batch: 1.784869
Test set: Loss per Batch: 1.7937, Accuracy: 537/2947 (18%)
Train Epoch: 101 Loss per Batch: 1.784915
Test set: Loss per Batch: 1.7954, Accuracy: 537/2947 (18%)
Train Epoch: 102 Loss per Batch: 1.784592
Test set: Loss per Batch: 1.7913, Accuracy: 532/2947 (18%)
Train Epoch: 103 Loss per Batch: 1.784817
Test set: Loss per Batch: 1.7920, Accuracy: 537/2947 (18%)
Train Epoch: 104 Loss per Batch: 1.784869
Test set: Loss per Batch: 1.7941, Accuracy: 532/2947 (18%)
Train Epoch: 105 Loss per Batch: 1.787049
Test set: Loss per Batch: 1.7915, Accuracy: 537/2947 (18%)
Train Epoch: 106 Loss per Batch: 1.785012
Test set: Loss per Batch: 1.7919, Accuracy: 537/2947 (18%)
Train Epoch: 107 Loss per Batch: 1.784757
Test set: Loss per Batch: 1.7927, Accuracy: 532/2947 (18%)
Train Epoch: 108 Loss per Batch: 1.784881
Test set: Loss per Batch: 1.7939, Accuracy: 537/2947 (18%)
Train Epoch: 109 Loss per Batch: 1.785723
Test set: Loss per Batch: 1.7926, Accuracy: 537/2947 (18%)
Train Epoch: 110 Loss per Batch: 1.784988
Test set: Loss per Batch: 1.7914, Accuracy: 532/2947 (18%)
Train Epoch: 111 Loss per Batch: 1.785197
Test set: Loss per Batch: 1.7928, Accuracy: 532/2947 (18%)
Train Epoch: 112 Loss per Batch: 1.784856
Test set: Loss per Batch: 1.7937, Accuracy: 537/2947 (18%)
Train Epoch: 113 Loss per Batch: 1.785120
Test set: Loss per Batch: 1.7932, Accuracy: 537/2947 (18%)
Train Epoch: 114 Loss per Batch: 1.784842
Test set: Loss per Batch: 1.7906, Accuracy: 537/2947 (18%)
Train Epoch: 115 Loss per Batch: 1.785119
Test set: Loss per Batch: 1.7918, Accuracy: 537/2947 (18%)
Train Epoch: 116 Loss per Batch: 1.784935
Test set: Loss per Batch: 1.7917, Accuracy: 537/2947 (18%)
Train Epoch: 117 Loss per Batch: 1.784734
Test set: Loss per Batch: 1.7931, Accuracy: 537/2947 (18%)
Train Epoch: 118 Loss per Batch: 1.784930
Test set: Loss per Batch: 1.7916, Accuracy: 532/2947 (18%)
Train Epoch: 119 Loss per Batch: 1.785058
Test set: Loss per Batch: 1.7931, Accuracy: 537/2947 (18%)
Train Epoch: 120 Loss per Batch: 1.784399
Test set: Loss per Batch: 1.7955, Accuracy: 532/2947 (18%)
Train Epoch: 121 Loss per Batch: 1.784580
Test set: Loss per Batch: 1.7921, Accuracy: 537/2947 (18%)
Train Epoch: 122 Loss per Batch: 1.784730
Test set: Loss per Batch: 1.7926, Accuracy: 537/2947 (18%)
Train Epoch: 123 Loss per Batch: 1.784921
Test set: Loss per Batch: 1.7946, Accuracy: 537/2947 (18%)
Train Epoch: 124 Loss per Batch: 1.785217
Test set: Loss per Batch: 1.7918, Accuracy: 537/2947 (18%)
Train Epoch: 125 Loss per Batch: 1.784670
Test set: Loss per Batch: 1.7926, Accuracy: 537/2947 (18%)
Train Epoch: 126 Loss per Batch: 1.784825
Test set: Loss per Batch: 1.7935, Accuracy: 537/2947 (18%)
Train Epoch: 127 Loss per Batch: 1.785658
Test set: Loss per Batch: 1.7926, Accuracy: 532/2947 (18%)
Train Epoch: 128 Loss per Batch: 1.785025
Test set: Loss per Batch: 1.7924, Accuracy: 537/2947 (18%)
Train Epoch: 129 Loss per Batch: 1.784739
Test set: Loss per Batch: 1.7949, Accuracy: 532/2947 (18%)
Train Epoch: 130 Loss per Batch: 1.785050
Test set: Loss per Batch: 1.7932, Accuracy: 532/2947 (18%)
Train Epoch: 131 Loss per Batch: 1.784816
Test set: Loss per Batch: 1.7909, Accuracy: 537/2947 (18%)
Train Epoch: 132 Loss per Batch: 1.784678
Test set: Loss per Batch: 1.7943, Accuracy: 537/2947 (18%)
Train Epoch: 133 Loss per Batch: 1.784638
Test set: Loss per Batch: 1.7934, Accuracy: 532/2947 (18%)
Train Epoch: 134 Loss per Batch: 1.784962
Test set: Loss per Batch: 1.7941, Accuracy: 537/2947 (18%)
Train Epoch: 135 Loss per Batch: 1.784798
Test set: Loss per Batch: 1.7941, Accuracy: 537/2947 (18%)
Train Epoch: 136 Loss per Batch: 1.784776
Test set: Loss per Batch: 1.7932, Accuracy: 532/2947 (18%)
Train Epoch: 137 Loss per Batch: 1.785209
Test set: Loss per Batch: 1.7908, Accuracy: 537/2947 (18%)
Train Epoch: 138 Loss per Batch: 1.784884
Test set: Loss per Batch: 1.7926, Accuracy: 532/2947 (18%)
Train Epoch: 139 Loss per Batch: 1.784842
Test set: Loss per Batch: 1.7931, Accuracy: 532/2947 (18%)
Train Epoch: 140 Loss per Batch: 1.784622
Test set: Loss per Batch: 1.7919, Accuracy: 537/2947 (18%)
Train Epoch: 141 Loss per Batch: 1.785023
Test set: Loss per Batch: 1.7925, Accuracy: 537/2947 (18%)
Train Epoch: 142 Loss per Batch: 1.784689
Test set: Loss per Batch: 1.7944, Accuracy: 537/2947 (18%)
Train Epoch: 143 Loss per Batch: 1.784810
Test set: Loss per Batch: 1.7931, Accuracy: 537/2947 (18%)
Train Epoch: 144 Loss per Batch: 1.784905
Test set: Loss per Batch: 1.7920, Accuracy: 532/2947 (18%)
Train Epoch: 145 Loss per Batch: 1.784574
Test set: Loss per Batch: 1.7946, Accuracy: 537/2947 (18%)
Train Epoch: 146 Loss per Batch: 1.784829
Test set: Loss per Batch: 1.7930, Accuracy: 537/2947 (18%)
Train Epoch: 147 Loss per Batch: 1.784832
Test set: Loss per Batch: 1.7933, Accuracy: 537/2947 (18%)
Train Epoch: 148 Loss per Batch: 1.784582
Test set: Loss per Batch: 1.7941, Accuracy: 537/2947 (18%)
Train Epoch: 149 Loss per Batch: 1.784732
Test set: Loss per Batch: 1.7926, Accuracy: 537/2947 (18%)
Train Epoch: 150 Loss per Batch: 1.784652
Test set: Loss per Batch: 1.7947, Accuracy: 537/2947 (18%)
Train Epoch: 151 Loss per Batch: 1.784722
Test set: Loss per Batch: 1.7926, Accuracy: 537/2947 (18%)
Train Epoch: 152 Loss per Batch: 1.784994
Test set: Loss per Batch: 1.7951, Accuracy: 532/2947 (18%)
Train Epoch: 153 Loss per Batch: 1.784773
Test set: Loss per Batch: 1.7938, Accuracy: 532/2947 (18%)
Train Epoch: 154 Loss per Batch: 1.784962
Test set: Loss per Batch: 1.7934, Accuracy: 537/2947 (18%)
Train Epoch: 155 Loss per Batch: 1.784522
Test set: Loss per Batch: 1.7934, Accuracy: 537/2947 (18%)
Train Epoch: 156 Loss per Batch: 1.784805
Test set: Loss per Batch: 1.7921, Accuracy: 537/2947 (18%)
Train Epoch: 157 Loss per Batch: 1.785455
Test set: Loss per Batch: 1.7944, Accuracy: 532/2947 (18%)
Train Epoch: 158 Loss per Batch: 1.784509
Test set: Loss per Batch: 1.7943, Accuracy: 537/2947 (18%)
Train Epoch: 159 Loss per Batch: 1.784982
Test set: Loss per Batch: 1.7926, Accuracy: 537/2947 (18%)
Train Epoch: 160 Loss per Batch: 1.784593
Test set: Loss per Batch: 1.7938, Accuracy: 537/2947 (18%)
Train Epoch: 161 Loss per Batch: 1.785101
Test set: Loss per Batch: 1.7930, Accuracy: 537/2947 (18%)
Train Epoch: 162 Loss per Batch: 1.785277
Test set: Loss per Batch: 1.7939, Accuracy: 537/2947 (18%)
Train Epoch: 163 Loss per Batch: 1.784970
Test set: Loss per Batch: 1.7935, Accuracy: 532/2947 (18%)
Train Epoch: 164 Loss per Batch: 1.784984
Test set: Loss per Batch: 1.7943, Accuracy: 537/2947 (18%)
Train Epoch: 165 Loss per Batch: 1.784928
Test set: Loss per Batch: 1.7931, Accuracy: 537/2947 (18%)
Train Epoch: 166 Loss per Batch: 1.784774
Test set: Loss per Batch: 1.7923, Accuracy: 537/2947 (18%)
Train Epoch: 167 Loss per Batch: 1.785107
Test set: Loss per Batch: 1.7935, Accuracy: 537/2947 (18%)
Train Epoch: 168 Loss per Batch: 1.784620
Test set: Loss per Batch: 1.7941, Accuracy: 537/2947 (18%)
Train Epoch: 169 Loss per Batch: 1.784884
Test set: Loss per Batch: 1.7931, Accuracy: 537/2947 (18%)
Train Epoch: 170 Loss per Batch: 1.784452
Test set: Loss per Batch: 1.7923, Accuracy: 537/2947 (18%)
Train Epoch: 171 Loss per Batch: 1.784965
Test set: Loss per Batch: 1.7935, Accuracy: 537/2947 (18%)
Train Epoch: 172 Loss per Batch: 1.784781
Test set: Loss per Batch: 1.7924, Accuracy: 537/2947 (18%)
Train Epoch: 173 Loss per Batch: 1.784724
Test set: Loss per Batch: 1.7941, Accuracy: 537/2947 (18%)
Train Epoch: 174 Loss per Batch: 1.784639
Test set: Loss per Batch: 1.7912, Accuracy: 532/2947 (18%)
Train Epoch: 175 Loss per Batch: 1.784592
Test set: Loss per Batch: 1.7936, Accuracy: 537/2947 (18%)
Train Epoch: 176 Loss per Batch: 1.784642
Test set: Loss per Batch: 1.7922, Accuracy: 532/2947 (18%)
Train Epoch: 177 Loss per Batch: 1.785098
Test set: Loss per Batch: 1.7931, Accuracy: 537/2947 (18%)
Train Epoch: 178 Loss per Batch: 1.784758
Test set: Loss per Batch: 1.7938, Accuracy: 537/2947 (18%)
Train Epoch: 179 Loss per Batch: 1.784484
Test set: Loss per Batch: 1.7910, Accuracy: 532/2947 (18%)
Train Epoch: 180 Loss per Batch: 1.784845
Test set: Loss per Batch: 1.7949, Accuracy: 537/2947 (18%)
Train Epoch: 181 Loss per Batch: 1.784970
Test set: Loss per Batch: 1.7926, Accuracy: 532/2947 (18%)
Train Epoch: 182 Loss per Batch: 1.785243
Test set: Loss per Batch: 1.7917, Accuracy: 532/2947 (18%)
Train Epoch: 183 Loss per Batch: 1.784563
Test set: Loss per Batch: 1.7932, Accuracy: 537/2947 (18%)
Train Epoch: 184 Loss per Batch: 1.784880
Test set: Loss per Batch: 1.7936, Accuracy: 537/2947 (18%)
Train Epoch: 185 Loss per Batch: 1.784850
Test set: Loss per Batch: 1.7961, Accuracy: 537/2947 (18%)
Train Epoch: 186 Loss per Batch: 1.784834
Test set: Loss per Batch: 1.7921, Accuracy: 532/2947 (18%)
Train Epoch: 187 Loss per Batch: 1.784880
Test set: Loss per Batch: 1.7917, Accuracy: 532/2947 (18%)
Train Epoch: 188 Loss per Batch: 1.784855
Test set: Loss per Batch: 1.7919, Accuracy: 532/2947 (18%)
edit1: Теперь я попытался проверить, обновляются ли параметры вообще, как это предложил Ахмад.
x1=net.model.linear2.weight.data # weights at 150 epoch
x3=net.model.linear2.weight.data # weights at 152 epoch
x1-x3
tensor([[0., 0., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0., 0.]], device='cuda:0')```
So,now i tried to check grad at 152 epoch
net.model.linear2.weight.grad
tensor([[-5.2324e-14, -1.8329e-14, -4.5901e-15, 3.9225e-14, -4.8923e-14,
-3.7663e-14, -6.6461e-14, -1.3109e-12, -1.4465e-12],
[ 3.0959e-14, 2.7681e-14, -1.2809e-14, -2.1927e-14, 4.5212e-14,
3.6410e-14, -1.3647e-13, -1.2680e-12, -1.4908e-12],
[-7.7439e-15, -1.8330e-14, 1.2772e-14, 4.4520e-15, -2.0769e-14,
-1.6550e-14, 1.3497e-13, 1.6057e-12, 1.8513e-12],
[-7.0256e-15, -3.3731e-14, 2.6417e-14, 1.2869e-14, -4.3155e-14,
-3.2898e-14, 3.1167e-13, 3.5165e-12, 4.0472e-12],
[ 4.1122e-14, 1.7677e-14, 4.3402e-16, -2.3387e-14, 3.5274e-14,
2.8663e-14, 3.6638e-14, 6.8818e-13, 7.4608e-13],
[-7.3279e-14, -3.9648e-14, 6.5658e-15, 4.4829e-14, -7.3947e-14,
-5.9679e-14, 2.8081e-14, -2.1435e-13, -1.6659e-13],
[ 3.9702e-15, 1.8115e-14, -1.4382e-14, -1.4078e-15, 1.8547e-14,
1.5202e-14, -1.5490e-13, -1.8519e-12, -2.1300e-12],
[ 1.9415e-14, 1.8774e-14, -8.1929e-15, -1.9029e-14, 3.2933e-14,
2.4555e-14, -9.4528e-14, -8.9978e-13, -1.0555e-12],
[-9.5713e-15, -3.9856e-14, 3.1216e-14, 1.8188e-14, -5.4189e-14,
-4.1472e-14, 3.7540e-13, 4.1245e-12, 4.7497e-12],
[ 6.3520e-15, -6.6954e-15, 7.5276e-15, -1.1499e-16, -5.6589e-15,
-3.1609e-15, 9.2859e-14, 1.1613e-12, 1.3267e-12],
[ 2.0287e-14, 1.1300e-14, -2.4096e-15, -1.6815e-14, 2.5231e-14,
1.9853e-14, -2.9268e-14, -4.7256e-14, -7.8026e-14],
[-7.6004e-14, -4.6867e-14, 1.1355e-14, 5.1983e-14, -8.6401e-14,
-6.8230e-14, 9.0062e-14, 4.3427e-13, 5.8239e-13],
[ 3.0111e-14, 3.5032e-14, -1.8398e-14, -2.6007e-14, 5.4304e-14,
4.1803e-14, -2.0308e-13, -2.1058e-12, -2.4519e12]],device='cuda:0')
Now i know that this is the case of vanishing gradient.
How should i proceed to remedy vanishing gradients for this architecture??