У меня есть CNN Conv-2 для набора данных CIFAR-10, вдохновленный CNN в стиле VGG-19, где у меня есть 2 сверточных слоя по 64 фильтра каждый, за которыми следует слой максимального объединения, за которым следуют 2 плотных слоя с 256 нейронами каждый, наконец за которым следует выходной слой с 10 нейронами.
Результирующая сеть, которую я получаю, является разреженной и имеет разреженность 86,3378%, поэтому количество оставшихся обучаемых параметров = 587701. Исходная сеть имеет 4301642 обучаемых параметра.
При обучении с использованием набора весов, полученных во время обрезки, журнал, который я получаю:
Epoch 1, Loss: 103.0380, Accuracy: 70.6800, Test Loss: 105.8176, Test Accuracy: 63.570000
Total number of trainable parameters = 587701
Epoch 2, Loss: 29.0848, Accuracy: 81.0300, Test Loss: 81.8922, Test Accuracy: 66.469994
Total number of trainable parameters = 587701
Epoch 3, Loss: 20.3189, Accuracy: 83.6880, Test Loss: 72.0549, Test Accuracy: 66.829994
Total number of trainable parameters = 587701
Epoch 4, Loss: 16.0905, Accuracy: 84.9020, Test Loss: 64.6104, Test Accuracy: 67.150002
Total number of trainable parameters = 587701
Epoch 5, Loss: 13.3211, Accuracy: 85.8980, Test Loss: 58.1929, Test Accuracy: 67.320000
Total number of trainable parameters = 587701
Epoch 6, Loss: 11.2026, Accuracy: 86.4420, Test Loss: 54.2645, Test Accuracy: 67.790001
Total number of trainable parameters = 587701
Epoch 7, Loss: 9.5162, Accuracy: 87.2980, Test Loss: 49.3573, Test Accuracy: 67.860001
Total number of trainable parameters = 587701
Epoch 8, Loss: 8.1179, Accuracy: 88.0380, Test Loss: 45.4765, Test Accuracy: 67.790001
Total number of trainable parameters = 587701
Epoch 9, Loss: 6.9544, Accuracy: 88.3760, Test Loss: 41.6912, Test Accuracy: 68.110001
Total number of trainable parameters = 587701
Epoch 10, Loss: 6.0337, Accuracy: 88.8740, Test Loss: 38.2918, Test Accuracy: 68.150002
Total number of trainable parameters = 587701
Epoch 11, Loss: 5.2442, Accuracy: 89.1940, Test Loss: 35.7297, Test Accuracy: 68.019997
Total number of trainable parameters = 587701
Epoch 12, Loss: 4.5318, Accuracy: 89.6760, Test Loss: 33.2810, Test Accuracy: 68.120003
Total number of trainable parameters = 587701
Epoch 13, Loss: 4.0094, Accuracy: 89.8400, Test Loss: 31.2073, Test Accuracy: 68.040001
Total number of trainable parameters = 587701
Epoch 14, Loss: 3.5019, Accuracy: 90.3280, Test Loss: 29.2610, Test Accuracy: 68.400002
Total number of trainable parameters = 587701
Epoch 15, Loss: 3.0685, Accuracy: 90.6940, Test Loss: 27.9433, Test Accuracy: 68.169998
Total number of trainable parameters = 587701
Epoch 16, Loss: 2.7589, Accuracy: 91.0460, Test Loss: 26.4150, Test Accuracy: 67.709999
Total number of trainable parameters = 587701
Epoch 17, Loss: 2.4893, Accuracy: 91.1740, Test Loss: 24.9556, Test Accuracy: 67.739998
Total number of trainable parameters = 587701
Epoch 18, Loss: 2.1595, Accuracy: 91.5040, Test Loss: 23.7613, Test Accuracy: 67.940002
Total number of trainable parameters = 587701
Epoch 19, Loss: 1.9053, Accuracy: 91.9860, Test Loss: 22.3948, Test Accuracy: 68.150002
Total number of trainable parameters = 587701
Epoch 20, Loss: 1.7348, Accuracy: 92.1840, Test Loss: 21.6371, Test Accuracy: 67.769997
Total number of trainable parameters = 587701
Epoch 21, Loss: 1.5420, Accuracy: 92.4800, Test Loss: 20.5312, Test Accuracy: 68.150002
Total number of trainable parameters = 587701
Epoch 22, Loss: 1.3645, Accuracy: 92.7820, Test Loss: 19.7076, Test Accuracy: 67.989998
Total number of trainable parameters = 587701
Epoch 23, Loss: 1.2485, Accuracy: 92.8820, Test Loss: 19.0369, Test Accuracy: 67.979996
Total number of trainable parameters = 587701
Epoch 24, Loss: 1.0946, Accuracy: 93.4040, Test Loss: 18.3569, Test Accuracy: 67.580002
Total number of trainable parameters = 587701
Epoch 25, Loss: 0.9884, Accuracy: 93.6820, Test Loss: 17.7242, Test Accuracy: 67.389999
Total number of trainable parameters = 587701
Epoch 26, Loss: 0.8946, Accuracy: 93.8420, Test Loss: 17.2129, Test Accuracy: 67.680000
Total number of trainable parameters = 587701
Epoch 27, Loss: 0.8196, Accuracy: 94.0660, Test Loss: 16.7172, Test Accuracy: 67.940002
Total number of trainable parameters = 587701
Epoch 28, Loss: 0.7288, Accuracy: 94.4160, Test Loss: 16.1018, Test Accuracy: 67.790001
Total number of trainable parameters = 587701
Epoch 29, Loss: 0.6455, Accuracy: 94.6660, Test Loss: 15.6097, Test Accuracy: 67.549995
Total number of trainable parameters = 587701
Epoch 30, Loss: 0.5945, Accuracy: 94.8360, Test Loss: 15.4291, Test Accuracy: 67.709999
Total number of trainable parameters = 587701
Epoch 31, Loss: 0.5325, Accuracy: 95.1600, Test Loss: 14.9845, Test Accuracy: 67.889999
Total number of trainable parameters = 587701
Epoch 32, Loss: 0.4755, Accuracy: 95.4400, Test Loss: 14.4926, Test Accuracy: 67.849998
Total number of trainable parameters = 587701
Epoch 33, Loss: 0.4244, Accuracy: 95.6340, Test Loss: 14.3344, Test Accuracy: 67.239998
Total number of trainable parameters = 587701
Epoch 34, Loss: 0.3877, Accuracy: 95.9100, Test Loss: 13.9322, Test Accuracy: 67.720001
Total number of trainable parameters = 587701
Epoch 35, Loss: 0.3378, Accuracy: 96.3580, Test Loss: 13.8225, Test Accuracy: 67.430000
Total number of trainable parameters = 587701
Epoch 36, Loss: 0.3133, Accuracy: 96.3340, Test Loss: 13.5336, Test Accuracy: 67.820000
Total number of trainable parameters = 587701
Epoch 37, Loss: 0.2928, Accuracy: 96.3520, Test Loss: 13.2876, Test Accuracy: 67.659996
Total number of trainable parameters = 587701
Epoch 38, Loss: 0.2655, Accuracy: 96.6880, Test Loss: 13.0513, Test Accuracy: 67.639999
Total number of trainable parameters = 587701
Epoch 39, Loss: 0.2318, Accuracy: 96.8980, Test Loss: 12.8556, Test Accuracy: 67.639999
Total number of trainable parameters = 587701
Epoch 40, Loss: 0.2115, Accuracy: 97.0360, Test Loss: 12.7627, Test Accuracy: 67.669998
Total number of trainable parameters = 587701
Epoch 41, Loss: 0.1935, Accuracy: 97.2400, Test Loss: 12.7046, Test Accuracy: 67.659996
Total number of trainable parameters = 587701
Epoch 42, Loss: 0.1754, Accuracy: 97.4040, Test Loss: 12.5659, Test Accuracy: 67.389999
Total number of trainable parameters = 587701
Epoch 43, Loss: 0.1683, Accuracy: 97.2800, Test Loss: 12.3408, Test Accuracy: 67.459999
Total number of trainable parameters = 587701
Epoch 44, Loss: 0.1404, Accuracy: 97.7360, Test Loss: 12.2870, Test Accuracy: 67.709999
Total number of trainable parameters = 587701
Epoch 45, Loss: 0.1405, Accuracy: 97.6180, Test Loss: 12.1623, Test Accuracy: 67.180000
Total number of trainable parameters = 587701
Epoch 46, Loss: 0.1167, Accuracy: 97.9140, Test Loss: 12.3521, Test Accuracy: 67.019997
Total number of trainable parameters = 587701
Epoch 47, Loss: 0.1141, Accuracy: 97.8640, Test Loss: 12.0325, Test Accuracy: 67.559998
Total number of trainable parameters = 587701
Epoch 48, Loss: 0.1055, Accuracy: 98.0440, Test Loss: 11.8428, Test Accuracy: 67.459999
Total number of trainable parameters = 587701
Epoch 49, Loss: 0.0820, Accuracy: 98.4340, Test Loss: 12.0961, Test Accuracy: 67.379997
Total number of trainable parameters = 587701
Epoch 50, Loss: 0.0896, Accuracy: 98.2280, Test Loss: 11.8406, Test Accuracy: 67.720001
Total number of trainable parameters = 587701
Epoch 51, Loss: 0.0743, Accuracy: 98.4160, Test Loss: 12.1112, Test Accuracy: 66.649994
Total number of trainable parameters = 587701
Epoch 52, Loss: 0.0717, Accuracy: 98.4260, Test Loss: 11.7410, Test Accuracy: 67.459999
Total number of trainable parameters = 587701
Epoch 53, Loss: 0.0672, Accuracy: 98.5360, Test Loss: 12.0165, Test Accuracy: 67.349998
Total number of trainable parameters = 587701
Epoch 54, Loss: 0.0631, Accuracy: 98.5660, Test Loss: 11.8877, Test Accuracy: 67.320000
Total number of trainable parameters = 587701
Epoch 55, Loss: 0.0597, Accuracy: 98.6900, Test Loss: 11.6518, Test Accuracy: 67.470001
Total number of trainable parameters = 587701
Epoch 56, Loss: 0.0573, Accuracy: 98.6900, Test Loss: 11.6304, Test Accuracy: 67.430000
Total number of trainable parameters = 587701
Epoch 57, Loss: 0.0519, Accuracy: 98.7800, Test Loss: 11.6220, Test Accuracy: 67.709999
Total number of trainable parameters = 587701
Epoch 58, Loss: 0.0378, Accuracy: 99.0560, Test Loss: 11.5965, Test Accuracy: 67.510002
Total number of trainable parameters = 587701
Epoch 59, Loss: 0.0469, Accuracy: 98.8880, Test Loss: 11.7024, Test Accuracy: 67.449997
Total number of trainable parameters = 587701
Epoch 60, Loss: 0.0362, Accuracy: 99.0420, Test Loss: 11.4755, Test Accuracy: 67.500000
Total number of trainable parameters = 587701
Epoch 61, Loss: 0.0434, Accuracy: 98.9360, Test Loss: 11.4118, Test Accuracy: 67.760002
Total number of trainable parameters = 587701
Epoch 62, Loss: 0.0312, Accuracy: 99.1920, Test Loss: 11.5527, Test Accuracy: 67.430000
Total number of trainable parameters = 587701
Epoch 63, Loss: 0.0391, Accuracy: 98.9420, Test Loss: 11.6315, Test Accuracy: 67.099998
Total number of trainable parameters = 587701
Epoch 64, Loss: 0.0430, Accuracy: 98.8760, Test Loss: 11.5841, Test Accuracy: 67.489998
Total number of trainable parameters = 587701
Я не понимаю, как модель имеет такие чрезвычайно высокие 'Test Loss' и для сравнения: приличная 'Точность теста'?
Кроме того, из-за столь высоких потерь в тестах модель пытается оптимизировать, уменьшая «Потери теста», в то время как «Точность теста» не Не менее go каких-либо улучшений.
Любое объяснение такого поведения или того, что происходит с моделью?