Я хочу обучить автоэнкодер LSTM. У меня есть N наблюдений с двумя функциями, одна из которых является непрерывной, а другая - двоичной: я хочу прогнозировать непрерывные по линейной + MAE-потере и двоичные по сигмоидальной + BCE-потере. На этапе предварительной обработки я создал последовательности в виде скользящего окна из трех наблюдений. Вот как выглядит модель
input_ae = Input(shape=(X_train.shape[1], X_train.shape[2]))
print(input_ae.shape)
LSTM1 = keras.layers.LSTM(units=6, return_sequences=True, activation = 'selu')(input_ae)
LSTM2 = keras.layers.LSTM(units=1, activation= 'tanh')(LSTM1)
bottleneck = keras.layers.RepeatVector(n=X_train.shape[1])(LSTM2) # bottleneck layer
LSTM3 = keras.layers.LSTM(units=1, return_sequences=True)(bottleneck)
LSTM4 = keras.layers.LSTM(units=6, return_sequences=True, activation = 'selu')(LSTM3)
out1 = keras.layers.TimeDistributed(
keras.layers.Dense(units=1, activation = 'linear')
)(LSTM4)
out2 = keras.layers.TimeDistributed(
keras.layers.Dense(units=1, activation = 'sigmoid')
)(LSTM4)
model = Model(input_ae, outputs=[out1, out2])
model.compile(optimizer='adam', loss=["mae","binary_crossentropy"],
loss_weights=[1.0, 1.0],) # this is trainable
history = model.fit(X_train, [X_train[:,:,0,np.newaxis], X_train[:,:,1,np.newaxis]],
epochs=50,
batch_size=3,
shuffle=False) # observations are sorted by date !
Однако я все еще получаю следующие ошибки, касающиеся топологии графика моей модели. Я новичок в python, и теперь я не знаю, неверно ли мое кодирование или есть какая-то концептуальная ошибка. Прогнозируемые значения переменной 2 (двоичные) по-прежнему составляют около 0,5 (от 0,4 до 0,6), и они не смещаются в сторону 0 или 1.
2020-05-25 13:44:33.862501: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX AVX2
2020-05-25 13:44:33.882555: I tensorflow/core/common_runtime/process_util.cc:147] Creating new thread pool with default inter op setting: 2. Tune using inter_op_parallelism_threads for best performance.
Epoch 1/50
2020-05-25 13:44:39.191662: E tensorflow/core/grappler/optimizers/meta_optimizer.cc:561] remapper failed: Invalid argument: MutableGraphView::MutableGraphView error: node 'loss/time_distributed_2_loss/binary_crossentropy/weighted_loss/concat' has self cycle fanin 'loss/time_distributed_2_loss/binary_crossentropy/weighted_loss/concat'.
2020-05-25 13:44:39.229395: E tensorflow/core/grappler/optimizers/dependency_optimizer.cc:717] Iteration = 0, topological sort failed with message: The graph couldn't be sorted in topological order.
2020-05-25 13:44:39.247559: E tensorflow/core/grappler/optimizers/dependency_optimizer.cc:717] Iteration = 1, topological sort failed with message: The graph couldn't be sorted in topological order.
2020-05-25 13:44:39.345620: E tensorflow/core/grappler/optimizers/meta_optimizer.cc:561] arithmetic_optimizer failed: Invalid argument: The graph couldn't be sorted in topological order.
2020-05-25 13:44:39.363350: E tensorflow/core/grappler/optimizers/meta_optimizer.cc:561] remapper failed: Invalid argument: MutableGraphView::MutableGraphView error: node 'loss/time_distributed_2_loss/binary_crossentropy/weighted_loss/concat' has self cycle fanin 'loss/time_distributed_2_loss/binary_crossentropy/weighted_loss/concat'.
2020-05-25 13:44:39.383818: E tensorflow/core/grappler/optimizers/dependency_optimizer.cc:717] Iteration = 0, topological sort failed with message: The graph couldn't be sorted in topological order.
2020-05-25 13:44:39.399577: E tensorflow/core/grappler/optimizers/dependency_optimizer.cc:717] Iteration = 1, topological sort failed with message: The graph couldn't be sorted in topological order.
2020-05-25 13:44:39.432358: W tensorflow/core/common_runtime/process_function_library_runtime.cc:697] Ignoring multi-device function optimization failure: Invalid argument: The graph couldn't be sorted in topological order.
36/36 [==============================] - 3s 82ms/step - loss: 0.9305 - time_distributed_1_loss: 0.2378 - time_distributed_2_loss: 0.6927
Epoch 2/50
36/36 [==============================] - 0s 2ms/step - loss: 0.8997 - time_distributed_1_loss: 0.2085 - time_distributed_2_loss: 0.6912
Ошибка каждый раз только в первую эпоху, а позже - нет.