Я пытаюсь выучить LSTM впервые, в основном объединяя два слоя LSTM.Ниже приведена архитектура модели.
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_2 (InputLayer) (None, 3) 0
__________________________________________________________________________________________________
input_3 (InputLayer) (None, 23) 0
__________________________________________________________________________________________________
embedding_2 (Embedding) (None, 3, 300) 27713700 input_2[0][0]
__________________________________________________________________________________________________
embedding_3 (Embedding) (None, 23, 300) 240724500 input_3[0][0]
__________________________________________________________________________________________________
lstm_1 (LSTM) (None, 300) 721200 embedding_2[0][0]
__________________________________________________________________________________________________
lstm_2 (LSTM) (None, 300) 721200 embedding_3[0][0]
__________________________________________________________________________________________________
concatenate_1 (Concatenate) (None, 600) 0 lstm_1[0][0]
lstm_2[0][0]
__________________________________________________________________________________________________
dense_1 (Dense) (None, 1) 601 concatenate_1[0][0]
==================================================================================================
Total params: 269,881,201
Trainable params: 1,443,001
Non-trainable params: 268,438,200
__________________________________________________________________________________________________
Код для одного из уровней LSTM.
# Query LSTM
max_query_length = 3
length_query_vocab = 92379
EMBEDDING_DIM = 300
batch_size = 1024
queryLSTM = Input(shape=(max_query_length, ))
queryLSTMx = Embedding(length_query_vocab, EMBEDDING_DIM, weights=[query_embedding_matrix],trainable=False)(queryLSTM)
queryLSTMx = LSTM(EMBEDDING_DIM, batch_input_shape=(batch_size, max_query_length, EMBEDDING_DIM), input_shape=(max_query_length, EMBEDDING_DIM))(queryLSTMx)
Я получаю сообщение об ошибке ниже:
Train on 4240822 samples, validate on 471203 samples
Epoch 1/2
70880/4240822 [..............................] - ETA: 6:40:34 - loss: 0.3239 - acc: 0.9021
---------------------------------------------------------------------------
InvalidArgumentError Traceback (most recent call last)
<ipython-input-34-3a0bd454aae9> in <module>()
----> 1 model.fit([Xq_train, Xp_train], y, epochs=2, validation_split=0.1)
/usr/local/lib/python3.6/dist-packages/keras/engine/training.py in fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, **kwargs)
1037 initial_epoch=initial_epoch,
1038 steps_per_epoch=steps_per_epoch,
-> 1039 validation_steps=validation_steps)
1040
1041 def evaluate(self, x=None, y=None,
/usr/local/lib/python3.6/dist-packages/keras/engine/training_arrays.py in fit_loop(model, f, ins, out_labels, batch_size, epochs, verbose, callbacks, val_f, val_ins, shuffle, callback_metrics, initial_epoch, steps_per_epoch, validation_steps)
197 ins_batch[i] = ins_batch[i].toarray()
198
--> 199 outs = f(ins_batch)
200 outs = to_list(outs)
201 for l, o in zip(out_labels, outs):
/usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py in __call__(self, inputs)
2713 return self._legacy_call(inputs)
2714
-> 2715 return self._call(inputs)
2716 else:
2717 if py_any(is_tensor(x) for x in inputs):
/usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py in _call(self, inputs)
2673 fetched = self._callable_fn(*array_vals, run_metadata=self.run_metadata)
2674 else:
-> 2675 fetched = self._callable_fn(*array_vals)
2676 return fetched[:len(self.outputs)]
2677
/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py in __call__(self, *args, **kwargs)
1437 ret = tf_session.TF_SessionRunCallable(
1438 self._session._session, self._handle, args, status,
-> 1439 run_metadata_ptr)
1440 if run_metadata:
1441 proto_data = tf_session.TF_GetBuffer(run_metadata_ptr)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/errors_impl.py in __exit__(self, type_arg, value_arg, traceback_arg)
526 None, None,
527 compat.as_text(c_api.TF_Message(self.status.status)),
--> 528 c_api.TF_GetCode(self.status.status))
529 # Delete the underlying status object from memory otherwise it stays alive
530 # as there is a reference to status from this from the traceback due to
InvalidArgumentError: indices[10,0] = 92379 is not in [0, 92379)
[[{{node embedding_2/embedding_lookup}} = GatherV2[Taxis=DT_INT32, Tindices=DT_INT32, Tparams=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"](embedding_2/embeddings/read, embedding_2/Cast, embedding_2/embedding_lookup/axis)]]
IЯ твердо верю, что проблема в том, что 92379 не соответствует индексу.Но я не могу выяснить, в чем причина.
РЕДАКТИРОВАТЬ: я запускаю этот код в Google Colab TPU.С GPU вроде бы нормально работает.