Использование набора данных Tensorflow приводит к сбою слоя Concatenate - PullRequest
Новая
       53

Использование набора данных Tensorflow приводит к сбою слоя Concatenate

0 голосов
/ 29 декабря 2018

У меня есть модель, которую я обучал на относительно небольших наборах данных, и она отлично работает.Но теперь мне нужно масштабировать и я хочу использовать API данных tenorflow.Я использую CSVDataset, к которому я применяю некоторые преобразования через карту перед подачей моей модели.Позже у меня есть слой Concatenate, который отлично работает с массивными массивами, но теперь не работает с моим набором данных.

Вот трассировка: 

Tensorflow version: 1.12.0
Keras version: 2.1.6-tf
Initializing training dataset iterator...
Generating model...
2018-12-29 16:26:04.654201: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2018-12-29 16:26:04.677006: W tensorflow/core/framework/allocator.cc:122] Allocation of 80000000 exceeds 10% of system memory.
2018-12-29 16:26:04.791312: W tensorflow/core/framework/allocator.cc:122] Allocation of 80000000 exceeds 10% of system memory.
2018-12-29 16:26:04.815792: W tensorflow/core/framework/allocator.cc:122] Allocation of 80000000 exceeds 10% of system memory.

 __________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
words_input (InputLayer)        (None, 20)           0                                            
__________________________________________________________________________________________________
casing_input (InputLayer)       (None, 20)           0                                            
__________________________________________________________________________________________________
embedding (Embedding)           (None, 20, 50)       20000000    words_input[0][0]                
__________________________________________________________________________________________________
embedding_1 (Embedding)         (None, 20, 9)        81          casing_input[0][0]               
__________________________________________________________________________________________________
concatenate (Concatenate)       (None, 20, 59)       0           embedding[0][0]                  
                                                                 embedding_1[0][0]                
__________________________________________________________________________________________________
bidirectional (Bidirectional)   [(None, 400), (None, 416000      concatenate[0][0]                
__________________________________________________________________________________________________
dense (Dense)                   (None, 100)          40100       bidirectional[0][0]              
__________________________________________________________________________________________________
dense_output (Dense)            (None, 1573)         158873      dense[0][0]                      
==================================================================================================
Total params: 20,615,054
Trainable params: 614,973
Non-trainable params: 20,000,081
__________________________________________________________________________________________________

Compiling model...
Training model...
Epoch 1/20
Traceback (most recent call last):
  File "/opt/program/test-pipe.py", line 176, in <module>
    model.fit(dataset, epochs=nb_epochs, steps_per_epoch=100)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/keras/engine/training.py", line 1639, in fit
    validation_steps=validation_steps)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/keras/engine/training_arrays.py", line 154, in fit_loop
    outs = f(ins)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/keras/backend.py", line 2986, in __call__
    run_metadata=self.run_metadata)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1439, in __call__
    run_metadata_ptr)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/errors_impl.py", line 528, in __exit__
    c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: ConcatOp : Dimensions of inputs should match: shape[0] = [1,20,1,50] vs. shape[1] = [1,20,1,9]
     [[{{node concatenate/concat}} = ConcatV2[N=2, T=DT_FLOAT, Tidx=DT_INT32, _device="/job:localhost/replica:0/task:0/device:CPU:0"](embedding/embedding_lookup, embedding_1/embedding_lookup, concatenate/concat/axis)]]

А вот упрощенная версия моего кода:

...
# CSV contains: text, label
dataset = tf.data.experimental.CsvDataset(['my.csv'], [tf.string, tf.int32])

def parser(text, label):
    words_indexes, cases_indexes, y_oh = tf.py_func(
        parse_one, [text, label], [tf.int32, tf.int32, tf.int32])
    words_indexes.set_shape((20))
    cases_indexes.set_shape((20))
    y_oh.set_shape((1573))
    return {"words_input": words_indexes, "casing_input": cases_indexes}, y_oh


dataset = dataset.map(parser).batch(1).repeat()

print("Generating model...")

words_input = Input(shape=(20, ), dtype='int32', name='words_input')
words = Embedding(
    input_dim=word_to_vec_map.shape[0], output_dim=word_to_vec_map.shape[1],
    weights=[word_to_vec_map], trainable=False)(words_input)

casing_input = Input(shape=(20, ), dtype='int32', name='casing_input')
casing = Embedding(
    output_dim=case_embeddings.shape[1], input_dim=case_embeddings.shape[0],
    weights=[case_embeddings], trainable=False)(casing_input)

output = Concatenate()([words, casing])

output = Bidirectional(
    LSTM(200, return_sequences=False, return_state=True, dropout=0.50, recurrent_dropout=0.25)
)(output)

output = Dense(100, activation='relu')(output[0])
output = Dense(1573, activation='softmax', name='dense_output')(output)

model = Model(inputs=[words_input, casing_input], outputs=[output])

model.summary()
model.compile(
    loss='categorical_crossentropy', optimizer='nadam', metrics=['accuracy'])

model.fit(dataset, epochs=20, steps_per_epoch=100)

Вы можете видеть, что размеры двух входных слоев одинаковы, за исключением последнего измерения.Таким образом, он должен слиться (и это происходит при загрузке моего CSV в массивы np).Это ошибка?Или я что-то пропустил.Большое спасибо :)

EDIT

В целях тестирования я просто изменил свой код, чтобы 2 слоя имели одинаковую форму ... Теперь я получаю эту ошибку: 

2018-12-29 16:59:14.775214: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at transpose_op.cc:157 : Invalid argument: transpose expects a vector of size 4. But input(1) is a vector of size 3
Traceback (most recent call last):
  File "/opt/program/test-pipe.py", line 176, in <module>
    model.fit(dataset, epochs=nb_epochs, steps_per_epoch=100)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/keras/engine/training.py", line 1639, in fit
    validation_steps=validation_steps)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/keras/engine/training_arrays.py", line 154, in fit_loop
    outs = f(ins)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/keras/backend.py", line 2986, in __call__
    run_metadata=self.run_metadata)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1439, in __call__
    run_metadata_ptr)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/errors_impl.py", line 528, in __exit__
    c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: transpose expects a vector of size 4. But input(1) is a vector of size 3
     [[{{node bidirectional/transpose}} = Transpose[T=DT_FLOAT, Tperm=DT_INT32, _class=["loc:@bidirectional/TensorArrayUnstack/TensorArrayScatter/TensorArrayScatterV3"], _device="/job:localhost/replica:0/task:0/device:CPU:0"](concatenate/concat, bidirectional/transpose/perm)]]

РЕДАКТИРОВАТЬ 2

Добавление измененного слоя исправило мою вторую проблему.Но 1 / Почему этот слой?Двунаправленный LSTM должен возвращать правильную форму (это было в моем предыдущем коде, не используя API набора данных) 2 / Появляется новая проблема.Этот кажется более убедительным, потому что, кажется, это происходит в пределах вычисления метрики точности.И я понятия не имею, где на этот раз «изменить» ... даже TFDBG не может найти «узел», где происходит ошибка.

Вот модель: 

    __________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
words_input (InputLayer)        (None, 20)           0                                            
__________________________________________________________________________________________________
casing_input (InputLayer)       (None, 20)           0                                            
__________________________________________________________________________________________________
words_embedding (Embedding)     (None, 20, 50)       20000000    words_input[0][0]                
__________________________________________________________________________________________________
case_embedding (Embedding)      (None, 20, 50)       2500        casing_input[0][0]               
__________________________________________________________________________________________________
concatenate (Concatenate)       (None, 20, 100)      0           words_embedding[0][0]            
                                                                 case_embedding[0][0]             
__________________________________________________________________________________________________
reshape (Reshape)               (None, 20, 100)      0           concatenate[0][0]                
__________________________________________________________________________________________________
bidirectional (Bidirectional)   [(None, 400), (None, 481600      reshape[0][0]                    
__________________________________________________________________________________________________
dense_output (Dense)            (None, 1573)         630773      bidirectional[0][0]                            
==================================================================================================
Total params: 21,114,873
Trainable params: 1,112,373
Non-trainable params: 20,002,500
__________________________________________________________________________________________________

ИВот таинственная ошибка: 

Epoch 1/20
Traceback (most recent call last):
  File "/opt/program/test-pipe.py", line 194, in <module>
    model.fit(dataset, epochs=nb_epochs, steps_per_epoch=10000)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/keras/engine/training.py", line 1639, in fit
    validation_steps=validation_steps)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/keras/engine/training_arrays.py", line 154, in fit_loop
    outs = f(ins)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/keras/backend.py", line 2986, in __call__
    run_metadata=self.run_metadata)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1439, in __call__
    run_metadata_ptr)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/errors_impl.py", line 528, in __exit__
    c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: Incompatible shapes: [32,1573] vs. [32]
     [[{{node metrics/acc/Equal}} = Equal[T=DT_INT64, _device="/job:localhost/replica:0/task:0/device:CPU:0"](metrics/acc/ArgMax, metrics/acc/ArgMax_1)]]
...