Проблема с функцией Prediction, использующей состояния LSTM в Autoencoder - PullRequest
Новая
       43

Проблема с функцией Prediction, использующей состояния LSTM в Autoencoder

0 голосов
/ 03 апреля 2019

Попытка поместить слой встраивания в модель автокодера keras на основе LSTM (https://machinelearningmastery.com/develop-encoder-decoder-model-sequence-sequence-prediction-keras/), где слой встраивания ожидает двухмерный ввод для ввода декодера и трехмерного вывода.

Проблема с размерами входного сигнала декодера в функции прогнозирования.

Ошибка: ошибка при проверке ввода: ожидалось, что input_4 будет иметь 2 измерения, но получил массив с формой (1, 5, 5)

Здесь 5 - исходное входное измерение, которое сжимается до фиксированного размера вектора 2 с помощью встраивания. 

source= np.array([1, 2,3])
encoder=source[None,:]
"Source is:" [[1 2 3]] ## 2D input

decoder_input=source[:-1]
shifted_target=np.concatenate(([0], decoder_input))
shifted_target_input=shifted_target[None,:]
"Shifted Target is:" [[0 1 2]] ##2D Decoder Input

target_output=encoder
target_output=to_categorical(encoder,num_classes=5)
Target is: [[[0. 1. 0. 0. 0.]  ##3D Decoder Output, cardinality=5
  [0. 0. 1. 0. 0.]
  [0. 0. 0. 1. 0.]]]

#########################################
LSTM Embedding Code:
def define_models(embedding,vocab_size,n_units):
    Embedding_Layer = Embedding(output_dim=embedding, input_dim=vocab_size, input_length=None, name="Embedding")

    encoder_inputs = Input(shape=(None,), name="Encoder_input")
    encoder =LSTM(n_units, return_state=True, name='Encoder_lstm') 

    embedding_encode = Embedding_Layer(encoder_inputs) 
    print(embedding_encode)

    encoder_outputs, state_h, state_c = encoder(embedding_encode) 
    encoder_states = [state_h, state_c] 


    decoder_inputs = Input(shape=(None,), name="Decoder_input")
    decoder_lstm =LSTM(n_units, return_sequences=True, return_state=True, name="Decoder_lstm")

    embedding_decode = Embedding_Layer(decoder_inputs) 

    decoder_outputs, _, _ = decoder_lstm(embedding_decode, initial_state=encoder_states) 

    decoder_dense = (Dense(vocab_size, activation='softmax', name="Dense_layer"))
    decoder_outputs = decoder_dense(decoder_outputs) 

    model = Model([encoder_inputs, decoder_inputs], decoder_outputs)

##Inference model used for prediction

    encoder_model = Model(encoder_inputs, encoder_states) 

    decoder_state_input_h = Input(shape=(n_units,), name="H_state_input") 
    decoder_state_input_c = Input(shape=(n_units,), name="C_state_input") 
    decoder_states_inputs = [decoder_state_input_h, decoder_state_input_c] 
    decoder_outputs, state_h, state_c = decoder_lstm(embedding_decode, initial_state=decoder_states_inputs) 
    decoder_states = [state_h, state_c] 
    decoder_outputs = decoder_dense(decoder_outputs)

    decoder_model = Model([decoder_inputs] + decoder_states_inputs, [decoder_outputs] + decoder_states)
    return model, encoder_model, decoder_model

where Encoder input is 2D, Decoder input is 2D and Decoder output is 3D
# define model
train, infenc, infdec = define_models(2,5,100)
train.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['acc'])
# train model
train.fit([encoder,shifted_target_input], target_output, epochs=2)

#####################################################
##Prediction function 
(https://machinelearningmastery.com/develop-encoder-decoder-model-sequence-sequence-prediction-keras/):

def predict_sequence(infenc, infdec, source, n_steps, cardinality):
    # encode
    state = infenc.predict(source)
    # start of sequence input
    target_seq = array([0.0 for _ in range(cardinality)]).reshape(1, 1, cardinality)
    # collect predictions
    output = list()
    for t in range(n_steps):
        # predict next char
        yhat, h, c = infdec.predict([target_seq] + state)
        # store prediction
        output.append(yhat[0,0,:])
        # update state
        state = [h, c]
        # update target sequence
        target_seq = yhat
    return array(output)
...