Намерение:
Я пытаюсь реализовать модель LSTM, которая предсказывает изменение ограничивающего прямоугольника при заданном действии.
У меня есть следующие входные данные:
- Действие
- точка ограничительной рамки 0
- точка ограничительной рамки 1
- точка ограничительной рамки 2
- точка ограничительной рамки 3
- идентификатор класса
- оценка
У меня есть набор данных, который имеет 5 рядов эпизодов (каждый эпизод занимает 5 строк набора данных), и 1-е действие эпизодов всегда равно 0 (начало action) и действие для прогнозирования - 5-е.
Набор данных:
act b0 b1 b2 b3 id score
0 85 238 129 256 69 0.9289865493774414
1 -1 -1 -1 -1 -1 -1
1 -1 -1 -1 -1 -1 -1
2 -1 -1 -1 -1 -1 -1
3 -1 -1 -1 -1 -1 -1 //row to to predict//
0 46 136 256 245 73 0.9369892477989197
1 18 18 256 252 73 0.8203921318054199
1 144 212 169 223 10 0.9630357623100281
1 13 9 252 199 73 0.9374213814735413
3 -1 -1 -1 -1 -1 -1 //row to predict//
0 215 141 255 233 72 0.9941028952598572
2 199 116 243 183 74 0.8685483932495117
3 215 141 255 233 72 0.9941184520721436
1 189 78 215 95 76 0.8610376119613647
3 206 50 255 169 72 0.8224002122879028 //row to predict//
0 -1 -1 -1 -1 -1 -1
3 19 129 249 253 73 0.8635225892066956
2 -1 -1 -1 -1 -1 -1
2 0 78 13 91 10 0.9488454461097717
3 -1 -1 -1 -1 -1 -1 //row to predict//
0 206 123 255 189 62 0.9980332255363464
2 221 197 256 255 62 0.9782524704933167
2 -1 -1 -1 -1 -1 -1
2 -1 -1 -1 -1 -1 -1
1 -1 -1 -1 -1 -1 -1 //row to predict//
0 184 78 243 169 72 0.9953457713127136
2 191 139 254 246 72 0.9929528832435608
3 184 78 243 169 72 0.9953963160514832
3 197 1 254 91 72 0.9956125020980835
2 184 78 243 169 72 0.9953963160514832 //row to predict//
-1 - это отсутствие ограничительной рамки.
Код:
Код, который у меня есть в настоящее время, является результатом обучения на YouTube, посвященного тому, как иметь несколько входов для LSTM: руководство
Построение модели:
def buildModel(episode_length, actions_to_predict):
action = Input(shape=(episode_length, 1), name='action')
b0 = Input(shape=(episode_length, 1), name='b0')
b1 = Input(shape=(episode_length, 1), name='b1')
b2 = Input(shape=(episode_length, 1), name='b2')
b3 = Input(shape=(episode_length, 1), name='b3')
class_id = Input(shape=(episode_length, 1), name='class_id')
score = Input(shape=(episode_length, 1), name='score')
action_layers = LSTM(64, input_shape=(episode_length, 1), return_sequences=False)(action)
b0_layers = LSTM(64, input_shape=(episode_length, 1), return_sequences=False)(b0)
b1_layers = LSTM(64, input_shape=(episode_length, 1), return_sequences=False)(b1)
b2_layers = LSTM(64, input_shape=(episode_length, 1), return_sequences=False)(b2)
b3_layers = LSTM(64, input_shape=(episode_length, 1), return_sequences=False)(b3)
class_id_layers = LSTM(64, input_shape=class_id.shape, return_sequences=False)(class_id)
score_layers = LSTM(64, input_shape=(episode_length, 1), return_sequences=False)(score)
output = concatenate(
[
action_layers,
b0_layers,
b1_layers,
b2_layers,
b3_layers,
class_id_layers,
score_layers,
]
)
output = Dense(actions_to_predict, activation='linear', name='weighted_average_output')(output)
model = Model(
inputs=
[
action,
b0,
b1,
b2,
b3,
class_id,
score
],
outputs=
[
output,
output,
output,
output,
]
)
model.compile(optimizer='rmsprop', loss='mse', metrics=["acc"])
return model
Подготовка набора данных:
# Split the data 80% training and 20% test
per_80 = int(data.shape[0]/ACTIONS_IN_EPISODE/100*80*ACTIONS_IN_EPISODE)
train, test = data[:per_80].copy(), data[per_80:].copy()
def prepareData(dataset):
column_names = ['index', 'act', 'b0', 'b1', 'b2', 'b3', 'class_id', 'score']
x, y = pd.DataFrame(columns = column_names), pd.DataFrame(columns = column_names)
i_start = dataset.index.values[0]
# Divide the in training actions and the one to predict
for i in range(dataset.shape[0]):
if i%ACTIONS_IN_EPISODE-1 == 0:
y = y.append(dataset.loc[i+i_start])
else:
x = x.append(dataset.loc[i+i_start])
return x, y
x_training, y_training = prepareData(train)
x_test, y_test = prepareData(test)
Изменение данных:
actions_to_train = 4
actions_to_predict = 1
model = buildModel(actions_to_train, actions_to_predict)
action_train = np.array(x_training["act"])
b0_train = np.array(x_training["b0"])
b1_train = np.array(x_training["b1"])
b2_train = np.array(x_training["b2"])
b3_train = np.array(x_training["b3"])
class_id_train = np.array(x_training["class_id"])
score_train = np.array(x_training["score"])
action_train = np.reshape(action_train,(-1, actions_to_train, 1))
b0_train = np.reshape(b0_train,(-1, actions_to_train, 1))
b1_train = np.reshape(b1_train,(-1, actions_to_train, 1))
b2_train = np.reshape(b2_train,(-1, actions_to_train, 1))
b3_train = np.reshape(b3_train,(-1, actions_to_train, 1))
class_id_train = np.reshape(class_id_train,(-1, actions_to_train, 1))
score_train = np.reshape(score_train,(-1, actions_to_train, 1))
action_test = np.array(x_test["act"])
b0_test = np.array(x_test["b0"])
b1_test = np.array(x_test["b1"])
b2_test = np.array(x_test["b2"])
b3_test = np.array(x_test["b3"])
class_id_test = np.array(x_test["class_id"])
score_test = np.array(x_test["score"])
action_test = np.reshape(action_test,(-1, actions_to_train, 1))
b0_test = np.reshape(b0_test,(-1, actions_to_train, 1))
b1_test = np.reshape(b1_test,(-1, actions_to_train, 1))
b2_test = np.reshape(b2_test,(-1, actions_to_train, 1))
b3_test = np.reshape(b3_test,(-1, actions_to_train, 1))
class_id_test = np.reshape(class_id_test,(-1, actions_to_train, 1))
score_test = np.reshape(score_test,(-1, actions_to_train, 1))
action_train_y = np.array(y_training["act"])
b0_train_y= np.array(y_training["b0"])
b1_train_y= np.array(y_training["b1"])
b2_train_y= np.array(y_training["b2"])
b3_train_y= np.array(y_training["b3"])
class_id_train_y= np.array(y_training["class_id"])
score_train_y= np.array(y_training["score"])
action_train_y = np.reshape(action_train_y,(-1, actions_to_predict, 1))
b0_train_y = np.reshape(b0_train_y,(-1, actions_to_predict, 1))
b1_train_y = np.reshape(b1_train_y,(-1, actions_to_predict, 1))
b2_train_y = np.reshape(b2_train_y,(-1, actions_to_predict, 1))
b3_train_y = np.reshape(b3_train_y,(-1, actions_to_predict, 1))
class_id_train_y = np.reshape(class_id_train_y,(-1, actions_to_predict, 1))
score_train_y = np.reshape(score_train_y,(-1, actions_to_predict, 1))
action_test_y = np.array(y_test["act"])
b0_test_y = np.array(y_test["b0"])
b1_test_y = np.array(y_test["b1"])
b2_test_y = np.array(y_test["b2"])
b3_test_y = np.array(y_test["b3"])
class_id_test_y = np.array(y_test["class_id"])
score_test_y = np.array(y_test["score"])
action_test_y = np.reshape(action_test_y,(-1, actions_to_predict, 1))
b0_test_y = np.reshape(b0_test_y,(-1, actions_to_predict, 1))
b1_test_y = np.reshape(b1_test_y,(-1, actions_to_predict, 1))
b2_test_y = np.reshape(b2_test_y,(-1, actions_to_predict, 1))
b3_test_y = np.reshape(b3_test_y,(-1, actions_to_predict, 1))
class_id_test_y = np.reshape(class_id_test_y,(-1, actions_to_predict, 1))
score_test_y = np.reshape(score_test_y,(-1, actions_to_predict, 1))
Подгонка модели:
history = model.fit(
[
action_train,
b0_train,
b1_train,
b2_train,
b3_train,
class_id_train,
score_train,
],
[
b0_train_y,
b1_train_y,
b2_train_y,
b3_train_y,
],
validation_data=(
[
action_test,
b0_test,
b1_test,
b2_test,
b3_test,
class_id_test,
score_test,
],
[
b0_test_y,
b1_test_y,
b2_test_y,
b3_test_y,
]
),
epochs=100,
batch_size=3000,
verbose=1
)
Вывод:
Epoch 1/100
...
Epoch 100/100
800/800 [==============================] - 0s 157us/step -
loss: 42213.5078 -
weighted_average_output_loss: 12503.5498 -
weighted_average_output_acc: 0.0012 -
weighted_average_output_acc_1: 0.0012 -
weighted_average_output_acc_2: 0.0000e+00 -
weighted_average_output_acc_3: 0.0025 -
val_loss: 31966.8652 -
val_weighted_average_output_loss: 9189.0039 -
val_weighted_average_output_acc: 0.0050 -
val_weighted_average_output_acc_1: 0.0000e+00 -
val_weighted_average_output_acc_2: 0.0000e+00 -
val_weighted_average_output_acc_3: 0.0000e+00
Вопросы:
Как изменить форму Input s, чтобы получить трехмерное измерение? [решено]
Как вывести графики для проверки точности и потерь?
Правки:
Я обновил код, мне удалось изменить форму pandas Series, передавая значения до numpy array.