Обычно слои keras имеют выходное измерение в форме (None, c, h, w) или (None, h, w, c), в зависимости от конфигурации channels_first или channels_last.
Я пытаюсь использовать пользовательские слои keras с двумявходы.Когда я распечатываю сводку модели, она не показывает измерение None.
Как мне запрограммировать свой пользовательский слой для включения этого измерения None?
Я думаю, что это может быть причинойпочему я получаю сообщение об ошибке
No data provided for "crfrnn". Need data for each key in: ['crfrnn']
crfrnn - это имя моего пользовательского слоя
Я попытался изменить вывод в методе call(), к сожалению, безрезультатно.
Я убедился, что входные данные сформированы и подготовлены правильно.Обратите внимание, что мой код тренировался нормально, прежде чем я попытался добавить этот слой поверх слоя plant_output (см. Резюме ниже)
Ниже приведен код для пользовательского слоя, он был принят с thisрепозиторий github :
class CrfRnnLayer(tf.keras.layers.Layer):
""" Implements the CRF-RNN layer described in:
Conditional Random Fields as Recurrent Neural Networks,
S. Zheng, S. Jayasumana, B. Romera-Paredes, V. Vineet, Z. Su, D. Du, C. Huang and P. Torr,
ICCV 2015
"""
def __init__(self, image_dims, num_classes,
theta_alpha, theta_beta, theta_gamma,
num_iterations, NCHW, **kwargs):
self.image_dims = image_dims
self.num_classes = num_classes
self.theta_alpha = theta_alpha
self.theta_beta = theta_beta
self.theta_gamma = theta_gamma
self.num_iterations = num_iterations
self.NCHW = NCHW
self.spatial_ker_weights = None
self.bilateral_ker_weights = None
self.compatibility_matrix = None
super(CrfRnnLayer, self).__init__(**kwargs)
def build(self, input_shape):
if not self.NCHW:
channel_axis = -1
if input_shape[channel_axis] is None:
raise ValueError('The channel dimension of the inputs '
'should be defined. Found `None`.')
input_dim = input_shape[0][channel_axis]
self.input_spec = [tf.keras.layers.InputSpec(shape=(None, input_shape[0][1], input_shape[0][2], input_shape[0][3])), tf.keras.layers.InputSpec(shape=(None, input_shape[1][1], input_shape[1][2], input_shape[1][3]))]
else:
channel_axis = 1
if input_shape[channel_axis] is None:
raise ValueError('The channel dimension of the inputs '
'should be defined. Found `None`.')
input_dim = input_shape[0][channel_axis]
self.input_spec = [tf.keras.layers.InputSpec(shape=(None, input_shape[0][1], input_shape[0][2], input_shape[0][3])), tf.keras.layers.InputSpec(shape=(None, input_shape[1][1], input_shape[1][2], input_shape[1][3]))]
# Weights of the spatial kernel
self.spatial_ker_weights = self.add_weight(name='spatial_ker_weights',
shape=(self.num_classes, self.num_classes),
initializer=_diagonal_initializer,
trainable=True)
# Weights of the bilateral kernel
self.bilateral_ker_weights = self.add_weight(name='bilateral_ker_weights',
shape=(self.num_classes, self.num_classes),
initializer=_diagonal_initializer,
trainable=True)
# Compatibility matrix
self.compatibility_matrix = self.add_weight(name='compatibility_matrix',
shape=(self.num_classes, self.num_classes),
initializer=_potts_model_initializer,
trainable=True)
super(CrfRnnLayer, self).build(input_shape)
def call(self, inputs):
unaries = tf.transpose(inputs[0][0, :, :, :], perm=(2, 0, 1))
rgb = tf.transpose(inputs[1][0, :, :, :], perm=(2, 0, 1))
#input is channels first
c, h, w = self.num_classes, self.image_dims[0], self.image_dims[1]
all_ones = np.ones((c, h, w), dtype=np.float32)
# Prepare filter normalization coefficients
spatial_norm_vals = custom_module.high_dim_filter(all_ones, rgb, bilateral=False,
theta_gamma=self.theta_gamma)
bilateral_norm_vals = custom_module.high_dim_filter(all_ones, rgb, bilateral=True,
theta_alpha=self.theta_alpha,
theta_beta=self.theta_beta)
q_values = unaries
for i in range(self.num_iterations):
softmax_out = tf.nn.softmax(q_values, 0)
# Spatial filtering
spatial_out = custom_module.high_dim_filter(softmax_out, rgb, bilateral=False,
theta_gamma=self.theta_gamma)
spatial_out = spatial_out / spatial_norm_vals
# Bilateral filtering
bilateral_out = custom_module.high_dim_filter(softmax_out, rgb, bilateral=True,
theta_alpha=self.theta_alpha,
theta_beta=self.theta_beta)
bilateral_out = bilateral_out / bilateral_norm_vals
# Weighting filter outputs
message_passing = (tf.matmul(self.spatial_ker_weights,
tf.reshape(spatial_out, (c, -1))) +
tf.matmul(self.bilateral_ker_weights,
tf.reshape(bilateral_out, (c, -1))))
# Compatibility transform
pairwise = tf.matmul(self.compatibility_matrix, message_passing)
# Adding unary potentials
pairwise = tf.reshape(pairwise, (c, h, w))
q_values = unaries - pairwise
#output is channels last
return tf.transpose(tf.reshape(q_values, (1, c, h, w)), perm=(0, 2, 3, 1))
def compute_output_shape(self, input_shape):
return input_shape
Этот слой имеет два входа, каждый с размером (None, 384, 512, 3).Я ожидаю, что результат будет таким же, но когда я скомпилирую модель, получится следующее резюме (обратите внимание, что я не показал много слоев в середине, потому что я нарушил ограничение на количество символов на этой платформе, наиболее важные слоиinput_node, plant_output и пользовательские crfrnn):
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_node (InputLayer) (None, 384, 512, 3) 0
__________________________________________________________________________________________________
encoder_conv1 (Conv2D) (None, 384, 512, 32) 128 input_node[0][0]
__________________________________________________________________________________________________
bneck1_dense_encoder1 (Conv2D) (None, 384, 512, 8) 264 encoder_conv1[0][0]
__________________________________________________________________________________________________
batch_normalization (BatchNorma (None, 384, 512, 8) 32 bneck1_dense_encoder1[0][0]
__________________________________________________________________________________________________
leaky_re_lu (LeakyReLU) (None, 384, 512, 8) 0 batch_normalization[0][0]
__________________________________________________________________________________________________
conv1_dense_encoder1 (Conv2D) (None, 384, 512, 4) 292 leaky_re_lu[0][0]
__________________________________________________________________________________________________
dropout (Dropout) (None, 384, 512, 4) 0 conv1_dense_encoder1[0][0]
__________________________________________________________________________________________________
concatenate (Concatenate) (None, 384, 512, 36) 0 encoder_conv1[0][0]
dropout[0][0]
__________________________________________________________________________________________________
bneck2_dense_encoder1 (Conv2D) (None, 384, 512, 8) 296 concatenate[0][0]
__________________________________________________________________________________________________
batch_normalization_1 (BatchNor (None, 384, 512, 8) 32 bneck2_dense_encoder1[0][0]
__________________________________________________________________________________________________
leaky_re_lu_1 (LeakyReLU) (None, 384, 512, 8) 0 batch_normalization_1[0][0]
__________________________________________________________________________________________________
conv2_dense_encoder1 (Conv2D) (None, 384, 512, 4) 292 leaky_re_lu_1[0][0]
__________________________________________________________________________________________________
dropout_1 (Dropout) (None, 384, 512, 4) 0 conv2_dense_encoder1[0][0]
__________________________________________________________________________________________________
concatenate_1 (Concatenate) (None, 384, 512, 40) 0 concatenate[0][0]
dropout_1[0][0]
__________________________________________________________________________________________________
bneck3_dense_encoder1 (Conv2D) (None, 384, 512, 8) 328 concatenate_1[0][0]
__________________________________________________________________________________________________
batch_normalization_2 (BatchNor (None, 384, 512, 8) 32 bneck3_dense_encoder1[0][0]
__________________________________________________________________________________________________
leaky_re_lu_2 (LeakyReLU) (None, 384, 512, 8) 0 batch_normalization_2[0][0]
__________________________________________________________________________________________________
conv3_dense_encoder1 (Conv2D) (None, 384, 512, 4) 292 leaky_re_lu_2[0][0]
__________________________________________________________________________________________________
dropout_2 (Dropout) (None, 384, 512, 4) 0 conv3_dense_encoder1[0][0]
__________________________________________________________________________________________________
concatenate_2 (Concatenate) (None, 384, 512, 12) 0 dropout[0][0]
dropout_1[0][0]
dropout_2[0][0]
__________________________________________________________________________________________________
encoder_concat1 (Concatenate) (None, 384, 512, 44) 0 encoder_conv1[0][0]
concatenate_2[0][0]
__________________________________________________________________________________________________
encoder_bneck1 (Conv2D) (None, 384, 512, 22) 990 encoder_concat1[0][0]
__________________________________________________________________________________________________
encoder_downsample1 (Conv2D) (None, 192, 256, 22) 12122 encoder_bneck1[0][0]
__________________________________________________________________________________________________
bneck1_dense_encoder2 (Conv2D) (None, 192, 256, 8) 184 encoder_downsample1[0][0]
__________________________________________________________________________________________________
batch_normalization_3 (BatchNor (None, 192, 256, 8) 32 bneck1_dense_encoder2[0][0]
__________________________________________________________________________________________________
leaky_re_lu_3 (LeakyReLU) (None, 192, 256, 8) 0 batch_normalization_3[0][0]
__________________________________________________________________________________________________
conv1_dense_encoder2 (Conv2D) (None, 192, 256, 4) 292 leaky_re_lu_3[0][0]
__________________________________________________________________________________________________
dropout_3 (Dropout) (None, 192, 256, 4) 0 conv1_dense_encoder2[0][0]
__________________________________________________________________________________________________
concatenate_3 (Concatenate) (None, 192, 256, 26) 0 encoder_downsample1[0][0]
dropout_3[0][0]
__________________________________________________________________________________________________
bneck2_dense_encoder2 (Conv2D) (None, 192, 256, 8) 216 concatenate_3[0][0]
__________________________________________________________________________________________________
batch_normalization_4 (BatchNor (None, 192, 256, 8) 32 bneck2_dense_encoder2[0][0]
__________________________________________________________________________________________________
leaky_re_lu_4 (LeakyReLU) (None, 192, 256, 8) 0 batch_normalization_4[0][0]
__________________________________________________________________________________________________
conv2_dense_encoder2 (Conv2D) (None, 192, 256, 4) 292 leaky_re_lu_4[0][0]
__________________________________________________________________________________________________
dropout_4 (Dropout) (None, 192, 256, 4) 0 conv2_dense_encoder2[0][0]
__________________________________________________________________________________________________
concatenate_4 (Concatenate) (None, 192, 256, 30) 0 concatenate_3[0][0]
dropout_4[0][0]
__________________________________________________________________________________________________
bneck3_dense_encoder2 (Conv2D) (None, 192, 256, 8) 248 concatenate_4[0][0]
__________________________________________________________________________________________________
batch_normalization_5 (BatchNor (None, 192, 256, 8) 32 bneck3_dense_encoder2[0][0]
__________________________________________________________________________________________________
leaky_re_lu_5 (LeakyReLU) (None, 192, 256, 8) 0 batch_normalization_5[0][0]
__________________________________________________________________________________________________
conv3_dense_encoder2 (Conv2D) (None, 192, 256, 4) 292 leaky_re_lu_5[0][0]
__________________________________________________________________________________________________
dropout_5 (Dropout) (None, 192, 256, 4) 0 conv3_dense_encoder2[0][0]
__________________________________________________________________________________________________
concatenate_5 (Concatenate) (None, 192, 256, 12) 0 dropout_3[0][0]
dropout_4[0][0]
dropout_5[0][0]
__________________________________________________________________________________________________
encoder_concat2 (Concatenate) (None, 192, 256, 34) 0 encoder_downsample1[0][0]
concatenate_5[0][0]
__________________________________________________________________________________________________
encoder_bneck2 (Conv2D) (None, 192, 256, 17) 595 encoder_concat2[0][0]
__________________________________________________________________________________________________
encoder_downsample2 (Conv2D) (None, 96, 128, 17) 7242 encoder_bneck2[0][0]
__________________________________________________________________________________________________
bneck1_dense_encoder3 (Conv2D) (None, 96, 128, 8) 144 encoder_downsample2[0][0]
__________________________________________________________________________________________________
batch_normalization_6 (BatchNor (None, 96, 128, 8) 32 bneck1_dense_encoder3[0][0]
__________________________________________________________________________________________________
leaky_re_lu_6 (LeakyReLU) (None, 96, 128, 8) 0 batch_normalization_6[0][0]
__________________________________________________________________________________________________
conv1_dense_encoder3 (Conv2D) (None, 96, 128, 4) 292 leaky_re_lu_6[0][0]
__________________________________________________________________________________________________
dropout_6 (Dropout) (None, 96, 128, 4) 0 conv1_dense_encoder3[0][0]
__________________________________________________________________________________________________
concatenate_6 (Concatenate) (None, 96, 128, 21) 0 encoder_downsample2[0][0]
dropout_6[0][0]
__________________________________________________________________________________________________
bneck2_dense_encoder3 (Conv2D) (None, 96, 128, 8) 176 concatenate_6[0][0]
__________________________________________________________________________________________________
batch_normalization_7 (BatchNor (None, 96, 128, 8) 32 bneck2_dense_encoder3[0][0]
__________________________________________________________________________________________________
leaky_re_lu_7 (LeakyReLU) (None, 96, 128, 8) 0 batch_normalization_7[0][0]
__________________________________________________________________________________________________
conv2_dense_encoder3 (Conv2D) (None, 96, 128, 4) 292 leaky_re_lu_7[0][0]
__________________________________________________________________________________________________
dropout_7 (Dropout) (None, 96, 128, 4) 0 conv2_dense_encoder3[0][0]
__________________________________________________________________________________________________
concatenate_7 (Concatenate) (None, 96, 128, 25) 0 concatenate_6[0][0]
dropout_7[0][0]
__________________________________________________________________________________________________
bneck3_dense_encoder3 (Conv2D) (None, 96, 128, 8) 208 concatenate_7[0][0]
__________________________________________________________________________________________________
batch_normalization_8 (BatchNor (None, 96, 128, 8) 32 bneck3_dense_encoder3[0][0]
__________________________________________________________________________________________________
leaky_re_lu_8 (LeakyReLU) (None, 96, 128, 8) 0 batch_normalization_8[0][0]
__________________________________________________________________________________________________
conv3_dense_encoder3 (Conv2D) (None, 96, 128, 4) 292 leaky_re_lu_8[0][0]
__________________________________________________________________________________________________
dropout_8 (Dropout) (None, 96, 128, 4) 0 conv3_dense_encoder3[0][0]
__________________________________________________________________________________________________
concatenate_8 (Concatenate) (None, 96, 128, 12) 0 dropout_6[0][0]
dropout_7[0][0]
dropout_8[0][0]
__________________________________________________________________________________________________
__________________________________________________________________________________________________
plant_conv (Conv2D) (None, 384, 512, 32) 416 concatenate_20[0][0]
__________________________________________________________________________________________________
plant_output (Conv2D) (None, 384, 512, 3) 99 plant_conv[0][0]
__________________________________________________________________________________________________
crfrnn (CrfRnnLayer) (1, 384, 512, 3) 27 plant_output[0][0]
input_node[0][0]
==================================================================================================
Total params: 41,833
Trainable params: 41,497
Non-trainable params: 336
__________________________________________________________________________________________________
Обратите внимание, как слой crfrnn имеет форму (1, 384, 512, 3).Я считаю, что это приводит к тому, что программа не работает, выдает ошибку:
Epoch 1/2000
No data provided for "crfrnn". Need data for each key in: ['crfrnn']