Keras - ValueError: Выход генератора должен быть кортежем `(x, y, sample_weight)` или `(x, y)`. Найдено: [[[[0.54901963] [0.5372549] - PullRequest
Новая
       5

Keras - ValueError: Выход генератора должен быть кортежем `(x, y, sample_weight)` или `(x, y)`. Найдено: [[[[0.54901963] [0.5372549]

0 голосов
/ 14 марта 2020

Поэтому я пытаюсь обучить автокодер, используя ImageDataGenerator, используя этот скрипт: 

from keras.preprocessing.image import ImageDataGenerator

batch_size = 128

train_datagen = ImageDataGenerator(rescale=1./255, validation_split = 0.2)

training_generator = train_datagen.flow_from_directory(train_dir,
                                                     target_size=(105, 105),
                                                     color_mode='grayscale',
                                                     batch_size = batch_size,
                                                     class_mode=None,
                                                     subset='training')
validation_generator = train_datagen.flow_from_directory(train_dir,
                                                     target_size=(105, 105),
                                                     color_mode='grayscale',
                                                     batch_size = batch_size,
                                                     class_mode=None,
                                                     subset='validation')

history = autoencoder.fit_generator(generator=training_generator, 
                                    epochs=5, 
                                    steps_per_epoch=int(training_generator.samples // batch_size),
                                    validation_data=validation_generator,
                                    validation_steps = int(validation_generator.samples // batch_size),
                                    use_multiprocessing=False)

Мой каталог данных выглядит так: 

real_train_500_2
    |----> train
             |-----> images

И вот краткая информация о моей модели автоэнкодера: 

Model: "sequential_1"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_1 (Conv2D)            (None, 48, 48, 64)        7808      
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 24, 24, 64)        0         
_________________________________________________________________
conv2d_2 (Conv2D)            (None, 24, 24, 128)       8320      
_________________________________________________________________
conv2d_transpose_1 (Conv2DTr (None, 24, 24, 64)        8256      
_________________________________________________________________
up_sampling2d_1 (UpSampling2 (None, 48, 48, 64)        0         
_________________________________________________________________
conv2d_transpose_2 (Conv2DTr (None, 105, 105, 1)       7745      
=================================================================
Total params: 32,129
Trainable params: 32,129
Non-trainable params: 0
_________________________________________________________________

Вот полное сообщение об ошибке: 

Found 1375004 images belonging to 1 classes.
Found 343750 images belonging to 1 classes.
Epoch 1/5

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-7-5990e699b664> in <module>
     23                                     validation_data=validation_generator,
     24                                     validation_steps = int(validation_generator.samples // batch_size),
---> 25                                     use_multiprocessing=False)

C:\MyProgramFiles\Anaconda3\envs\tf_gpu\lib\site-packages\keras\legacy\interfaces.py in wrapper(*args, **kwargs)
     89                 warnings.warn('Update your `' + object_name + '` call to the ' +
     90                               'Keras 2 API: ' + signature, stacklevel=2)
---> 91             return func(*args, **kwargs)
     92         wrapper._original_function = func
     93         return wrapper

C:\MyProgramFiles\Anaconda3\envs\tf_gpu\lib\site-packages\keras\engine\training.py in fit_generator(self, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, validation_freq, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch)
   1730             use_multiprocessing=use_multiprocessing,
   1731             shuffle=shuffle,
-> 1732             initial_epoch=initial_epoch)
   1733 
   1734     @interfaces.legacy_generator_methods_support

C:\MyProgramFiles\Anaconda3\envs\tf_gpu\lib\site-packages\keras\engine\training_generator.py in fit_generator(model, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, validation_freq, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch)
    201                                      'a tuple `(x, y, sample_weight)` '
    202                                      'or `(x, y)`. Found: ' +
--> 203                                      str(generator_output))
    204                 if x is None or len(x) == 0:
    205                     # Handle data tensors support when no input given

ValueError: Output of generator should be a tuple `(x, y, sample_weight)` or `(x, y)`. Found: [[[[0.54901963]
   [0.5372549 ]
   [0.5294118 ]
   ...
   [0.6117647 ]
   [0.6117647 ]
   [0.63529414]]

  [[0.56078434]
   [0.5529412 ]
   [0.5411765 ]
   ...
   [0.6       ]
   [0.59607846]
   [0.60784316]]

  [[0.5764706 ]
   [0.5647059 ]
   [0.5529412 ]
   ...
   [0.60784316]
   [0.60784316]
   [0.64705884]]

  ...

  [[0.54509807]
   [0.54509807]
   [0.54901963]
   ...
   [0.63529414]
   [0.6156863 ]
   [0.6392157 ]]

  [[0.5254902 ]
   [0.54901963]
   [0.5686275 ]
   ...
   [0.627451  ]
   [0.6       ]
   [0.6       ]]

  [[0.5647059 ]
   [0.54509807]
   [0.56078434]
   ...
   [0.627451  ]
   [0.63529414]
   [0.62352943]]]


 [[[0.5529412 ]
   [0.5686275 ]
   [0.58431375]
   ...
   [0.54509807]
   [0.5764706 ]
   [0.54901963]]

  [[0.56078434]
   [0.5686275 ]
   [0.57254905]
   ...
   [0.5411765 ]
   [0.5764706 ]
   [0.58431375]]

  [[0.56078434]
   [0.56078434]
   [0.5568628 ]
   ...
   [0.54509807]
   [0.5764706 ]
   [0.5294118 ]]

  ...

  [[0.3647059 ]
   [0.37254903]
   [0.3647059 ]
   ...
   [0.36862746]
   [0.37647063]
   [0.33333334]]

  [[0.36078432]
   [0.37647063]
   [0.3803922 ]
   ...
   [0.3529412 ]
   [0.38431376]
   [0.34509805]]

  [[0.36862746]
   [0.3647059 ]
   [0.37254903]
   ...
   [0.3529412 ]
   [0.34901962]
   [0.3372549 ]]]


 [[[0.78823537]
   [0.75294125]
   [0.7686275 ]
   ...
   [0.76470596]
   [0.7686275 ]
   [0.7411765 ]]

  [[0.7725491 ]
   [0.74509805]
   [0.7725491 ]
   ...
   [0.7607844 ]
   [0.76470596]
   [0.7294118 ]]

  [[0.7568628 ]
   [0.7372549 ]
   [0.77647066]
   ...
   [0.75294125]
   [0.75294125]
   [0.7490196 ]]

  ...

  [[0.54509807]
   [0.53333336]
   [0.5372549 ]
   ...
   [0.54901963]
   [0.5294118 ]
   [0.5803922 ]]

  [[0.5294118 ]
   [0.52156866]
   [0.5254902 ]
   ...
   [0.56078434]
   [0.54509807]
   [0.57254905]]

  [[0.5254902 ]
   [0.54509807]
   [0.5568628 ]
   ...
   [0.5058824 ]
   [0.5803922 ]
   [0.53333336]]]


 ...


 [[[0.4039216 ]
   [0.37647063]
   [0.3647059 ]
   ...
   [0.5058824 ]
   [0.48627454]
   [0.5137255 ]]

  [[0.39607847]
   [0.40000004]
   [0.4039216 ]
   ...
   [0.5254902 ]
   [0.52156866]
   [0.43921572]]

  [[0.35686275]
   [0.39607847]
   [0.41176474]
   ...
   [0.5176471 ]
   [0.5294118 ]
   [0.53333336]]

  ...

  [[0.4039216 ]
   [0.41960788]
   [0.42352945]
   ...
   [0.4901961 ]
   [0.48235297]
   [0.4666667 ]]

  [[0.39607847]
   [0.40784317]
   [0.41176474]
   ...
   [0.52156866]
   [0.5019608 ]
   [0.48235297]]

  [[0.3647059 ]
   [0.3921569 ]
   [0.4156863 ]
   ...
   [0.49803925]
   [0.50980395]
   [0.52156866]]]


 [[[0.8431373 ]
   [0.85098046]
   [0.854902  ]
   ...
   [0.9176471 ]
   [0.94117653]
   [0.9490197 ]]

  [[0.8470589 ]
   [0.854902  ]
   [0.86274517]
   ...
   [0.90196085]
   [0.909804  ]
   [0.93725497]]

  [[0.85098046]
   [0.854902  ]
   [0.86666673]
   ...
   [0.909804  ]
   [0.91372555]
   [0.9294118 ]]

  ...

  [[0.8941177 ]
   [0.8745099 ]
   [0.854902  ]
   ...
   [0.9215687 ]
   [0.92549026]
   [0.92549026]]

  [[0.8941177 ]
   [0.8705883 ]
   [0.8470589 ]
   ...
   [0.9490197 ]
   [0.9803922 ]
   [0.8588236 ]]

  [[0.854902  ]
   [0.854902  ]
   [0.89019614]
   ...
   [0.9058824 ]
   [0.9490197 ]
   [0.9058824 ]]]


 [[[0.7019608 ]
   [0.7254902 ]
   [0.70980394]
   ...
   [0.7058824 ]
   [0.72156864]
   [0.7254902 ]]

  [[0.69411767]
   [0.7058824 ]
   [0.69411767]
   ...
   [0.69803923]
   [0.69411767]
   [0.69803923]]

  [[0.69803923]
   [0.69411767]
   [0.6901961 ]
   ...
   [0.69803923]
   [0.69411767]
   [0.7294118 ]]

  ...

  [[0.94117653]
   [0.9176471 ]
   [0.9058824 ]
   ...
   [0.9215687 ]
   [0.92549026]
   [0.8980393 ]]

  [[0.95294124]
   [0.92549026]
   [0.91372555]
   ...
   [0.9176471 ]
   [0.9215687 ]
   [0.9450981 ]]

  [[0.9215687 ]
   [0.9176471 ]
   [0.91372555]
   ...
   [0.9450981 ]
   [0.91372555]
   [0.93725497]]]]

Я искал решения в github и любых других, но это не решает мою проблему. Потому что я не использую никаких пользовательских генераторов данных. Я новичок в keras, любое предложение - это благословение!

1 Ответ

0 голосов
/ 15 марта 2020

Нашел из здесь

, добавив эту функцию: 

def fixed_generator(generator):
    for batch in generator:
        yield (batch, batch)

и затем внедрив ее в .fit_generator: 

history = autoencoder.fit_generator(generator=fixed_generator(training_generator), 
                                    epochs=5, 
                                    steps_per_epoch=training_generator.samples // batch_size,
                                    validation_data=fixed_generator(validation_generator),
                                    validation_steps = validation_generator.samples // batch_size,
                                    use_multiprocessing=False)

Этот код, кажется, работает для меня. Эта проблема также упоминается здесь в стеке потока

вывод: 

Epoch 1/20
10741/10742 [============================>.] - ETA: 0s - loss: 0.0069Epoch 1/20
10742/10742 [==============================] - 1645s 153ms/step - loss: 0.0069 - val_loss: 0.0053
Epoch 2/20
10741/10742 [============================>.] - ETA: 0s - loss: 0.0043Epoch 1/20
10742/10742 [==============================] - 747s 70ms/step - loss: 0.0043 - val_loss: 0.0049
Epoch 3/20
10741/10742 [============================>.] - ETA: 0s - loss: 0.0041Epoch 1/20
10742/10742 [==============================] - 596s 55ms/step - loss: 0.0041 - val_loss: 0.0047
Epoch 4/20
10740/10742 [============================>.] - ETA: 0s - loss: 0.0040Epoch 1/20
10742/10742 [==============================] - 590s 55ms/step - loss: 0.0040 - val_loss: 0.0046
Epoch 5/20
10740/10742 [============================>.] - ETA: 0s - loss: 0.0040Epoch 1/20
10742/10742 [==============================] - 609s 57ms/step - loss: 0.0040 - val_loss: 0.0046
Epoch 6/20
   21/10742 [..............................] - ETA: 5:21 - loss: 0.0040
10741/10742 [============================>.] - ETA: 0s - loss: 0.0039Epoch 1/20
10742/10742 [==============================] - 594s 55ms/step - loss: 0.0039 - val_loss: 0.0046
Epoch 7/20
10740/10742 [============================>.] - ETA: 0s - loss: 0.0039Epoch 1/20
10742/10742 [==============================] - 587s 55ms/step - loss: 0.0039 - val_loss: 0.0045
...