ошибка преобразования данных при обучении собственного набора данных (BDD100K) обнаружению тензорного объекта - PullRequest
0 голосов
/ 20 декабря 2018

Ошибка в tenorflow / models / research / object_detection
Основной код, который преобразует изображения и их аннотации в файлы TFRecord следующим образом:

writer = tf.python_io.TFRecordWriter(output_path)
img_w, img_h = 1280, 720
with open(json_path, 'rb') as json_f:
    items = ijson.items(json_f, 'item')
    img_counter = 0
    skiped_img_counter = 0
    for item in items: # item is a dict, which contains a jpg image and its labels etc.
        img_counter += 1
        img_name = item['name']
        xmins = []
        ymins = []
        xmaxs = []
        ymaxs = []
        classes = []
        labels = []
        occluded = []
        truncated = []

        labels_ = item['labels']
        for label in labels_:
            category = label['category']
            if category in categories:
                nums[category] += 1
                labels.append(label_id[category])
                classes.append(category.encode('utf8'))
                att_ = label['attributes']
                occluded.append(int(att_['occluded'] == 'true'))
                truncated.append(int(att_['truncated'] == 'true'))
                box2d = label['box2d']
                xmins.append(float(box2d['x1'])/img_w) 
                ymins.append(float(box2d['y1'])/img_h) 
                xmaxs.append(float(box2d['x2'])/img_w)
                ymaxs.append(float(box2d['y2'])/img_h)
        difficult_obj = [0] * len(xmins)
        if 0 == len(xmins):
            skiped_img_counter += 1
            print("{0} has no object, skip it and continue.".format(img_name))
            continue
        assert len(xmins) == len(labels) == len(classes) == len(difficult_obj) == len(occluded) == len(truncated), 'not same list length'
        img_path = os.path.join(img_folder, img_name)
        with tf.gfile.GFile(img_path, 'rb') as fid:
            encoded_jpg = fid.read()
        key = hashlib.sha256(encoded_jpg).hexdigest()
        # att = item['attributes']
        # weather, scene, timeofday = att['weather'], att['scene'], att['timeofday']
        tf_example = tf.train.Example(features=tf.train.Features(feature={
            'image/height': int64_feature(img_h),
            'image/width': int64_feature(img_w),
            'image/filename': bytes_feature(img_name.encode('utf8')),
            'image/source_id': bytes_feature(img_name.encode('utf8')),
            'image/key/sha256': bytes_feature(key.encode('utf8')),
            'image/encoded': bytes_feature(encoded_jpg),
            'image/format': bytes_feature('jpg'.encode('utf8')),
            'image/object/bbox/xmin': float_list_feature(xmins),
            'image/object/bbox/xmax': float_list_feature(xmaxs),
            'image/object/bbox/ymin': float_list_feature(ymins),
            'image/object/bbox/ymax': float_list_feature(ymaxs),
            'image/object/bbox/text': bytes_list_feature(classes),
            'image/object/bbox/label': int64_list_feature(labels),
            'image/object/bbox/difficult': int64_list_feature(difficult_obj),
            'image/object/bbox/occluded': int64_list_feature(occluded),
            'image/object/bbox/truncated': int64_list_feature(truncated),
        }))
        print(img_name, 'precessed.')
        writer.write(tf_example.SerializeToString())
    print('{0} images were processed and {1} were skipped.'.format(img_counter, skiped_img_counter))
    print(nums)
    writer.close()

Ошибки следующие:

tenorflow.python.framework.errors_impl.InvalidArgumentError: indices [0] = 0 отсутствует в [0, 0) [[{{node GatherV2_2}} = GatherV2 [Taxis = DT_INT32, Tindices = DT_INT64, Tparams = DT_INT64, _device = "/ device: CPU: 0"] (cond_1 / Merge, Reshape_8, ​​GatherV2_1 / axis)]] [[{{node IteratorGetNext}} = IteratorGetNextoutput_shapes = [[8], [8,300,300,3][8,2], [8,3], [8 100], [8 100,4], [8 100,7], [8 100,7], [8 100], [8 100], [8 100], [8]], output_types = [DT_INT32, DT_FLOAT, DT_INT32, DT_INT32, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_INT32, DT_BOOL, DT_FLOAT, DT_INT32], _device = "/ job: localhost / реплика устройство: 0 / CPU: 0: задача: 0: задача: 00 "]]

1 Ответ

0 голосов
/ 20 декабря 2018

Функция в tf.train.Example

'image/object/bbox/text': bytes_list_feature(classes),
'image/object/bbox/label': int64_list_feature(labels),
'image/object/bbox/difficult': int64_list_feature(difficult_obj),
'image/object/bbox/occluded': int64_list_feature(occluded),
'image/object/bbox/truncated': int64_list_feature(truncated),

должна быть:

'image/object/class/text': bytes_list_feature(classes),
'image/object/class/label': int64_list_feature(labels),
'image/object/difficult': int64_list_feature(difficult_obj),
'image/object/occluded': int64_list_feature(occluded),
'image/object/truncated': int64_list_feature(truncated),   

Это ошибка, вызванная моей небрежностью, все ключи в примере функции должны вcore.standard_field.TfExampleFields

...