带mask的图像如何使用yolo训练

Question

使用yolo8进行图像分割，但是只有图像和对应的掩膜mask，不知道yolo怎么配置mask进行训练？

Answer 1

让阿豪来帮你解答，本回答参考chatgpt3.5编写提供，如果还有疑问可以评论或留言

解答： 在使用yolo8进行图像分割时，我们可以通过指定掩膜的路径，让yolo8加载掩膜，并在训练时将掩膜信息加入到相应的类别中。 以下是配置yolo8进行图像分割的示例代码：

# 导入必要的库
import tensorflow as tf
from yolo8.model import Yolo8
from yolo8.loss import yolov4_loss
from yolo8.dataset import create_dataset
from yolo8.utils import load_yaml, load_weights
# 载入配置文件
config = load_yaml('config.yaml')
# 创建数据集
train_dataset = create_dataset(config['train_path'], config['batch_size'])
val_dataset = create_dataset(config['val_path'], config['batch_size'], is_training=False)
# 创建模型
model = Yolo8(config['num_classes'], config['anchors'], config['weights_path'])
model.summary()
# 编译模型
optimizer = tf.keras.optimizers.Adam(learning_rate=config['learning_rate'])
model.compile(loss=yolov4_loss(config), optimizer=optimizer, metrics=['accuracy'])
# 加载预训练权重
load_weights(model, config['weights_path'], 'yolo8')
# 训练模型
history = model.fit(train_dataset,
                    epochs=config['epochs'],
                    validation_data=val_dataset,
                    callbacks=[tf.keras.callbacks.TensorBoard(log_dir='./logs')])

在以上示例代码中，我们需要在配置文件中指定训练集存储路径、验证集存储路径、类别数、anchors信息、预训练权重路径等信息，具体可参考以下示例：

num_classes: 2
anchors:
  - [12, 16, 19, 36, 40, 28]
  - [36, 75, 76, 55, 72, 146]
  - [142, 110, 192, 243, 459, 401]
weights_path: 'yolo8_weights.h5'
batch_size: 16
epochs: 50
learning_rate: 0.001
train_path: '/path/to/train/data'
val_path: '/path/to/val/data'

在加载数据集时，我们需要从训练集中读取图像和掩膜，并将它们分别存储在x和y中，具体可参考以下示例：

def load_data(image_path, mask_path):
    image = tf.io.read_file(image_path)
    image = tf.image.decode_jpeg(image, channels=3)
    image = tf.image.resize(image, (448, 448)) / 255.0
    mask = tf.io.read_file(mask_path)
    mask = tf.image.decode_png(mask, channels=1)
    mask = tf.image.resize(mask, (448, 448)) / 255.0
    return image, mask
def create_dataset(data_path, batch_size, is_training=True):
    dataset = tf.data.Dataset.list_files(data_path)
    dataset = dataset.map(lambda x: tf.numpy_function(load_data, [x, x.numpy().decode().replace('.jpg', '.png')], [tf.float32, tf.float32]),
                          num_parallel_calls=tf.data.experimental.AUTOTUNE)
    if is_training:
        dataset = dataset.shuffle(buffer_size=1000)
        dataset = dataset.repeat()
    dataset = dataset.batch(batch_size)
    dataset = dataset.prefetch(buffer_size=tf.data.experimental.AUTOTUNE)
    return dataset

以上示例中，load_data函数用于读取图像和掩膜，并标准化图像和掩膜的像素值；create_dataset函数用于创建数据集，并在训练时对数据进行shuffle和repeat操作。 在加载模型时，我们需要指定掩膜的通道数为1，具体可参考以下示例：

def conv_bn_relu(x, filters, kernel_size, strides=1):
    x = tf.keras.layers.Conv2D(filters, kernel_size, strides=strides, padding='same', use_bias=False)(x)
    x = tf.keras.layers.BatchNormalization()(x)
    x = tf.keras.layers.ReLU()(x)
    return x
def upsample(x, size):
    return tf.image.resize(x, size, method='bilinear')
class Yolo8(tf.keras.models.Model):
    def __init__(self, num_classes, anchors, weights=None):
        super(Yolo8, self).__init__()
        self.num_classes = num_classes
        self.anchors = anchors
        self.backbone = tf.keras.applications.VGG16(include_top=False, weights=None)
        self.heads = [
            tf.keras.Sequential([
                conv_bn_relu(filters=508, kernel_size=3),
                tf.keras.layers.Conv2D(filters=len(anchors[i]) * (num_classes + 5), kernel_size=1, strides=1, padding='same')]) for i in range(3)]
        if weights is not None:
            load_weights(self, weights, 'yolo8')
    def call(self, inputs):
        x = self.backbone(inputs)
        y1 = self.heads[0](x)
        y2 = self.heads[1](x)
        y3 = self.heads[2](x)
        return [y1, y2, y3]
    def inference(self, inputs, conf_thresh=0.5, iou_thresh=0.5):
        outputs = self(inputs)
        results = []
        for i, output in enumerate(outputs):
            results.append(self.decode(output, self.anchors[i], self.num_classes, i, conf_thresh, iou_thresh))
        if len(results) > 1:
            return tf.concat(results, axis=1)
        return results[0]
    def decode(self, output, anchors, num_classes, level, conf_thresh, iou_thresh):
        grid_size = tf.shape(output)[1:3]
        num_anchors = len(anchors)
        box_raw = tf.reshape(output[:, :, :, :num_anchors * 5], [-1, grid_size[0] * grid_size[1], num_anchors, 5])
        class_raw = tf.reshape(output[:, :, :, num_anchors * 5:], [-1, grid_size[0] * grid_size[1], num_classes])
        box_xy = tf.sigmoid(box_raw[..., :2]) + tf.meshgrid(tf.range(grid_size[1]), tf.range(grid_size[0]))[::-1]
        box_wh = tf.exp(box_raw[..., 2:4]) * tf.cast(tf.expand_dims(tf.reshape(anchors, [1, 1, num_anchors, 2]), axis=2), tf.float32)
        box_conf = tf.sigmoid(box_raw[..., 4:5])
        class_prob = tf.nn.softmax(class_raw)
        box_xy /= tf.cast(tf.expand_dims(grid_size, axis=-1), tf.float32)
        box_wh /= tf.cast(tf.expand_dims(grid_size, axis=-1), tf.float32)
        box_min = box_xy - box_wh / 2
        box_max = box_xy + box_wh / 2
        boxes = tf.concat([box_min[..., :1], box_min[..., 1:2],
                           box_max[..., :1], box_max[..., 1:2]], axis=-1)
        scores = box_conf * class_prob
        scores = tf.reduce_max(scores, axis=-1)
        boxes, scores, classes, valid_detections = tf.image.combined_non_max_suppression(
            boxes=tf.expand_dims(boxes, axis=2),
            scores=tf.expand_dims(scores, axis=2),
            max_output_size_per_class=100,
            max_total_size=100,
            iou_threshold=iou_thresh,
            score_threshold=conf_thresh)
        classes = tf.squeeze(classes, axis=2)
        return tf.concat([boxes, scores[..., tf.newaxis], classes[..., tf.newaxis]], axis=-1)    

在以上示例代码中，我们对原有的Yolo8模型进行了修改，使得它可以处理mask信息，并在训练时将掩膜信息加入到相应的类别中。 参考文献： [1]https://github.com/hunglc007/tensorflow-yolov4-tflite [2]https://zhuanlan.zhihu.com/p/145680550