CN112288755A - Video-based deep learning segmentation method and system for vehicle exterior parts - Google Patents

Video-based deep learning segmentation method and system for vehicle exterior parts Download PDF

Info

Publication number
CN112288755A
CN112288755A CN202011355098.4A CN202011355098A CN112288755A CN 112288755 A CN112288755 A CN 112288755A CN 202011355098 A CN202011355098 A CN 202011355098A CN 112288755 A CN112288755 A CN 112288755A
Authority
CN
China
Prior art keywords
image
video
segmentation
frame
starting
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202011355098.4A
Other languages
Chinese (zh)
Inventor
丛建亭
黄贤俊
侯进
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenyuan Hengji Technology Co ltd
Original Assignee
Shenyuan Hengji Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenyuan Hengji Technology Co ltd filed Critical Shenyuan Hengji Technology Co ltd
Priority to CN202011355098.4A priority Critical patent/CN112288755A/en
Publication of CN112288755A publication Critical patent/CN112288755A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/11Region-based segmentation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00Computing arrangements using knowledge-based models
    • G06N5/04Inference or reasoning models
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10016Video; Image sequence
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20081Training; Learning
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30248Vehicle exterior or interior
    • G06T2207/30252Vehicle exterior; Vicinity of vehicle

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Evolutionary Computation (AREA)
  • Software Systems (AREA)
  • Artificial Intelligence (AREA)
  • Mathematical Physics (AREA)
  • General Engineering & Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Biomedical Technology (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Biophysics (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Image Analysis (AREA)

Abstract

本发明公开了一种基于视频的车辆外观部件深度学习分割方法和系统,方法包括:获取车辆外观部件的录制视频,确定视频中可辨识车辆外观部件的起始位置作为起始帧;将视频由起始帧开始每间隔预设数量帧取一帧图像存入预设的图像缓冲区;对起始帧的图像进行语义分割,并基于语义分割图像掩码标签进行上色;将语义分割上色图片和图像缓冲区内其余图像输入至训练完成的半监督视频目标分割模型中进行推理分割,输出图像缓冲区中所有图像对应的分割图像。通过本发明的技术方案,解决了不同距离下的图片部件分割识别问题,实现了像素级的目标追踪,无需通过图像特征匹配和逻辑关系对不同图片的部件区域进行关联,提高了视频分割的精度和鲁棒性。

Figure 202011355098

The invention discloses a video-based deep learning segmentation method and system for vehicle exterior parts. The method includes: acquiring a recorded video of the vehicle exterior parts, determining the starting position of the recognizable vehicle exterior parts in the video as a starting frame; Starting from the starting frame, take a frame of image every preset number of frames and store it in the preset image buffer; perform semantic segmentation on the image of the starting frame, and colorize the image based on the semantic segmentation image mask label; color the semantic segmentation The image and the rest of the images in the image buffer are input to the trained semi-supervised video target segmentation model for inference segmentation, and the segmented images corresponding to all the images in the image buffer are output. The technical solution of the present invention solves the problem of image component segmentation and recognition at different distances, realizes pixel-level target tracking, does not need to associate component regions of different pictures through image feature matching and logical relationship, and improves the accuracy of video segmentation and robustness.

Figure 202011355098

Description

Video-based vehicle appearance component deep learning segmentation method and system
Technical Field
The invention relates to the technical field of image processing, in particular to a video-based vehicle appearance component deep learning segmentation method and a video-based vehicle appearance component deep learning segmentation system.
Background
Vehicle appearance part identification is an important part in various automobile businesses, and the vehicle appearance part identification is required in vehicle taking and returning processes such as automobile insurance claim, time-sharing lease and automobile daily lease. The current common implementation modes include two types, one is that field workers survey the vehicle to be recognized and complete the vehicle appearance part recognition, and the other is that users take pictures (or videos) and process the pictures with a deep learning model, such as the multi-task vehicle part recognition model, method and system based on deep learning proposed in the prior art.
The existing system has the following problems:
1. all the assembly component information of the vehicle appearance cannot be accurately identified by singly using a classification model or a detection model, because the vehicle appearance components and the sub-component components are different in size and size in reality, and the detailed images of the parts are similar, so that the damage made from a single image at a longer distance is always the information covering the incomplete vehicle appearance components, the appearance is obviously weaker than the human eye identification capability, and the adoption of image feature matching or logic matching (such as spatial position forced correspondence) usually causes matching failure or error, has low accuracy and is not robust to different scenes (such as night); logical matching may require the user to be careful when taking a picture, and attempting to match by manually and deliberately adjusting the angle and distance may significantly degrade the user's acquisition experience.
2. In actual automobile business application scenes such as insurance claims, car inspection, time-sharing lease and the like, long-distance shooting and short-distance shooting of vehicles are required, and accurate identification of appearances of the vehicles at different distances is a very challenging visual task. The existing system is only suitable for long-distance shooting scenes, and how to accurately identify vehicle components under the condition of short distance becomes a difficult problem influencing the landing of the whole vehicle component identification system.
3. In reality, the loss assessment picture of the claim case is read from far to near in sequence, and is a relatively continuous progressive process, the human eye can accurately identify the appearance part from the vehicle appearance picture under the condition of medium distance or long distance, but the identification capability of the human eye is reduced along with the continuous reduction of the distance, the reduction is mainly caused by that the vehicle detail structures are similar, when only the detail structures exist in the shot picture, even an experienced person can judge that a plurality of possible vehicle positions correspond to the detail structures, the identification capability is consistent with the cognitive knowledge, and therefore, for the appearance parts related to the picture under the special near distance, namely only 1 or 2, the identification capability of the background person or a deep learning model is insufficient. Most mature deep learning image recognition technologies are single-picture recognition, and from the perspective of a single picture, the practical contradiction exists that the vehicle appearance part where the damage is located is easy to recognize at a longer distance but difficult to see or see the damage details, and the vehicle appearance part where the damage is located and the sub-component information on the vehicle appearance part where the damage is located are easy to see and recognize but difficult to recognize at a shorter distance.
Disclosure of Invention
Aiming at the problems, the invention provides a video-based vehicle appearance component deep learning segmentation method and a video-based vehicle appearance component deep learning segmentation system, which are used for segmenting and coloring a start frame image through semantic segmentation, conducting learning reasoning is carried out on a segmented coloring image based on the start frame image by utilizing a semi-supervised video target segmentation model, and pixel-level target tracking is realized, so that the problem of image component segmentation identification at different distances is solved, the association of component areas of different images is not required through image feature matching and logical relations, and the precision and the robustness of video segmentation are improved.
In order to achieve the above object, the present invention provides a video-based vehicle exterior part deep learning segmentation method, including: acquiring a recorded video of the vehicle appearance component, and determining a starting position of the vehicle appearance component recognizable in the video as a starting frame; storing one frame of image of the video, which is started from the initial frame and is spaced by a preset number of frames, into a preset image buffer area; performing semantic segmentation on the image of the initial frame, and coloring based on a mask label of the semantic segmentation image to form a semantic segmentation coloring picture; and inputting the semantic segmentation coloring picture and the rest images in the image buffer area into a trained semi-supervised video target segmentation model for reasoning and segmentation, and outputting segmentation images corresponding to all the images in the image buffer area.
In the above technical solution, preferably, the training method of the semi-supervised video target segmentation model includes: acquiring a recorded video of the vehicle appearance component, and determining a starting position of the vehicle appearance component recognizable in the video as a starting frame; storing one frame of image of the video, which is started from the initial frame and is spaced by a preset number of frames, into a preset image buffer area; performing semantic segmentation on the image of the initial frame, and coloring based on a mask label of the semantic segmentation image to form a semantic segmentation coloring picture; segmenting the rest images in the image buffer area, and labeling the segmented images; and taking the semantic segmentation coloring picture and other images in the image buffer area as input, and taking the image which corresponds to the segmentation and labeling of the image buffer area as output, and training the semi-supervised video target segmentation model.
In the above technical solution, preferably, the semantically segmenting the image of the start frame, and coloring based on the semantically segmented image mask label, and forming the semantically segmented colored picture specifically includes: performing semantic segmentation on the image of the initial frame by adopting a semantic segmentation algorithm; and coloring the segmented image through a preset conversion function according to the mask label of the semantically segmented image, wherein the colored image is used as a semantically segmented coloring image.
In the above technical solution, preferably, the video takes one frame of image from the start frame every 3 frames and stores the frame of image in the image buffer.
The invention also provides a video-based vehicle appearance component deep learning segmentation system, and a video-based vehicle appearance component deep learning segmentation method provided by any one of the technical schemes comprises the following steps: the video acquisition module is used for acquiring a recorded video of the vehicle appearance component and determining a starting position of the vehicle appearance component which can be identified in the video as a starting frame; the image inter-taking module is used for taking one frame of image from the initial frame of the video every preset number of frames and storing the frame of image into a preset image buffer area; the segmentation coloring module is used for performing semantic segmentation on the image of the initial frame and coloring based on a mask label of the semantic segmentation image to form a semantic segmentation coloring image; and the video segmentation module is used for inputting the semantic segmentation coloring picture and other images in the image buffer area into a trained semi-supervised video target segmentation model for reasoning and segmentation and outputting segmentation images corresponding to all the images in the image buffer area.
In the above technical solution, preferably, the training method of the semi-supervised video target segmentation model includes: acquiring a recorded video of the vehicle appearance component, and determining a starting position of the vehicle appearance component recognizable in the video as a starting frame; storing one frame of image of the video, which is started from the initial frame and is spaced by a preset number of frames, into a preset image buffer area; performing semantic segmentation on the image of the initial frame, and coloring based on a mask label of the semantic segmentation image to form a semantic segmentation coloring picture; segmenting the rest images in the image buffer area, and labeling the segmented images; and taking the semantic segmentation coloring picture and other images in the image buffer area as input, and taking the image which corresponds to the segmentation and labeling of the image buffer area as output, and training the semi-supervised video target segmentation model.
In the above technical solution, preferably, the segmentation coloring module is specifically configured to: performing semantic segmentation on the image of the initial frame by adopting a semantic segmentation algorithm; and coloring the segmented image through a preset conversion function according to the mask label of the semantically segmented image, wherein the colored image is used as a semantically segmented coloring image.
In the above technical solution, preferably, the image inter-taking module takes one frame of image of the video from the start frame every 3 frames and stores the frame of image in the image buffer area.
Compared with the prior art, the invention has the beneficial effects that: the starting frame image is segmented and colored through semantic segmentation, a semi-supervised video target segmentation model is utilized, conducting learning reasoning is carried out on the segmented colored image based on the starting frame image, and pixel-level target tracking is achieved, so that the problem of picture component segmentation identification under different distances is solved, component areas of different pictures do not need to be associated through image feature matching and logical relations, and the precision and robustness of video segmentation are improved.
Drawings
FIG. 1 is a schematic flow chart illustrating a video-based method for deep learning and segmenting vehicle exterior parts according to an embodiment of the present invention;
FIG. 2 is a schematic diagram illustrating segmentation and coloring of an initial frame image according to an embodiment of the present invention;
FIG. 3 is a block diagram illustrating a video-based vehicle appearance component deep learning segmentation system according to an embodiment of the present disclosure.
In the drawings, the correspondence between each component and the reference numeral is:
11. the system comprises a video acquisition module, 12, an image inter-taking module, 13, a segmentation and coloring module and 14, a video segmentation module.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, are within the scope of the present invention.
The invention is described in further detail below with reference to the attached drawing figures:
as shown in fig. 1, the method for deeply learning and segmenting the vehicle appearance part based on the video provided by the invention comprises the following steps: acquiring a recorded video of the vehicle appearance component, and determining a starting position of the vehicle appearance component recognizable in the video as a starting frame; storing one frame of image of the video, which is started from the initial frame and is spaced by a preset number of frames, into a preset image buffer area; performing semantic segmentation on the image of the initial frame, and coloring based on a mask label of the semantic segmented image to form a semantic segmented colored image; and inputting the semantic segmentation coloring picture and the rest of images in the image buffer area into a trained semi-supervised video target segmentation model for reasoning and segmentation, and outputting segmentation images corresponding to all the images in the image buffer area.
In the embodiment, the starting frame image is segmented and colored through semantic segmentation, a semi-supervised video target segmentation model is utilized, and conducted learning reasoning is carried out on the segmented colored image based on the starting frame image, so that the pixel-level target tracking is realized, the problem of image component segmentation identification at different distances is solved, the association of component areas of different images is not needed through image feature matching and logical relations, and the precision and the robustness of video segmentation are improved.
Specifically, taking the segmentation of the video for damage assessment as an example, a category of a "recognizable image of vehicle appearance parts" is captured in the video damage assessment recording, the category can be obtained by using a deep learning classification model, and the current frame is used as a starting image for the segmentation of the semi-supervised video target. An image buffer area is established in advance as a data storage space of the video loss assessment, assuming that the video frame rate is 30 frames/s, preferably, one frame of image is taken to be placed in the image buffer area every interval N-3 frames, where N-3 frames are tested experimentally, and if N is taken too large, the pixel offset of the image sequence in the buffer area is large, which exceeds the learning capability of a deep learning network. And performing semantic segmentation of a single picture on the 1 st frame (initial frame) of the image buffer area, and coloring based on the output semantic segmentation image mask label, thereby converting the frame into a colorful semantic segmentation coloring picture. And inputting the pictures in the image buffer area and the semantic segmentation coloring pictures of the 1 st frame into a semi-supervised video target segmentation model, and reasoning to obtain video segmentation images of all the pictures in the image buffer area, thereby realizing the pixel-level part identification information of the vehicle appearance parts and the sub-part accessories from far to near.
As shown in fig. 2, specifically, the image semantic segmentation algorithm, such as: and performing appearance component segmentation on the vehicle picture after training, such as deep, PSPNet, SegNet, FCN, DIS, IDW-CNN and the like, and obtaining a segmentation result. The principle is illustrated below by the example of Deeplab:
1) a deep convolution neural network, such as VGG-16 or ResNet-101, adopts a full convolution mode to reduce the degree of signal down-sampling (from 32x to 8x) by using porous convolution;
2) in a bilinear interpolation stage, increasing the resolution of the feature map to the original image;
3) and optimizing a segmentation result by using a conditional random field, and better grabbing the edge of the object.
The meaning of segmentation is to give specific pixel information of each part while distinguishing various vehicle appearance parts. The semi-supervised video object segmentation technology only gives a correct segmentation mask of a 1 st frame of a video, and then segments a labeled object in each subsequent continuous frame in a pixel level, which is actually a pixel-level object tracking problem, and the following methods are commonly used: STM, CFBI, VOT, FTMU, TVOS, etc. Taking TVOS as an example, the method adopts a label propagation mode, is simple, high in performance and high in efficiency, conducts learning based on the current frame, the historical frame and the image label to infer the image label of the current frame, and has a certain short-time memory function.
In the above embodiment, preferably, the training method of the semi-supervised video object segmentation model includes: acquiring a recorded video of the vehicle appearance component, and determining a starting position of the vehicle appearance component recognizable in the video as a starting frame; storing one frame of image of the video, which is started from the initial frame and is spaced by a preset number of frames, into a preset image buffer area; performing semantic segmentation on the image of the initial frame, and coloring based on a mask label of the semantic segmented image to form a semantic segmented colored image; segmenting the rest images in the image buffer area, and labeling the segmented images; and training a semi-supervised video target segmentation model by taking the semantic segmentation coloring picture and the rest of images in the image buffer area as input and taking the segmented and labeled images of the corresponding image buffer area as output.
In the embodiment, the image in the image buffer area and the colored initial frame image are used as input, the pre-labeled segmentation image is used as output, the semi-supervised video target segmentation model is trained until convergence, and after a new initial frame semantic segmentation coloring image and a video image of a subsequent frame are input, the segmentation image of the video image of the subsequent frame can be obtained through reasoning, so that the pixel-level target tracking is realized. By utilizing the trained semi-supervised video target segmentation model, the vehicle appearance part identification capability is more robust and the accuracy is higher.
In the foregoing embodiment, preferably, performing semantic segmentation on the image of the start frame, and coloring based on the semantic segmentation image mask label, and forming a semantic segmentation colored picture specifically includes: performing semantic segmentation on the image of the initial frame by adopting a semantic segmentation algorithm; and coloring the segmented image through a preset conversion function according to the mask label of the semantically segmented image, wherein the colored image is used as a semantically segmented coloring image.
As shown in fig. 3, the present invention further provides a video-based vehicle exterior component deep learning segmentation system, to which the video-based vehicle exterior component deep learning segmentation method proposed in any one of the above embodiments is applied, including: the video acquisition module 11 is configured to acquire a recorded video of the vehicle appearance component, and determine a starting position of the vehicle appearance component recognizable in the video as a starting frame; the image inter-taking module 12 is used for taking one frame of image from the initial frame of the video at intervals of a preset number of frames and storing the frame of image into a preset image buffer area; the segmentation coloring module 13 is configured to perform semantic segmentation on the image of the start frame, and color the image based on the mask label of the semantic segmentation image to form a semantic segmentation coloring image; and the video segmentation module 14 is used for inputting the semantic segmentation coloring picture and the rest of images in the image buffer area into the trained semi-supervised video target segmentation model for reasoning and segmentation, and outputting segmentation images corresponding to all the images in the image buffer area.
In the embodiment, the vehicle appearance component deep learning segmentation system based on the video applies the vehicle appearance component deep learning segmentation method in the embodiment, the starting frame image is segmented and colored through semantic segmentation, and a semi-supervised video target segmentation model is utilized to conduct learning inference based on the segmented colored image of the starting frame image, so that pixel-level target tracking is realized, the problem of image component segmentation identification at different distances is solved, association of component areas of different images is not needed through image feature matching and logical relations, and the precision and robustness of video segmentation are improved.
In the above embodiment, preferably, the training method of the semi-supervised video object segmentation model includes: acquiring a recorded video of the vehicle appearance component, and determining a starting position of the vehicle appearance component recognizable in the video as a starting frame; storing one frame of image of the video, which is started from the initial frame and is spaced by a preset number of frames, into a preset image buffer area; performing semantic segmentation on the image of the initial frame, and coloring based on a mask label of the semantic segmented image to form a semantic segmented colored image; segmenting the rest images in the image buffer area, and labeling the segmented images; and training a semi-supervised video target segmentation model by taking the semantic segmentation coloring picture and the rest of images in the image buffer area as input and taking the segmented and labeled images of the corresponding image buffer area as output.
In the embodiment, the image in the image buffer area and the colored initial frame image are used as input, the pre-labeled segmentation image is used as output, the semi-supervised video target segmentation model is trained until convergence, and after a new initial frame semantic segmentation coloring image and a video image of a subsequent frame are input, the segmentation image of the video image of the subsequent frame can be obtained through reasoning, so that the pixel-level target tracking is realized. By utilizing the trained semi-supervised video target segmentation model, the vehicle appearance part identification capability is more robust and the accuracy is higher.
In the above embodiment, preferably, the segmentation and coloring module is specifically configured to: performing semantic segmentation on the image of the initial frame by adopting a semantic segmentation algorithm; and coloring the segmented image through a preset conversion function according to the mask label of the semantically segmented image, wherein the colored image is used as a semantically segmented coloring image.
In the above embodiment, preferably, the inter-image fetching module fetches one frame of image of the video from the start frame every 3 frames and stores the fetched frame of image in the image buffer.
The above is only a preferred embodiment of the present invention, and is not intended to limit the present invention, and various modifications and changes will occur to those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (8)

1.一种基于视频的车辆外观部件深度学习分割方法,其特征在于,包括:1. a video-based deep learning segmentation method of vehicle appearance parts, is characterized in that, comprises: 获取车辆外观部件的录制视频,确定视频中可辨识车辆外观部件的起始位置作为起始帧;Obtain the recorded video of the vehicle exterior parts, and determine the starting position of the recognizable vehicle exterior parts in the video as the starting frame; 将所述视频由所述起始帧开始每间隔预设数量帧取一帧图像存入预设的图像缓冲区;The video is stored in a preset image buffer by taking a frame of image every interval preset number of frames starting from the starting frame; 对所述起始帧的图像进行语义分割,并基于语义分割图像掩码标签进行上色,形成语义分割上色图片;Semantic segmentation is performed on the image of the initial frame, and coloring is performed based on the semantic segmentation image mask label to form a semantic segmentation colored image; 将所述语义分割上色图片和所述图像缓冲区内其余图像输入至训练完成的半监督视频目标分割模型中进行推理分割,输出所述图像缓冲区中所有图像对应的分割图像。Input the semantically segmented colored picture and the remaining images in the image buffer into the trained semi-supervised video target segmentation model for inference segmentation, and output the segmented images corresponding to all the images in the image buffer. 2.根据权利要求1所述的基于视频的车辆外观部件深度学习分割方法,其特征在于,所述半监督视频目标分割模型的训练方法包括:2. The video-based vehicle appearance component deep learning segmentation method according to claim 1, wherein the training method of the semi-supervised video target segmentation model comprises: 获取车辆外观部件的录制视频,确定视频中可辨识车辆外观部件的起始位置作为起始帧;Obtain the recorded video of the vehicle exterior parts, and determine the starting position of the recognizable vehicle exterior parts in the video as the starting frame; 将所述视频由所述起始帧开始每间隔预设数量帧取一帧图像存入预设的图像缓冲区;The video is stored in a preset image buffer by taking a frame of image every interval preset number of frames starting from the starting frame; 对所述起始帧的图像进行语义分割,并基于语义分割图像掩码标签进行上色,形成语义分割上色图片;Semantic segmentation is performed on the image of the initial frame, and coloring is performed based on the semantic segmentation image mask label to form a semantic segmentation colored image; 对所述图像缓冲区中的其余图像进行分割,并对分割后的图像进行标注;Segmenting the remaining images in the image buffer, and labeling the segmented images; 将所述语义分割上色图片以及所述图像缓冲区内的其余图像作为输入、以对应所述图像缓冲区分割和标注后的图像作为输出,对所述半监督视频目标分割模型进行训练。The semi-supervised video target segmentation model is trained using the semantically segmented colored picture and the remaining images in the image buffer as inputs, and the segmented and labeled images corresponding to the image buffer as outputs. 3.根据权利要求1所述的基于视频的车辆外观部件深度学习分割方法,其特征在于,所述对所述起始帧的图像进行语义分割,并基于语义分割图像掩码标签进行上色,形成语义分割上色图片具体包括:3. The video-based deep learning segmentation method for vehicle appearance components according to claim 1, wherein the image of the starting frame is semantically segmented, and the image is colored based on the semantic segmentation image mask label, Forming a semantically segmented colored image specifically includes: 采用语义分割算法对所述起始帧的图像进行语义分割;Semantic segmentation is performed on the image of the initial frame by using a semantic segmentation algorithm; 根据语义分割后图像的掩码标签,通过预设的转换函数对分割后的图像进行上色,上色后的图像作为语义分割上色图片。According to the mask label of the semantically segmented image, the segmented image is colored by a preset conversion function, and the colored image is used as a semantically segmented colored image. 4.根据权利要求1所述的基于视频的车辆外观部件深度学习分割方法,其特征在于,所述视频由所述起始帧每隔3帧取一帧图像存入所述图像缓冲区。4 . The video-based deep learning segmentation method for vehicle exterior parts according to claim 1 , wherein the video is stored in the image buffer by taking one frame of images every 3 frames from the starting frame. 5 . 5.一种基于视频的车辆外观部件深度学习分割系统,应用权利要求1至4中任一项所述的基于视频的车辆外观部件深度学习分割方法,其特征在于,包括:5. A video-based deep learning segmentation system for vehicle appearance parts, applying the video-based vehicle appearance parts deep learning segmentation method according to any one of claims 1 to 4, characterized in that, comprising: 视频获取模块,用于获取车辆外观部件的录制视频,确定视频中可辨识车辆外观部件的起始位置作为起始帧;The video acquisition module is used to acquire the recorded video of the vehicle exterior parts, and determine the starting position of the recognizable vehicle exterior parts in the video as the starting frame; 图像间取模块,用于将所述视频由所述起始帧开始每间隔预设数量帧取一帧图像存入预设的图像缓冲区;an image-to-image module, which is used to store a frame of the video into a preset image buffer at every interval of a preset number of frames starting from the start frame; 分割上色模块,用于对所述起始帧的图像进行语义分割,并基于语义分割图像掩码标签进行上色,形成语义分割上色图片;A segmentation and coloring module, which is used to perform semantic segmentation on the image of the initial frame, and perform coloring based on the semantic segmentation image mask label, to form a semantic segmentation and coloring picture; 视频分割模块,用于将所述语义分割上色图片和所述图像缓冲区内其余图像输入至训练完成的半监督视频目标分割模型中进行推理分割,输出所述图像缓冲区中所有图像对应的分割图像。The video segmentation module is used to input the semantically segmented colored pictures and the remaining images in the image buffer into the semi-supervised video target segmentation model that has been trained for inference segmentation, and output the corresponding images of all the images in the image buffer. Split the image. 6.根据权利要求5所述的基于视频的车辆外观部件深度学习分割系统,其特征在于,所述半监督视频目标分割模型的训练方法包括:6. The video-based vehicle appearance component deep learning segmentation system according to claim 5, wherein the training method of the semi-supervised video target segmentation model comprises: 获取车辆外观部件的录制视频,确定视频中可辨识车辆外观部件的起始位置作为起始帧;Obtain the recorded video of the vehicle exterior parts, and determine the starting position of the recognizable vehicle exterior parts in the video as the starting frame; 将所述视频由所述起始帧开始每间隔预设数量帧取一帧图像存入预设的图像缓冲区;The video is stored in a preset image buffer by taking a frame of image every interval preset number of frames starting from the starting frame; 对所述起始帧的图像进行语义分割,并基于语义分割图像掩码标签进行上色,形成语义分割上色图片;Semantic segmentation is performed on the image of the initial frame, and coloring is performed based on the semantic segmentation image mask label to form a semantic segmentation colored image; 对所述图像缓冲区中的其余图像进行分割,并对分割后的图像进行标注;Segmenting the remaining images in the image buffer, and labeling the segmented images; 将所述语义分割上色图片以及所述图像缓冲区内的其余图像作为输入、以对应所述图像缓冲区分割和标注后的图像作为输出,对所述半监督视频目标分割模型进行训练。The semi-supervised video target segmentation model is trained using the semantically segmented colored picture and the remaining images in the image buffer as inputs, and the segmented and labeled images corresponding to the image buffer as outputs. 7.根据权利要求5所述的基于视频的车辆外观部件深度学习分割系统,其特征在于,所述分割上色模块具体用于:7. The video-based deep learning segmentation system for vehicle exterior parts according to claim 5, wherein the segmentation and coloring module is specifically used for: 采用语义分割算法对所述起始帧的图像进行语义分割;Semantic segmentation is performed on the image of the initial frame by using a semantic segmentation algorithm; 根据语义分割后图像的掩码标签,通过预设的转换函数对分割后的图像进行上色,上色后的图像作为语义分割上色图片。According to the mask label of the semantically segmented image, the segmented image is colored by a preset conversion function, and the colored image is used as a semantically segmented colored image. 8.根据权利要求5所述的基于视频的车辆外观部件深度学习分割系统,其特征在于,所述图像间取模块将所述视频由所述起始帧每隔3帧取一帧图像存入所述图像缓冲区。8 . The video-based deep learning segmentation system for vehicle exterior parts according to claim 5 , wherein the inter-image fetching module stores an image of the video every 3 frames from the starting frame. 9 . the image buffer.
CN202011355098.4A 2020-11-26 2020-11-26 Video-based deep learning segmentation method and system for vehicle exterior parts Pending CN112288755A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011355098.4A CN112288755A (en) 2020-11-26 2020-11-26 Video-based deep learning segmentation method and system for vehicle exterior parts

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011355098.4A CN112288755A (en) 2020-11-26 2020-11-26 Video-based deep learning segmentation method and system for vehicle exterior parts

Publications (1)

Publication Number Publication Date
CN112288755A true CN112288755A (en) 2021-01-29

Family

ID=74425643

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011355098.4A Pending CN112288755A (en) 2020-11-26 2020-11-26 Video-based deep learning segmentation method and system for vehicle exterior parts

Country Status (1)

Country Link
CN (1) CN112288755A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113962989A (en) * 2021-12-08 2022-01-21 成都数之联科技有限公司 Vehicle appearance assembly part identification method, system, device and medium
CN114677567A (en) * 2022-05-27 2022-06-28 成都数联云算科技有限公司 Model training method and device, storage medium and electronic equipment

Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107610141A (en) * 2017-09-05 2018-01-19 华南理工大学 A kind of remote sensing images semantic segmentation method based on deep learning
CN108898618A (en) * 2018-06-06 2018-11-27 上海交通大学 A kind of Weakly supervised video object dividing method and device
CN108986136A (en) * 2018-07-23 2018-12-11 南昌航空大学 A kind of binocular scene flows based on semantic segmentation determine method and system
CN109640168A (en) * 2018-11-27 2019-04-16 Oppo广东移动通信有限公司 Method for processing video frequency, device, electronic equipment and computer-readable medium
CN109657599A (en) * 2018-12-13 2019-04-19 深源恒际科技有限公司 Image identification method apart from adaptive vehicle appearance component
CN109657596A (en) * 2018-12-12 2019-04-19 天津卡达克数据有限公司 A kind of vehicle appearance component identification method based on deep learning
CN109753913A (en) * 2018-12-28 2019-05-14 东南大学 Computationally Efficient Multimodal Video Semantic Segmentation Method
US20190311202A1 (en) * 2018-04-10 2019-10-10 Adobe Inc. Video object segmentation by reference-guided mask propagation
CN110910391A (en) * 2019-11-15 2020-03-24 安徽大学 A dual-module neural network structure video object segmentation method
CN111462132A (en) * 2020-03-20 2020-07-28 西北大学 A method and system for video object segmentation based on deep learning
CN111652899A (en) * 2020-05-29 2020-09-11 中国矿业大学 A video object segmentation method based on spatiotemporal component graph
US20200364461A1 (en) * 2019-05-17 2020-11-19 Shanghai Bilibili Technology Co., Ltd. Method of obtaining mask frame data, computing device, and readable storage medium

Patent Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107610141A (en) * 2017-09-05 2018-01-19 华南理工大学 A kind of remote sensing images semantic segmentation method based on deep learning
US20190311202A1 (en) * 2018-04-10 2019-10-10 Adobe Inc. Video object segmentation by reference-guided mask propagation
CN108898618A (en) * 2018-06-06 2018-11-27 上海交通大学 A kind of Weakly supervised video object dividing method and device
CN108986136A (en) * 2018-07-23 2018-12-11 南昌航空大学 A kind of binocular scene flows based on semantic segmentation determine method and system
CN109640168A (en) * 2018-11-27 2019-04-16 Oppo广东移动通信有限公司 Method for processing video frequency, device, electronic equipment and computer-readable medium
CN109657596A (en) * 2018-12-12 2019-04-19 天津卡达克数据有限公司 A kind of vehicle appearance component identification method based on deep learning
CN109657599A (en) * 2018-12-13 2019-04-19 深源恒际科技有限公司 Image identification method apart from adaptive vehicle appearance component
CN109753913A (en) * 2018-12-28 2019-05-14 东南大学 Computationally Efficient Multimodal Video Semantic Segmentation Method
US20200364461A1 (en) * 2019-05-17 2020-11-19 Shanghai Bilibili Technology Co., Ltd. Method of obtaining mask frame data, computing device, and readable storage medium
CN110910391A (en) * 2019-11-15 2020-03-24 安徽大学 A dual-module neural network structure video object segmentation method
CN111462132A (en) * 2020-03-20 2020-07-28 西北大学 A method and system for video object segmentation based on deep learning
CN111652899A (en) * 2020-05-29 2020-09-11 中国矿业大学 A video object segmentation method based on spatiotemporal component graph

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
S. CAELLES等: "One-Shot Video Object Segmentation", 《CVPR》, pages 221 - 230 *
TARUN KALLURI等: "Universal Semi-Supervised Semantic Segmentation", 《ICCV》, pages 5259 - 5270 *
常勤伟等: "基于共生关系的车辆部件识别", 《中国优秀硕士学位论文全文数据库 工程科技II辑》, no. 2020, pages 034 - 819 *
张婷: "多样化场景中的半监督视频目标分割算法研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》, no. 2020, pages 138 - 927 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113962989A (en) * 2021-12-08 2022-01-21 成都数之联科技有限公司 Vehicle appearance assembly part identification method, system, device and medium
CN114677567A (en) * 2022-05-27 2022-06-28 成都数联云算科技有限公司 Model training method and device, storage medium and electronic equipment

Similar Documents

Publication Publication Date Title
CN110569702B (en) Video stream processing method and device
CN108921782B (en) Image processing method, device and storage medium
CN110569700B (en) Method and device for optimizing damage identification result
CN110956088B (en) Overlapped text line positioning and segmentation method and system based on deep learning
CN101601287A (en) Produce the equipment and the method for photorealistic image thumbnails
CN111382647B (en) Picture processing method, device, equipment and storage medium
CN112288755A (en) Video-based deep learning segmentation method and system for vehicle exterior parts
CN109558505A (en) Visual search method, apparatus, computer equipment and storage medium
CN110390308A (en) A Video Action Recognition Method Based on Spatio-temporal Adversarial Generative Network
CN109657599B (en) Picture identification method of distance-adaptive vehicle appearance part
CN118314611A (en) Face living body detection method and device
CN112381840B (en) Method and system for marking vehicle appearance parts in loss assessment video
CN111325107A (en) Detection model training method and device, electronic equipment and readable storage medium
CN111311601B (en) Segmentation method and device for spliced image
CN111062347B (en) Traffic element segmentation method in automatic driving, electronic equipment and storage medium
CN114399801A (en) Target detection method and device
CN117218109B (en) Vehicle lateral mosaic image integrity detection method, system, equipment and medium
CN115909254B (en) DMS system based on camera original image and image processing method thereof
CN113221818B (en) An intelligent detection method and system for sea surface oil spill characteristics
CN113111730B (en) Quick high-precision image blurring detection method and device
Žižakić et al. Efficient local image descriptors learned with autoencoders
Luo et al. Localization-aware logit mimicking for object detection in adverse weather conditions
CN114120394A (en) Face key point detection method and device
CN114495044A (en) Label identification method, label identification device, computer equipment and storage medium
US10896333B2 (en) Method and device for aiding the navigation of a vehicle

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20210129