WO2020119624A1 - Procédé de détection de bord sensible à la classe basé sur l'apprentissage profond - Google Patents

Procédé de détection de bord sensible à la classe basé sur l'apprentissage profond Download PDF

Info

Publication number
WO2020119624A1
WO2020119624A1 PCT/CN2019/123968 CN2019123968W WO2020119624A1 WO 2020119624 A1 WO2020119624 A1 WO 2020119624A1 CN 2019123968 W CN2019123968 W CN 2019123968W WO 2020119624 A1 WO2020119624 A1 WO 2020119624A1
Authority
WO
WIPO (PCT)
Prior art keywords
deep learning
class
feature
edge
edge detection
Prior art date
Application number
PCT/CN2019/123968
Other languages
English (en)
Chinese (zh)
Inventor
王磊
徐成俊
程俊
Original Assignee
中国科学院深圳先进技术研究院
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中国科学院深圳先进技术研究院 filed Critical 中国科学院深圳先进技术研究院
Publication of WO2020119624A1 publication Critical patent/WO2020119624A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/13—Edge detection

Definitions

  • the invention relates to the field of image processing, in particular to the edge detection technology of images.
  • Edge detection is a basic problem in the field of image processing.
  • the research on edge segmentation has always been the basic task of computer vision and the technical foundation of many researches and applications.
  • cell segmentation of medical images, road segmentation in autonomous driving, etc. are all specific applications of edge detection technology.
  • edge detection algorithms Sobel operator, Laplacian operator, and Canny operator. These methods have their own applicable situations, and the efficiency and robustness of obtaining edge images are limited.
  • Some technologies have gradually proposed the use of deep learning to obtain image edges, such as references [1] and [2]. Although these technologies have improved efficiency and robustness, they have been greatly improved.
  • the edge of is relatively thicker than the real edge, and the detection accuracy still needs to be improved.
  • these edge detection methods can only obtain binary edge images, and cannot classify the obtained edges, and further obtain the semantic information of the edges.
  • the present invention provides an edge detection method with high detection accuracy and capable of obtaining category information of edges.
  • a class-sensitive edge detection method based on deep learning using a deep learning neural network model to obtain edge classification detection results of several class targets.
  • the deep learning neural network model is a CNN convolutional neural network model, which includes a feature extraction step 1, an upsampling step 2, and a feature fusion and classification step 3.
  • the feature extraction step 1 includes five stages of feature extraction, S1-S5, to obtain edge feature maps of binary classification with different scales.
  • an adaptive scale transformation label is used to supervise the training.
  • the up-sampling step 2 is up-sampling the edge features extracted by the feature extraction step 1, the up-sampled image size is consistent with the size of the image to be detected; the feature fusion and classification step 3 is based on Based on the result of the upsampling step 2, feature fusion is performed, and the edge categories are classified at the same time.
  • F new ⁇ F 1 ,E,F 2 ,E,...,F n ,E ⁇ ,
  • E represents the set of edge feature images obtained by supervised learning in feature extraction step 1
  • F represents the feature obtained in the sixth stage S6
  • F n means that F has n channels in total
  • n is the number of categories of the target to be identified
  • F new is the new feature obtained after feature fusion
  • the sixth-stage feature S6 is the up-sampling of the fifth-stage feature S5.
  • the loss function of the CNN convolutional neural network model is used as follows: the loss function of the reset weight is used in the supervision of the binary image of the S1-S5 stage; the general category is used in the multi-class edge supervision of the S6 stage Cross loss function; the loss function of the reset weight used in the final feature fusion and classification.
  • the obtained edges of different categories are represented by different colors.
  • the present invention can detect specific target edges and classify the obtained edges at the same time. Compared with the existing deep learning edge detection method, it not only improves the detection accuracy, but also obtains a more detailed image Edge, and almost no post-processing is required; and the function has been expanded to provide higher performance guarantee for other tasks based on edge, such as target segmentation and instance segmentation.
  • Figure 1 is an algorithm block diagram of the present invention.
  • Figure 2 is a comparison chart using different loss functions.
  • the algorithm block diagram of the class-sensitive edge detection method based on deep learning is shown in Figure 1.
  • the feature extraction step 1 includes five stages, namely S1 to S5, which can obtain edge feature maps of binary classification with different scales, that is, S1-edge and S2-edge in FIG. 1.
  • the size of the S1 to S4 process is reduced twice, and the S4 size is as large as the S5 size.
  • the up-sampling step 2 includes the S6 stage, as well as S1-edge-upsample, S2-edge-upsample, and S3-edge-upsample.
  • S6 is the up-sampling of S5 features, the size of the up-sampling is consistent with the image to be detected.
  • S1-edge-upsample, S2-edge-upsample, and S3-edge-upsample are the up-sampling of S1-edge, S2-edge, and S3-edge, respectively, and the size is also consistent with the image to be detected.
  • Feature fusion and classification step 3 is to fuse all the up-sampled features and get the final multi-class target edge detection result.
  • Feature fusion and classification are as follows:
  • F new ⁇ F 1 ,E,F 2 ,E,...,F n ,E ⁇
  • E represents the edge feature image set supervised and learned in the middle stage
  • F is the feature obtained in the sixth stage
  • F n means that there are n channels in total, and n is the number of categories that want to identify the target.
  • F new is a new feature obtained after feature fusion.
  • the new feature F new is obtained by fusion for the final multi-category classification of edges.
  • supervised learning is adopted for CNN convolutional neural network.
  • Supervise training of multi-scale binary edge images in S1-S5 process that is, only perform edge two-class classification; then perform multi-classification of edge pixels at S6, classification output at S6 stage, which is used for preliminary supervised learning of the model ,
  • the number of output channels is consistent with the number of categories to be detected.
  • the training data label because the feature scales obtained in each stage are different during feature extraction, such as S1-edge, S2-edge, and S3-edge in Figure 1, the training data label only provides one, so it is proposed
  • An adaptive scale transformation label is used for supervised training of intermediate processes. When performing stage supervision training, the label will be adjusted to the size of the corresponding stage feature map according to the size of the current feature map, and then the loss function calculation will be performed.
  • the reset weight loss function and the general cross loss entropy function as the loss function of the entire neural network, specifically: the reset weight is used in the supervision of the binary image in the S1-S5 stage The loss function of Classification; the multi-class edge supervision in the Classification of S6 stage uses the general cross loss function; the loss function of the final weight used in the final feature fusion and classification. After many experiments, the results show that the cross-reuse loss function and the general cross loss entropy function can improve the detection accuracy and refine the edges.
  • the present invention uses two data sets for training and testing.
  • One is the SBD (Semantic Boundary Dataset) dataset.
  • the SBD dataset contains 11355 pictures, of which 8498 are used for training and 2857 for verification.
  • the second is the Cityscapes dataset, which contains 5000 images, of which 2975 are used for training, 500 for verification, and 1525 for testing.
  • the input image size is 400x400, and the evaluation index used is F-measure.
  • the calculation method is as follows:
  • the 20 types of verification indicators on the SBD data set are as follows:
  • FIG. 2(c) The effect of the present invention is shown in Figure 2, where (a) and (d) are the original pictures, (b) and (e) are the detection results obtained by using only the loss function of reset weights, (c) and (f) Test results obtained by using the method of the present invention.
  • Fig. 2(c) different categories of areas are marked with different colors, in which chairs and sofas are represented by red and green frames, namely dark and light frames in Fig. 2(c).

Abstract

L'invention concerne un procédé de détection de bord sensible à la classe basé sur l'apprentissage profond. Un modèle de réseau neuronal d'apprentissage profond est utilisé pour obtenir des résultats de détection de classification de bord de plusieurs cibles de classe. Un procédé de supervision profonde est utilisé pour l'entraînement de modèle. Des étiquettes pour une transformation d'échelle adaptative sont utilisées pendant un processus d'entraînement. Une fonction de perte d'une pondération de réinitialisation et une fonction d'entropie de perte croisée générale sont utilisées de façon croisée. En employant le présent procédé, des bords cibles spécifiques peuvent être détectés et les bords obtenus peuvent être classés en même temps. Par rapport aux procédés de détection de bord par apprentissage profond existants, la présente invention non seulement améliore la précision de détection, mais obtient également des bords d'image plus détaillés tout en nécessitant peu de retraitement ultérieur. De plus, des fonctions ont été étendues, ce qui peut offrir une meilleure garantie de performance pour d'autres tâches qui se basent sur des bords, telles que la segmentation de cible et la segmentation d'instance.
PCT/CN2019/123968 2018-12-12 2019-12-09 Procédé de détection de bord sensible à la classe basé sur l'apprentissage profond WO2020119624A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811520198.0A CN109741351A (zh) 2018-12-12 2018-12-12 一种基于深度学习的类别敏感型边缘检测方法
CN201811520198.0 2018-12-12

Publications (1)

Publication Number Publication Date
WO2020119624A1 true WO2020119624A1 (fr) 2020-06-18

Family

ID=66359395

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/123968 WO2020119624A1 (fr) 2018-12-12 2019-12-09 Procédé de détection de bord sensible à la classe basé sur l'apprentissage profond

Country Status (2)

Country Link
CN (1) CN109741351A (fr)
WO (1) WO2020119624A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115588024A (zh) * 2022-11-25 2023-01-10 东莞市兆丰精密仪器有限公司 一种基于人工智能的复杂工业影像边缘提取方法及装置

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109741351A (zh) * 2018-12-12 2019-05-10 中国科学院深圳先进技术研究院 一种基于深度学习的类别敏感型边缘检测方法
CN110310254B (zh) * 2019-05-17 2022-11-29 广东技术师范大学 一种基于深度学习的房角图像自动分级方法

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018023734A1 (fr) * 2016-08-05 2018-02-08 深圳大学 Procédé de test de signification pour image 3d
CN108710919A (zh) * 2018-05-25 2018-10-26 东南大学 一种基于多尺度特征融合深度学习的裂缝自动化勾画方法
CN109741351A (zh) * 2018-12-12 2019-05-10 中国科学院深圳先进技术研究院 一种基于深度学习的类别敏感型边缘检测方法

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104952073B (zh) * 2015-06-15 2017-12-15 上海交通大学 基于深度学习的镜头边缘检测方法
CN105069807B (zh) * 2015-08-28 2018-03-23 西安工程大学 一种基于图像处理的冲压工件缺陷检测方法
CN107610140A (zh) * 2017-08-07 2018-01-19 中国科学院自动化研究所 基于深度融合修正网络的精细边缘检测方法、装置

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018023734A1 (fr) * 2016-08-05 2018-02-08 深圳大学 Procédé de test de signification pour image 3d
CN108710919A (zh) * 2018-05-25 2018-10-26 东南大学 一种基于多尺度特征融合深度学习的裂缝自动化勾画方法
CN109741351A (zh) * 2018-12-12 2019-05-10 中国科学院深圳先进技术研究院 一种基于深度学习的类别敏感型边缘检测方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
YU, ZHIDING ET AL.: "CASENet: Deep Category-Aware Semantic Edge Detection", 2017 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 31 December 2017 (2017-12-31), XP033249517, DOI: 20200303111926X *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115588024A (zh) * 2022-11-25 2023-01-10 东莞市兆丰精密仪器有限公司 一种基于人工智能的复杂工业影像边缘提取方法及装置

Also Published As

Publication number Publication date
CN109741351A (zh) 2019-05-10

Similar Documents

Publication Publication Date Title
CN110543837B (zh) 一种基于潜在目标点的可见光机场飞机检测方法
CN110348319B (zh) 一种基于人脸深度信息和边缘图像融合的人脸防伪方法
US10846566B2 (en) Method and system for multi-scale cell image segmentation using multiple parallel convolutional neural networks
CN110781897B (zh) 一种基于深度学习的语义边缘检测方法
CN106408030B (zh) 基于中层语义属性和卷积神经网络的sar图像分类方法
Zhang et al. Infrared image segmentation for photovoltaic panels based on Res-UNet
CN104361313B (zh) 一种基于多核学习异构特征融合的手势识别方法
CN105138998B (zh) 基于视角自适应子空间学习算法的行人重识别方法及系统
WO2020119624A1 (fr) Procédé de détection de bord sensible à la classe basé sur l'apprentissage profond
CN109740686A (zh) 一种基于区域池化和特征融合的深度学习图像多标记分类方法
CN106126585B (zh) 基于质量分级与感知哈希特征组合的无人机图像检索方法
Zang et al. Traffic sign detection based on cascaded convolutional neural networks
CN104036284A (zh) 基于Adaboost算法的多尺度行人检测方法
CN108734200B (zh) 基于bing特征的人体目标视觉检测方法和装置
CN103106265A (zh) 相似图像分类方法及系统
CN112926652B (zh) 一种基于深度学习的鱼类细粒度图像识别方法
CN110008899B (zh) 一种可见光遥感图像候选目标提取与分类方法
CN103186776A (zh) 基于多特征和深度信息的人体检测方法
Intwala et al. Indian sign language converter using convolutional neural networks
CN111026898A (zh) 一种基于交叉空间池化策略的弱监督图像情感分类与定位方法
CN114492634B (zh) 一种细粒度装备图片分类识别方法及系统
Li et al. HEp-2 specimen classification via deep CNNs and pattern histogram
Khellal et al. Pedestrian classification and detection in far infrared images
CN106203448A (zh) 一种基于非线性尺度空间的场景分类方法
CN113487610B (zh) 疱疹图像识别方法、装置、计算机设备和存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19897270

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 03.11.2021)

122 Ep: pct application non-entry in european phase

Ref document number: 19897270

Country of ref document: EP

Kind code of ref document: A1