CN114419078A - Surface defect region segmentation method and device based on convolutional neural network - Google Patents

Surface defect region segmentation method and device based on convolutional neural network Download PDF

Info

Publication number
CN114419078A
CN114419078A CN202210335463.8A CN202210335463A CN114419078A CN 114419078 A CN114419078 A CN 114419078A CN 202210335463 A CN202210335463 A CN 202210335463A CN 114419078 A CN114419078 A CN 114419078A
Authority
CN
China
Prior art keywords
information
surface defect
defect region
image
neural network
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202210335463.8A
Other languages
Chinese (zh)
Other versions
CN114419078B (en
Inventor
弭宝瞳
周展
王涛
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Jushi Intelligent Technology Co ltd
Original Assignee
Beijing Jushi Intelligent Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Jushi Intelligent Technology Co ltd filed Critical Beijing Jushi Intelligent Technology Co ltd
Priority to CN202210335463.8A priority Critical patent/CN114419078B/en
Publication of CN114419078A publication Critical patent/CN114419078A/en
Application granted granted Critical
Publication of CN114419078B publication Critical patent/CN114419078B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/11Region-based segmentation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20016Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20081Training; Learning
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20084Artificial neural networks [ANN]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30108Industrial image inspection
    • G06T2207/30136Metal

Abstract

The method comprises the steps of obtaining a source software code, constructing a code attribute graph according to the source software code, inputting the code attribute graph into a preset source code defect detection model based on a graph neural network, presetting the source code defect detection model based on the graph neural network for generating a self-adaptive receiving path, and outputting a detection result according to the self-adaptive receiving path, so that the influence of irrelevant code information can be reduced, and the efficiency of code vulnerability detection is improved.

Description

Surface defect region segmentation method and device based on convolutional neural network
Technical Field
The application belongs to the technical field of defect detection, and particularly relates to a surface defect region segmentation method and device based on a convolutional neural network.
Background
In the industrial production process, the surface defect detection of related products is required. For example, in the production and processing process of steel, the defects such as holes, scratches, inclusions, scratches, roll marks and the like are easily generated due to the influence of a plurality of factors such as raw materials, rolling equipment, operating techniques of workers and the like, and the appearance of the steel is affected and the properties such as corrosion resistance, wear resistance, fatigue strength and the like are also affected due to the existence of the defects, so that the quality of the steel is seriously reduced. With the rapid development of deep learning technology, the workpiece surface defect region segmentation method based on deep learning has achieved tremendous achievement in recent years. The existing workpiece surface defect region segmentation method based on deep learning carries out defect region segmentation based on multi-scale pooling and convolution operation, but because the method only focuses on aggregation of global context information of a defect region in an image, when the image is complex, because scene information included by the context information is incomplete, the workpiece surface defect region segmentation is often inaccurate.
Disclosure of Invention
In order to overcome the problem that the existing workpiece surface defect region segmentation method based on deep learning only focuses on aggregation of global context information of a defect region in an image to a certain extent, and when the image is complex, the segmentation of the workpiece surface defect region is inaccurate due to incomplete scene information included by the context information, the application provides a surface defect region segmentation method and a surface defect region segmentation device based on a convolutional neural network.
In a first aspect, the present application provides a surface defect region segmentation method based on a convolutional neural network, including:
extracting feature maps of different scales corresponding to the input image;
acquiring image structure information and scene information of different levels of the input image according to the feature maps of different scales;
and outputting a surface defect region segmentation result according to the image structure information and the scene information of different layers.
Further, the extracting feature maps of different scales corresponding to the input image includes:
inputting the input image into a pre-trained ResNet network to extract bottom layer feature maps with different scales;
inputting the bottom-layer feature maps of different scales into a channel feature pyramid, wherein the channel feature pyramid is used for N ^ or N ^ of the bottom-layer feature maps of different scales4 channels utilize 3
Figure 727215DEST_PATH_IMAGE001
3 convolution operations for processing, N/4 channels using 5
Figure 442361DEST_PATH_IMAGE001
5 convolution operation processing, N/2 channels utilizing 7
Figure 860704DEST_PATH_IMAGE001
7, carrying out convolution operation;
and performing jumping connection splicing on the feature map subjected to channel feature pyramid processing to obtain a multi-scale feature map.
Further, the acquiring image structure information and scene information of different levels of the input image according to the feature maps of different scales includes:
inputting the feature diagrams of different scales into a dynamic context polymerization module to obtain mask diagrams of different structural layers;
performing matrix transposition operation on the mask image;
multiplying the feature graph with the mask graph after the matrix transposition operation to obtain a context information vector;
remapping the context information vector to the mask image to obtain refined image interpretation information;
and obtaining the characteristic vectors of the image structure information and the scene information according to the refined image interpretation information.
Further, the inputting the feature maps of different scales into the dynamic context aggregation module to obtain mask maps of different structural layers includes:
1 is carried out on the feature maps with different scales
Figure 28511DEST_PATH_IMAGE001
1, performing convolution operation to obtain an intermediate characteristic diagram;
inputting the intermediate feature map into a dynamic context aggregation module;
if the number of layers of the dynamic context aggregation module is more than 1, outputting the dynamic context aggregation module of the previous layerSplicing the feature graph with the output features of the current layer, and performing 3 on the spliced feature graph
Figure 823292DEST_PATH_IMAGE001
3 convolution, 1
Figure 392945DEST_PATH_IMAGE001
And (3) carrying out convolution and Softmax function processing to obtain mask graphs of different structural layers of the dynamic context aggregation module.
Further, the inputting the feature maps of different scales into the dynamic context aggregation module to obtain mask maps of different structural layers further includes:
if the number of layers of the dynamic context aggregation module is 1, performing 3 on the intermediate feature map
Figure 908153DEST_PATH_IMAGE001
3 convolution, 1
Figure 891153DEST_PATH_IMAGE001
And (4) processing the convolution 1 and the Softmax function to obtain a mask image of the dynamic context aggregation module.
Further, the context information vector includes context information corresponding to a plurality of categories, and the remapping the context information vector to the mask map to obtain refined image interpretation information includes:
and calculating the average value of the context information corresponding to a plurality of categories in the context information vector to obtain refined image interpretation information.
Further, the outputting a surface defect region segmentation result according to the image structure information and the scene information of the different layers includes:
repeatedly acquiring a characteristic diagram corresponding to a plurality of refined image interpretation information in a recursive mode;
and performing up-sampling operation on the feature maps corresponding to the plurality of refined image interpretation information to obtain a surface defect area segmentation result.
In a second aspect, the present application provides a surface defect region segmentation apparatus based on a convolutional neural network, including:
the extraction module is used for extracting feature maps with different scales corresponding to the input image;
the acquisition module is used for acquiring image structure information and scene information of different levels of the input image according to the feature maps with different scales;
and the output module is used for outputting the segmentation result of the surface defect region according to the image structure information and the scene information of different layers.
The technical scheme provided by the embodiment of the application can have the following beneficial effects:
according to the surface defect region segmentation method and device based on the convolutional neural network, the feature maps of different scales corresponding to the input image are extracted, the image structure information and the scene information of different levels of the input image are obtained according to the feature maps of different scales, the segmentation result of the surface defect region is output according to the image structure information and the scene information of different levels, the hierarchical dynamic aggregation of context information is achieved on the premise that no additional supervision is introduced, and the accuracy of workpiece surface defect region segmentation is improved by obtaining the image structure information and the scene information of different levels of the input image.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application.
Fig. 1 is a flowchart of a surface defect region segmentation method based on a convolutional neural network according to an embodiment of the present application.
Fig. 2 is a flowchart of a surface defect region segmentation method based on a convolutional neural network according to another embodiment of the present application.
Fig. 3 is a flowchart of another surface defect region segmentation method based on a convolutional neural network according to an embodiment of the present application.
Fig. 4 is a flowchart of another surface defect region segmentation method based on a convolutional neural network according to an embodiment of the present application.
Fig. 5 is a flowchart of another surface defect region segmentation method based on a convolutional neural network according to an embodiment of the present application.
Fig. 6 is a functional block diagram of a surface defect area segmentation apparatus based on a convolutional neural network according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the technical solutions of the present application will be described in detail below. It is to be understood that the embodiments described are only a few embodiments of the present application and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the examples given herein without making any creative effort, shall fall within the protection scope of the present application.
Fig. 1 is a flowchart of a surface defect region segmentation method based on a convolutional neural network according to an embodiment of the present application, and as shown in fig. 1, the surface defect region segmentation method based on a convolutional neural network includes:
s11: extracting feature maps of different scales corresponding to the input image;
s12: acquiring image structure information and scene information of different levels of an input image according to feature maps of different scales;
s13: and outputting a surface defect region segmentation result according to the image structure information and the scene information of different layers.
The traditional workpiece surface defect region segmentation method based on deep learning carries out defect region segmentation based on multi-scale pooling and convolution operation, but because the method only focuses on aggregation of global context information of a defect region in an image, when the image is complex, because scene information included by the context information is incomplete, the workpiece surface defect region segmentation is often inaccurate.
In the embodiment, the feature maps of different scales corresponding to the input image are extracted, the image structure information and the scene information of different levels of the input image are obtained according to the feature maps of different scales, the segmentation result of the surface defect region is output according to the image structure information and the scene information of different levels, the hierarchical dynamic aggregation of context information is realized on the premise of not introducing extra supervision, and the segmentation accuracy of the workpiece surface defect region is improved by obtaining the image structure information and the scene information of different levels of the input image.
Fig. 2 is a flowchart of a surface defect region segmentation method based on a convolutional neural network according to another embodiment of the present disclosure, and as shown in fig. 2, the surface defect region segmentation method based on a convolutional neural network includes:
s201: inputting an input image into a pre-trained ResNet network to extract bottom layer feature maps with different scales;
for the input image, extracting feature maps with different scales by using a pre-trained ResNet network, and outputting a bottom layer feature map
Figure 161728DEST_PATH_IMAGE002
S202: inputting the bottom layer characteristic graphs of different scales into a channel characteristic pyramid, wherein the channel characteristic pyramid is used for utilizing 3 channels of N/4 bottom layer characteristic graphs of different scales
Figure 648204DEST_PATH_IMAGE001
3 convolution operations for processing, N/4 channels using 5
Figure 611612DEST_PATH_IMAGE001
5 convolution operation processing, N/2 channels utilizing 7
Figure 754012DEST_PATH_IMAGE001
7, carrying out convolution operation;
s203: and performing jump connection splicing on the feature map subjected to channel feature pyramid processing to obtain a multi-scale feature map.
For the underlying feature map
Figure 890595DEST_PATH_IMAGE003
Wherein R isThe figure of the figure, C is the number of channels, H is the height of the image, W is the width of the image, as the input of the channel feature pyramid, the flow of the channel feature pyramid is shown in figure 3, the bottom layer feature graph is
Figure 169261DEST_PATH_IMAGE004
N/4 channels of (3)
Figure 303570DEST_PATH_IMAGE001
3 convolution operations followed by N/4 channels with 5
Figure 933266DEST_PATH_IMAGE001
5 convolution operation process, utilizing 7 for the remaining N/2 channels
Figure 607960DEST_PATH_IMAGE001
7, performing convolution operation, splicing the processed characteristic graphs by jump connection to obtain final output characteristic graphs
Figure 69029DEST_PATH_IMAGE005
And through multi-channel characteristic pyramid processing, on the premise of controlling the calculated amount, a multi-scale characteristic diagram is effectively extracted.
The method and the device for capturing the image structure information and the scene structure hierarchy learn the multi-scale pyramid representation of the hierarchical context of the input image by designing the network.
S204: inputting the feature diagrams of different scales into a dynamic context polymerization module to obtain mask diagrams of different structural layers;
in some embodiments, inputting feature maps of different scales into the dynamic context aggregation module to obtain mask maps of different structural layers includes:
1 for feature maps of different scales
Figure 374239DEST_PATH_IMAGE001
1, performing convolution operation to obtain an intermediate characteristic diagram;
inputting the intermediate feature map into a dynamic context aggregation module; the dynamic context aggregation module flow is shown in fig. 4.
If the number of layers of the dynamic context aggregation module is more than 1, splicing the output characteristic diagram of the previous layer of dynamic context aggregation module with the output characteristic of the current layer, and performing 3 on the spliced characteristic diagram
Figure 225652DEST_PATH_IMAGE001
3 convolution, 1
Figure 704038DEST_PATH_IMAGE001
And (3) carrying out convolution and Softmax function processing to obtain mask graphs of different structural layers of the dynamic context aggregation module.
Or, if the number of layers of the dynamic context aggregation module is 1, performing 3 on the intermediate feature map
Figure 222875DEST_PATH_IMAGE001
3 convolution, 1
Figure 495724DEST_PATH_IMAGE001
And (4) processing the convolution 1 and the Softmax function to obtain a mask image of the dynamic context aggregation module.
The method and the device utilize a plurality of layered dynamic context aggregation modules to extract scene information of different layers of an input image in a recursive mode, and the processing flow of the layered dynamic context aggregation modules is shown in fig. 4. Feature maps for backbone network output
Figure 100012DEST_PATH_IMAGE006
First of all by using
Figure 116510DEST_PATH_IMAGE007
Convolution reduces the number of channels to obtain a characteristic diagram
Figure 489853DEST_PATH_IMAGE008
. Then the feature map is processed
Figure 757823DEST_PATH_IMAGE006
And
Figure 849407DEST_PATH_IMAGE009
as input to the first level dynamic context aggregation module. For a dynamic context aggregation module with n =1, first the feature map is compared
Figure 669595DEST_PATH_IMAGE009
Make use of successively
Figure 897445DEST_PATH_IMAGE010
Convolution and,
Figure 777677DEST_PATH_IMAGE011
The convolution is processed with the Softmax function. For the dynamic context aggregation module with n > 1, obtaining the output characteristic diagram of the dynamic context aggregation module at the n-1 layer
Figure 90977DEST_PATH_IMAGE012
And characteristic diagram
Figure 449278DEST_PATH_IMAGE009
After splicing, reuse
Figure 797213DEST_PATH_IMAGE010
Convolution and,
Figure 520450DEST_PATH_IMAGE011
The convolution is processed with the Softmax function. Finally obtaining the mask graph of the layer
Figure 321047DEST_PATH_IMAGE013
Wherein
Figure 420721DEST_PATH_IMAGE014
Representing the number of classes that need to be predicted.
S205: performing matrix transposition operation on the mask image;
s206: multiplying the feature graph and the mask graph after the matrix transposition operation to obtain a context information vector;
mask map to be obtained
Figure 623163DEST_PATH_IMAGE013
Performing matrix transposition operation and on the feature map
Figure 782880DEST_PATH_IMAGE006
Performing matrix multiplication to obtain context information vector
Figure 133090DEST_PATH_IMAGE015
The calculation formula is as follows:
Figure 770876DEST_PATH_IMAGE016
s207: remapping the context information vector to a mask image to obtain refined image interpretation information;
(Vector)
Figure 890142DEST_PATH_IMAGE017
comprises a
Figure 220760DEST_PATH_IMAGE018
Calculating a vector from the context information corresponding to each category
Figure 58266DEST_PATH_IMAGE017
Is obtained as an average of
Figure 499743DEST_PATH_IMAGE019
Figure 473515DEST_PATH_IMAGE020
(ii) a Where i represents the ith object class, j represents j as a pixel in the input image,
Figure 963316DEST_PATH_IMAGE019
represents the refined image interpretation information of the nth layer. Then combining the n vectors to obtain the output context information vector
Figure 288118DEST_PATH_IMAGE021
In some embodiments, the context information vector includes context information corresponding to a plurality of categories, and remapping the context information vector to the mask map results in refined image interpretation information, including:
and calculating the average value of the context information corresponding to a plurality of categories in the context information vector to obtain refined image interpretation information.
And obtaining the characteristic vectors of the image structure information and the scene information according to the refined image interpretation information.
Remapping the obtained context information vector to a mask map
Figure 267707DEST_PATH_IMAGE022
In (1), obtaining an output feature vector
Figure 361565DEST_PATH_IMAGE023
Feature vector
Figure 33985DEST_PATH_IMAGE024
Namely the output of the dynamic context aggregation module of the nth layer.
S208: repeatedly acquiring a characteristic diagram corresponding to a plurality of refined image interpretation information in a recursive mode;
s209: the feature maps corresponding to the plurality of refined image interpretation information are subjected to upsampling operation to obtain a surface defect area segmentation result, and the whole process is shown in fig. 5.
The existing refined image interpretation method based on multi-scale pooling and convolution operation focuses on aggregation of global context information in an image, structural hierarchy and objects of a scene are omitted, the hierarchical structure of the image is composed of the scene and a series of objects, and therefore the hierarchical structure of the image is beneficial to understanding context information containing complex scene images and refined image interpretation.
In the embodiment, the hierarchical dynamic aggregation of the context information is realized by using the plurality of dynamic context aggregation modules on the premise of not introducing additional supervision, and the accuracy of workpiece surface defect area segmentation is improved by performing recursive refined image interpretation on the input image.
An embodiment of the present invention provides a surface defect region segmentation apparatus based on a convolutional neural network, and as shown in a functional structure diagram of fig. 6, the surface defect region segmentation apparatus based on the convolutional neural network includes:
the extraction module 61 is used for extracting feature maps of different scales corresponding to the input image;
an obtaining module 62, configured to obtain image structure information and scene information of different levels of the input image according to feature maps of different scales;
and the output module 63 is configured to output the surface defect region segmentation result according to the image structure information and the scene information of different layers.
In the embodiment, feature maps of different scales corresponding to the input image are extracted through an extraction module; the acquisition module acquires image structure information and scene information of different levels of the input image according to the feature maps of different scales; the output module outputs the segmentation result of the surface defect region according to the image structure information and the scene information of different layers, can realize the layered dynamic aggregation of context information on the premise of not introducing additional supervision, and improves the segmentation accuracy of the surface defect region of the workpiece by acquiring the image structure information and the scene information of different layers of the input image.
It is understood that the same or similar parts in the above embodiments may be mutually referred to, and the same or similar parts in other embodiments may be referred to for the content which is not described in detail in some embodiments.
It should be noted that, in the description of the present application, the terms "first", "second", etc. are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. Further, in the description of the present application, the meaning of "a plurality" means at least two unless otherwise specified.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and the scope of the preferred embodiments of the present application includes other implementations in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present application.
It should be understood that portions of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
It will be understood by those skilled in the art that all or part of the steps carried by the method for implementing the above embodiments may be implemented by hardware related to instructions of a program, which may be stored in a computer readable storage medium, and when the program is executed, the program includes one or a combination of the steps of the method embodiments.
In addition, functional units in the embodiments of the present application may be integrated into one processing module, or each unit may exist alone physically, or two or more units are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional component mode. The integrated module, if implemented in the form of a software functional component and sold or used as a stand-alone product, may also be stored in a computer readable storage medium.
The storage medium mentioned above may be a read-only memory, a magnetic or optical disk, etc.
In the description herein, reference to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the application. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
Although embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present application, and that variations, modifications, substitutions and alterations may be made to the above embodiments by those of ordinary skill in the art within the scope of the present application.
It should be noted that the present invention is not limited to the above-mentioned preferred embodiments, and those skilled in the art can obtain other products in various forms without departing from the spirit of the present invention, but any changes in shape or structure can be made within the scope of the present invention with the same or similar technical solutions as those of the present invention.

Claims (8)

1. A surface defect region segmentation method based on a convolutional neural network is characterized by comprising the following steps:
extracting feature maps of different scales corresponding to the input image;
acquiring image structure information and scene information of different levels of the input image according to the feature maps of different scales;
and outputting a surface defect region segmentation result according to the image structure information and the scene information of different layers.
2. The surface defect region segmentation method based on the convolutional neural network as claimed in claim 1, wherein the extracting feature maps of different scales corresponding to the input image comprises:
inputting the input image into a pre-trained ResNet network to extract bottom layer feature maps with different scales;
inputting the bottom layer feature maps of different scales into a channel feature pyramid, wherein the channel feature pyramid is used for utilizing 3 channels of N/4 bottom layer feature maps of different scales
Figure 208160DEST_PATH_IMAGE001
3 convolution operations for processing, N/4 channels using 5
Figure 319335DEST_PATH_IMAGE001
5 convolution operation processing, N/2 channels utilizing 7
Figure 205383DEST_PATH_IMAGE001
7, carrying out convolution operation;
and performing jumping connection splicing on the feature map subjected to channel feature pyramid processing to obtain a multi-scale feature map.
3. The surface defect region segmentation method based on the convolutional neural network as claimed in claim 1, wherein the obtaining of image structure information and scene information of different levels of the input image according to the feature maps of different scales comprises:
inputting the feature diagrams of different scales into a dynamic context polymerization module to obtain mask diagrams of different structural layers;
performing matrix transposition operation on the mask image;
multiplying the feature graph with the mask graph after the matrix transposition operation to obtain a context information vector;
remapping the context information vector to the mask image to obtain refined image interpretation information;
and obtaining the characteristic vectors of the image structure information and the scene information according to the refined image interpretation information.
4. The surface defect region segmentation method based on the convolutional neural network as claimed in claim 3, wherein the inputting the feature maps of different scales into the dynamic context aggregation module to obtain the mask maps of different structural layers comprises:
1 is carried out on the feature maps with different scales
Figure 111022DEST_PATH_IMAGE001
1, performing convolution operation to obtain an intermediate characteristic diagram;
inputting the intermediate feature map into a dynamic context aggregation module;
if the number of layers of the dynamic context aggregation module is more than 1, splicing the output characteristic diagram of the previous layer of dynamic context aggregation module with the output characteristic of the current layer, and performing 3 on the spliced characteristic diagram
Figure 82520DEST_PATH_IMAGE001
3 convolution, 1
Figure 466228DEST_PATH_IMAGE001
And (3) carrying out convolution and Softmax function processing to obtain mask graphs of different structural layers of the dynamic context aggregation module.
5. The surface defect region segmentation method based on the convolutional neural network as claimed in claim 4, wherein the inputting the feature maps of different scales into the dynamic context aggregation module to obtain mask maps of different structural layers further comprises:
if the number of layers of the dynamic context aggregation module is 1, performing 3 on the intermediate feature map
Figure 472361DEST_PATH_IMAGE001
3 convolution, 1
Figure 498303DEST_PATH_IMAGE001
And (4) processing the convolution 1 and the Softmax function to obtain a mask image of the dynamic context aggregation module.
6. The surface defect region segmentation method based on convolutional neural network as claimed in claim 4, wherein the context information vector comprises context information corresponding to a plurality of classes, and the remapping the context information vector to the mask map yields refined image interpretation information, comprising:
and calculating the average value of the context information corresponding to a plurality of categories in the context information vector to obtain refined image interpretation information.
7. The surface defect region segmentation method based on the convolutional neural network as claimed in claim 6, wherein the outputting the surface defect region segmentation result according to the different levels of image structure information and scene information comprises:
repeatedly acquiring a characteristic diagram corresponding to a plurality of refined image interpretation information in a recursive mode;
and performing up-sampling operation on the feature maps corresponding to the plurality of refined image interpretation information to obtain a surface defect area segmentation result.
8. A surface defect region segmentation apparatus based on a convolutional neural network, comprising:
the extraction module is used for extracting feature maps with different scales corresponding to the input image;
the acquisition module is used for acquiring image structure information and scene information of different levels of the input image according to the feature maps with different scales;
and the output module is used for outputting the segmentation result of the surface defect region according to the image structure information and the scene information of different layers.
CN202210335463.8A 2022-04-01 2022-04-01 Surface defect region segmentation method and device based on convolutional neural network Active CN114419078B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210335463.8A CN114419078B (en) 2022-04-01 2022-04-01 Surface defect region segmentation method and device based on convolutional neural network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210335463.8A CN114419078B (en) 2022-04-01 2022-04-01 Surface defect region segmentation method and device based on convolutional neural network

Publications (2)

Publication Number Publication Date
CN114419078A true CN114419078A (en) 2022-04-29
CN114419078B CN114419078B (en) 2022-06-24

Family

ID=81263613

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210335463.8A Active CN114419078B (en) 2022-04-01 2022-04-01 Surface defect region segmentation method and device based on convolutional neural network

Country Status (1)

Country Link
CN (1) CN114419078B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2020190485A (en) * 2019-05-22 2020-11-26 日本製鉄株式会社 Identification model generation device, identification model generation method and identification model generation program, and steel defect determination device, steel defect determination method and steel defect determination program

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109584246A (en) * 2018-11-16 2019-04-05 成都信息工程大学 Based on the pyramidal DCM cardiac muscle diagnosis and treatment irradiation image dividing method of Analysis On Multi-scale Features
CN113421267A (en) * 2021-05-07 2021-09-21 江苏大学 Point cloud semantic and instance joint segmentation method and system based on improved PointConv
US20210342579A1 (en) * 2018-09-05 2021-11-04 Daimler Ag Method for identifying a hand pose in a vehicle
CN113658180A (en) * 2021-10-20 2021-11-16 北京矩视智能科技有限公司 Surface defect region segmentation method and device based on spatial context guidance
CN113887613A (en) * 2021-09-29 2022-01-04 平安银行股份有限公司 Deep learning method, device and equipment based on attention mechanism and storage medium

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20210342579A1 (en) * 2018-09-05 2021-11-04 Daimler Ag Method for identifying a hand pose in a vehicle
CN109584246A (en) * 2018-11-16 2019-04-05 成都信息工程大学 Based on the pyramidal DCM cardiac muscle diagnosis and treatment irradiation image dividing method of Analysis On Multi-scale Features
CN113421267A (en) * 2021-05-07 2021-09-21 江苏大学 Point cloud semantic and instance joint segmentation method and system based on improved PointConv
CN113887613A (en) * 2021-09-29 2022-01-04 平安银行股份有限公司 Deep learning method, device and equipment based on attention mechanism and storage medium
CN113658180A (en) * 2021-10-20 2021-11-16 北京矩视智能科技有限公司 Surface defect region segmentation method and device based on spatial context guidance

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
张旺: "基于深度学习的多尺度边缘检测算法研究", 《中国优秀博硕士学位论文全文数据库(硕士)信息科技辑》, no. 1, 15 January 2022 (2022-01-15) *
陈裕如等: "基于自适应像素级注意力模型的场景深度估计", 《应用光学》, no. 03, 15 May 2020 (2020-05-15) *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2020190485A (en) * 2019-05-22 2020-11-26 日本製鉄株式会社 Identification model generation device, identification model generation method and identification model generation program, and steel defect determination device, steel defect determination method and steel defect determination program

Also Published As

Publication number Publication date
CN114419078B (en) 2022-06-24

Similar Documents

Publication Publication Date Title
CN108647585B (en) Traffic identifier detection method based on multi-scale circulation attention network
CN111179229B (en) Industrial CT defect detection method based on deep learning
CN108830285B (en) Target detection method for reinforcement learning based on fast-RCNN
CN111126472A (en) Improved target detection method based on SSD
JP2024509411A (en) Defect detection method, device and system
CN111461212B (en) Compression method for point cloud target detection model
CN114627052A (en) Infrared image air leakage and liquid leakage detection method and system based on deep learning
CN111914843B (en) Character detection method, system, equipment and storage medium
CN113313706B (en) Power equipment defect image detection method based on detection reference point offset analysis
CN114565770B (en) Image segmentation method and system based on edge auxiliary calculation and mask attention
CN114972759A (en) Remote sensing image semantic segmentation method based on hierarchical contour cost function
CN115131797A (en) Scene text detection method based on feature enhancement pyramid network
CN114419078B (en) Surface defect region segmentation method and device based on convolutional neural network
CN111739037A (en) Semantic segmentation method for indoor scene RGB-D image
CN112070040A (en) Text line detection method for video subtitles
CN113963333B (en) Traffic sign board detection method based on improved YOLOF model
CN115147418A (en) Compression training method and device for defect detection model
CN114494823A (en) Commodity identification, detection and counting method and system in retail scene
CN114092467A (en) Scratch detection method and system based on lightweight convolutional neural network
CN112200766A (en) Industrial product surface defect detection method based on area-associated neural network
CN114419036B (en) Surface defect region segmentation method and device based on boundary information fusion
CN114998222A (en) Automobile differential shell surface detection method, electronic equipment and medium
CN114332473A (en) Object detection method, object detection device, computer equipment, storage medium and program product
CN114882298B (en) Optimization method and device for confrontation complementary learning model
CN116310613A (en) Industrial product surface defect detection model training method, detection method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant