CN114202508A - A detection method and system adapting to the highlighting of surface defects of multi-variety ceramic tiles - Google Patents
A detection method and system adapting to the highlighting of surface defects of multi-variety ceramic tiles Download PDFInfo
- Publication number
- CN114202508A CN114202508A CN202111301767.4A CN202111301767A CN114202508A CN 114202508 A CN114202508 A CN 114202508A CN 202111301767 A CN202111301767 A CN 202111301767A CN 114202508 A CN114202508 A CN 114202508A
- Authority
- CN
- China
- Prior art keywords
- module
- image
- dbl
- output
- input end
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000007547 defect Effects 0.000 title claims abstract description 57
- 238000001514 detection method Methods 0.000 title claims abstract description 36
- 239000000919 ceramic Substances 0.000 title claims abstract description 32
- 238000000034 method Methods 0.000 claims abstract description 21
- 238000003062 neural network model Methods 0.000 claims abstract description 20
- 238000007781 pre-processing Methods 0.000 claims abstract description 15
- 230000006870 function Effects 0.000 claims abstract description 8
- 238000012549 training Methods 0.000 claims description 21
- 238000005070 sampling Methods 0.000 claims description 16
- 238000000605 extraction Methods 0.000 claims description 12
- 238000012795 verification Methods 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 8
- 230000004913 activation Effects 0.000 claims description 6
- 238000010606 normalization Methods 0.000 claims description 6
- 238000012545 processing Methods 0.000 claims description 5
- 238000001914 filtration Methods 0.000 claims description 4
- 238000011478 gradient descent method Methods 0.000 claims description 4
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 claims description 3
- 230000009466 transformation Effects 0.000 claims description 3
- 230000009977 dual effect Effects 0.000 abstract description 4
- 238000013527 convolutional neural network Methods 0.000 abstract description 2
- 238000013135 deep learning Methods 0.000 abstract description 2
- 238000007499 fusion processing Methods 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 239000002253 acid Substances 0.000 description 1
- 239000003513 alkali Substances 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 238000005034 decoration Methods 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000000227 grinding Methods 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 229910044991 metal oxide Inorganic materials 0.000 description 1
- 150000004706 metal oxides Chemical class 0.000 description 1
- 238000002156 mixing Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 229910052573 porcelain Inorganic materials 0.000 description 1
- 238000003825 pressing Methods 0.000 description 1
- 239000003870 refractory metal Substances 0.000 description 1
- 238000005245 sintering Methods 0.000 description 1
- 239000004575 stone Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
- G06T7/0008—Industrial image inspection checking presence/absence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
本发明涉及一种适应多品种瓷砖表面缺陷凸显的检测方法，包括以下，获取第一相机所拍摄的第一图像，获取第二相机所拍摄的第二图像；对第一图像以及第二图像分别进行图像预处理操作后得到定位后的第一图像以及第二图像；将定位后的图像输入预建立的神经网络模型，输出三个维度的特征图，每个特征图中包含每一个锚框的偏移信息与对应预测的置信度，选取置信度大于第一阈值的边界框作为预测缺陷的最终输出。利用双相机获取双图像，双图像的融合处理适应多种复杂纹理瓷砖，凸显出瓷砖表面的不明显缺陷。利用卷积神经网络训练样本，使网络获得缺陷识别功能，实现瓷砖缺陷高精度检测。利用深度学习的方法进行缺陷检测，减少人工劳动成本，提高检测效率。
The invention relates to a detection method adapted to the highlighting of surface defects of various types of ceramic tiles, comprising the following steps: acquiring a first image shot by a first camera, acquiring a second image shot by a second camera; After the image preprocessing operation, the positioned first image and the second image are obtained; the positioned image is input into the pre-established neural network model, and a three-dimensional feature map is output, and each feature map contains the data of each anchor frame. The offset information and the confidence of the corresponding prediction, and the bounding box with the confidence greater than the first threshold is selected as the final output of the prediction defect. Using dual cameras to obtain dual images, the fusion processing of dual images is suitable for a variety of complex textured tiles, highlighting the inconspicuous defects on the surface of the tiles. The convolutional neural network is used to train samples, so that the network can obtain the function of defect recognition and realize high-precision detection of tile defects. Using deep learning methods to detect defects, reduce labor costs and improve detection efficiency.
Description
Technical Field
The invention relates to the technical field of machine vision, in particular to a detection method and a detection system suitable for highlighting surface defects of various tiles.
Background
The ceramic tile is made of refractory metal oxide and semimetal oxide through the processes of grinding, mixing, pressing, glazing and sintering, and is a kind of acid and alkali resistant building or decoration material made of porcelain or stone, etc., which is generally called as ceramic tile.
When ceramic tiles are produced in batches, the detection of surface defects of various ceramic tiles is often required to ensure that products of a batch of ceramic tiles meet acceptance standards.
In the prior art, most of tiles in batches are detected by experienced workers in a manual detection mode, and the method is easy to solve the problems of low efficiency, high cost and the like existing in a method for detecting various defects.
Disclosure of Invention
The invention aims to at least solve one of the defects of the prior art and provides a detection method and a detection system suitable for highlighting the surface defects of various ceramic tiles.
In order to achieve the purpose, the invention adopts the following technical scheme:
specifically, the detection method suitable for highlighting the surface defects of various tiles comprises the following steps:
acquiring a first image shot by a first camera and a second image shot by a second camera, wherein the first camera is a color line camera and is arranged perpendicular to the shooting plane, and the second camera is a black-and-white line camera and is arranged obliquely to the shooting plane;
respectively carrying out image preprocessing operation on the first image and the second image to obtain a positioned first image and a positioned second image;
inputting the positioned first image and the second image into a pre-established neural network model, outputting feature maps of three dimensions, wherein each feature map comprises the offset information of each anchor frame and the confidence coefficient of corresponding prediction, and selecting a boundary frame with the confidence coefficient larger than a first threshold value as the final output of the predicted defect.
Further, specifically, the image preprocessing operation includes the following,
and respectively carrying out filtering and denoising processing on the first image and the second image, and then carrying out Harris corner detection to position the tile images in the first image and the second image to obtain the positioned first image and second image.
Further, the method may further comprise,
before inputting the first and second positioned images into the pre-established neural network model, the sizes of the first and second positioned images are uniformly adjusted to be integral multiples of 416x416 pixels, and then the first and second positioned images are cut into a plurality of small-size images of 416x416 pixels.
Further, specifically, the pre-established neural network model includes,
a feature extraction module, configured to extract feature information of the first and second images after positioning, where an output end of the feature extraction module is connected to an input end of a first RESn module, an output end of the first RESn module is connected to an input end of a second RESn module and an input end of a second concat module, an output end of the second RESn module is connected to an input end of a first DBL module and an input end of the first concat module, an output end of the first DBL module is connected to input ends of a first output convolution layer conv and a second DBL module respectively for up-sampling, an output end of the second DBL module is connected to an input end of the first concat module, an output end of the first concat module is connected to an input end of a third DBL module, and an output end of the third DBL module is connected to an input end of the second output convolution layer conv and an input end of a fourth DBL module respectively for up-sampling, the output end of the fourth DBL module is connected with the input end of the first concat module, the output end of the first concat module is connected with the input end of a fifth DBL module, the output end of the fifth DBL module is connected with the input end of a third output convolution layer conv, and finally the output ends of the first output convolution layer conv, the second output convolution layer conv and the third output convolution layer conv respectively output characteristic graphs with three dimensions;
the DBL module comprises a convolution layer, a normalization BN layer and a leak relu activation layer, the concat module is formed by splicing feature graphs, RESn is a jump layer structure, n represents a number and represents the number of res _ units contained in the res _ block, the res _ unit structure is formed by connecting two DBL modules and jump layers, the former DBL module adopts a convolution kernel of 1x1, and the latter DBL module adopts a convolution kernel of 3x 3.
Further, specifically, the neural network model is obtained by training in the following way,
the loss function is the same as that of a YOLOv3 network, an Adam gradient descent method is used for training, the learning rate is set to be 0.001, the iteration times are 10 epochs, the verification precision of the verification set is recorded, when the recall rate of the tile verification set reaches over 97%, the training is stopped, the model is tested, and if the recall rate of the tile on the verification set reaches over 95%, the model training is finished.
Further, specifically, the anchor frame is obtained a priori by a kmeans clustering algorithm according to the ceramic tile training data set, and the formula of the position transformation operation is shown as follows
bx＝σ(tx)+cx
by＝σ(ty)+cy
Wherein t isx,tyIs the predicted coordinate offset value, tw,thIs a scaling, pw、phIs a preset anchor box mapped to width and height in the feature map, cx，cyThe feature map output cells are shifted from the upper left corner of the cell of the corresponding image, and sigma (-) is a sigmoid function.
Further, the sizes of the specific three-dimensional feature maps are 13x13, 26x26 and 52x52 respectively, wherein the feature map of 13x13 has a larger receptive field and is used for predicting large defects; the 26 × 26 signature has a moderate receptive field for predicting moderate defects; the 52x52 signature has a smaller receptive field for predicting small defects.
The invention also provides a detection system suitable for highlighting the surface defects of various ceramic tiles, which comprises the following steps:
the image acquisition module is used for acquiring a first image shot by a first camera and acquiring a second image shot by a second camera, wherein the first camera is a color linear array camera and is arranged perpendicular to the shooting plane, and the second camera is a black and white linear array camera and is arranged obliquely to the shooting plane;
the image preprocessing module is used for respectively carrying out image preprocessing operation on the first image and the second image to obtain a positioned first image and a positioned second image;
and the detection module is used for inputting the positioned first image and the second image into a pre-established neural network model, outputting three-dimensional feature maps, wherein each feature map comprises the offset information of each anchor frame and the confidence coefficient of the corresponding prediction, and selecting a boundary frame with the confidence coefficient larger than a first threshold value as the final output of the predicted defect.
Further, specifically, the pre-established neural network model includes,
a feature extraction module, configured to extract feature information of the first and second images after positioning, where an output end of the feature extraction module is connected to an input end of a first RESn module, an output end of the first RESn module is connected to an input end of a second RESn module and an input end of a second concat module, an output end of the second RESn module is connected to an input end of a first DBL module and an input end of the first concat module, an output end of the first DBL module is connected to input ends of a first output convolution layer conv and a second DBL module respectively for up-sampling, an output end of the second DBL module is connected to an input end of the first concat module, an output end of the first concat module is connected to an input end of a third DBL module, and an output end of the third DBL module is connected to an input end of the second output convolution layer conv and an input end of a fourth DBL module respectively for up-sampling, the output end of the fourth DBL module is connected with the input end of the first concat module, the output end of the first concat module is connected with the input end of a fifth DBL module, the output end of the fifth DBL module is connected with the input end of a third output convolution layer conv, and finally the output ends of the first output convolution layer conv, the second output convolution layer conv and the third output convolution layer conv respectively output characteristic graphs with three dimensions;
the DBL module comprises a convolution layer, a normalization BN layer and a leak relu activation layer, the concat module is formed by splicing feature graphs, RESn is a jump layer structure, n represents a number and represents the number of res _ units contained in the res _ block, the res _ unit structure is formed by connecting two DBL modules and jump layers, the former DBL module adopts a convolution kernel of 1x1, and the latter DBL module adopts a convolution kernel of 3x 3.
The invention also proposes a computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the method as described above.
The invention has the beneficial effects that:
the method of the invention utilizes a double camera to obtain the tile image, the fusion processing of the two images can be suitable for the tiles with various complex textures, and various unobvious defects on the surface of the tile are highlighted. And training a sample by using a convolutional neural network to enable the network to obtain a defect identification function, thereby realizing high-precision detection of the ceramic tile defects. The detection system can be suitable for detecting the defects of various tiles and has detection diversity. And the defect detection is carried out by utilizing a deep learning method, so that the labor cost is reduced, and the detection efficiency is improved.
Drawings
The foregoing and other features of the present disclosure will become more apparent from the detailed description of the embodiments shown in conjunction with the drawings in which like reference characters designate the same or similar elements throughout the several views, and it is apparent that the drawings in the following description are merely some examples of the present disclosure and that other drawings may be derived therefrom by those skilled in the art without the benefit of any inventive faculty, and in which:
FIG. 1 is a flow chart of a detection method for highlighting surface defects of various types of ceramic tiles according to the present invention;
FIG. 2 is a schematic structural diagram of a neural network model of a detection method for highlighting surface defects of various tiles according to the present invention;
FIG. 3 is a schematic structural diagram of an RESn module of the detection method for highlighting surface defects of various ceramic tiles according to the present invention;
FIG. 4 is a schematic structural diagram of a res _ unit module of the detection method for highlighting surface defects of various types of tiles according to the present invention;
FIG. 5 is a schematic view of a dual camera adapted to the detection method for highlighting surface defects of various types of tiles of the present invention.
Detailed Description
The conception, the specific structure and the technical effects of the present invention will be clearly and completely described in conjunction with the embodiments and the accompanying drawings to fully understand the objects, the schemes and the effects of the present invention. It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The same reference numbers will be used throughout the drawings to refer to the same or like parts.
Referring to fig. 1 and 5, in embodiment 1, the present invention provides a detection method for highlighting surface defects of various types of tiles, including the following steps:
step 110, acquiring a first image shot by a first camera and a second image shot by a second camera, wherein the first camera is a color line camera and is arranged perpendicular to the shooting plane, and the second camera is a black-and-white line camera and is arranged obliquely to the shooting plane;
step 120, performing image preprocessing operation on the first image and the second image respectively to obtain a first image and a second image after positioning;
step 130, inputting the positioned first image and second image into a pre-established neural network model, outputting feature maps of three dimensions, wherein each feature map comprises the offset information of each anchor frame and the confidence degree of corresponding prediction, and selecting a boundary frame with the confidence degree larger than a first threshold value as the final output of the predicted defect.
The first threshold is set manually, and is set to 0.6 in the present embodiment.
Specifically, a double camera is used for shooting images of various tiles, a color camera is used for shooting color defects such as dirt and glaze dripping, and a black and white camera is used for shooting concave-convex defects such as shallow cracks and small pinholes. Preprocessing the two images, dividing the two images into small pictures, inputting the divided images into a neural network architecture for training, and processing the contrast characteristics of the two images to quickly locate the defects of the ceramic tiles. And finally, detecting the defects of the new ceramic tiles by using the trained network.
As a preferred embodiment of the present invention, specifically, the image preprocessing operation includes the following,
and respectively carrying out filtering and denoising processing on the first image and the second image, and then carrying out Harris corner detection to position the tile images in the first image and the second image to obtain the positioned first image and second image.
As a preferred embodiment of the present invention, the method further comprises,
before inputting the first and second positioned images into the pre-established neural network model, the sizes of the first and second positioned images are uniformly adjusted to be integral multiples of 416x416 pixels, and then the first and second positioned images are cut into a plurality of small-size images of 416x416 pixels.
Referring to fig. 2, 3 and 4, as a preferred embodiment of the present invention, specifically, the neural network model pre-established includes,
a feature extraction module, configured to extract feature information of the first and second images after positioning, where an output end of the feature extraction module is connected to an input end of a first RESn module, an output end of the first RESn module is connected to an input end of a second RESn module and an input end of a second concat module, an output end of the second RESn module is connected to an input end of a first DBL module and an input end of the first concat module, an output end of the first DBL module is connected to input ends of a first output convolution layer conv and a second DBL module respectively for up-sampling, an output end of the second DBL module is connected to an input end of the first concat module, an output end of the first concat module is connected to an input end of a third DBL module, and an output end of the third DBL module is connected to an input end of the second output convolution layer conv and an input end of a fourth DBL module respectively for up-sampling, the output end of the fourth DBL module is connected with the input end of the first concat module, the output end of the first concat module is connected with the input end of a fifth DBL module, the output end of the fifth DBL module is connected with the input end of a third output convolution layer conv, and finally the output ends of the first output convolution layer conv, the second output convolution layer conv and the third output convolution layer conv respectively output characteristic graphs with three dimensions;
the DBL module comprises a convolution layer, a normalization BN layer and a leak relu activation layer, the concat module is formed by splicing feature graphs, RESn is a jump layer structure, n represents a number and represents the number of res _ units contained in the res _ block, the res _ unit structure is formed by connecting two DBL modules and jump layers, the former DBL module adopts a convolution kernel of 1x1, and the latter DBL module adopts a convolution kernel of 3x 3.
In the preferred embodiment, n in the first RESn block is 8, and n in the second RESn block is 4.
In fig. 2, the first RES8 is a first RES module, the second RES4 is a second RES module, the DBL and conv in the first row are the first output convolution layer conv, the DBL and conv in the second row are the second output convolution layer conv, and the DBL and conv in the third row are the third output convolution layer conv, where the up-sampling in the first row is the up-sampling process corresponding to the second DBL module, and the up-sampling in the second row is the up-sampling process of the fourth DBL module.
As a preferred embodiment of the present invention, specifically, the neural network model is obtained by training,
the loss function is the same as that of a YOLOv3 network, an Adam gradient descent method is used for training, the learning rate is set to be 0.001, the iteration times are 10 epochs, the verification precision of the verification set is recorded, when the recall rate of the tile verification set reaches over 97%, the training is stopped, the model is tested, and if the recall rate of the tile on the verification set reaches over 95%, the model training is finished.
Specifically, the anchor frame is obtained a priori by a kmeans clustering algorithm according to the ceramic tile training data set, and the position transformation operation formula is shown as follows
bx＝σ(tx)+cx
by＝σ(ty)+cy
Wherein t isx,tyIs the predicted coordinate offset value, tw,thIs a scaling, pw、phIs a preset anchor box mapped to width and height in the feature map, cx，cyThe feature map output cells are shifted from the upper left corner of the cell of the corresponding image, and sigma (-) is a sigmoid function.
As a preferred embodiment of the present invention, the sizes of the specific three-dimensional feature maps are 13x13, 26x26, and 52x52, respectively, wherein the feature map of 13x13 has a larger receptive field for predicting large defects; the 26 × 26 signature has a moderate receptive field for predicting moderate defects; the 52x52 signature has a smaller receptive field for predicting small defects.
The invention also provides a detection system suitable for highlighting the surface defects of various ceramic tiles, which comprises the following steps:
the image acquisition module is used for acquiring a first image shot by a first camera and acquiring a second image shot by a second camera, wherein the first camera is a color linear array camera and is arranged perpendicular to the shooting plane, and the second camera is a black and white linear array camera and is arranged obliquely to the shooting plane;
the image preprocessing module is used for respectively carrying out image preprocessing operation on the first image and the second image to obtain a positioned first image and a positioned second image;
and the detection module is used for inputting the positioned first image and the second image into a pre-established neural network model, outputting three-dimensional feature maps, wherein each feature map comprises the offset information of each anchor frame and the confidence coefficient of the corresponding prediction, and selecting a boundary frame with the confidence coefficient larger than a first threshold value as the final output of the predicted defect.
In particular, the above-described system, when in operation,
the color camera and the black and white camera are combined with the line light source to polish and photograph the ceramic tiles, when the conveying belt conveys the ceramic tiles to enter a photographing area, the two-line-array camera starts photographing, and multi-view images of the ceramic tiles are obtained.
And respectively carrying out a series of preprocessing such as filtering and denoising, corner point detection, image segmentation and the like on the black-white tile image and the color image shot by the two cameras.
Inputting the ceramic tiles into a neural network for training, inputting black and white ceramic tile images and color images in a training set into an improved YOLOv3 network, training by using a gradient descent method, and stopping training when the recall rate of the test set reaches a set value.
And in the testing stage, inputting the ceramic tile image to be detected into the network model, and calculating the coordinates, length, width, category and confidence coefficient of the candidate frame of the ceramic tile defect at the output of the last layer to realize the detection of the defect.
As a preferred embodiment of the present invention, specifically, the neural network model pre-established includes,
a feature extraction module, configured to extract feature information of the first and second images after positioning, where an output end of the feature extraction module is connected to an input end of a first RESn module, an output end of the first RESn module is connected to an input end of a second RESn module and an input end of a second concat module, an output end of the second RESn module is connected to an input end of a first DBL module and an input end of the first concat module, an output end of the first DBL module is connected to input ends of a first output convolution layer conv and a second DBL module respectively for up-sampling, an output end of the second DBL module is connected to an input end of the first concat module, an output end of the first concat module is connected to an input end of a third DBL module, and an output end of the third DBL module is connected to an input end of the second output convolution layer conv and an input end of a fourth DBL module respectively for up-sampling, the output end of the fourth DBL module is connected with the input end of the first concat module, the output end of the first concat module is connected with the input end of a fifth DBL module, the output end of the fifth DBL module is connected with the input end of a third output convolution layer conv, and finally the output ends of the first output convolution layer conv, the second output convolution layer conv and the third output convolution layer conv respectively output characteristic graphs with three dimensions;
the DBL module comprises a convolution layer, a normalization BN layer and a leak relu activation layer, the concat module is formed by splicing feature graphs, RESn is a jump layer structure, n represents a number and represents the number of res _ units contained in the res _ block, the res _ unit structure is formed by connecting two DBL modules and jump layers, the former DBL module adopts a convolution kernel of 1x1, and the latter DBL module adopts a convolution kernel of 3x 3.
The invention also proposes a computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the method as described above.
The modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical modules, may be located in one place, or may be distributed on a plurality of network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution in this embodiment.
In addition, functional modules in the embodiments of the present invention may be integrated into one processing module, or each of the modules may exist alone physically, or two or more modules are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode.
The integrated module, if implemented in the form of a software functional module and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, all or part of the flow of the method according to the embodiments of the present invention may also be implemented by a computer program, which may be stored in a computer-readable storage medium and can implement the steps of the above-described method embodiments when the computer program is executed by a processor. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, and the like. It should be noted that the computer readable medium includes content that can be suitably increased or decreased according to the requirements of legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable media does not include electrical carrier signals and telecommunication signals according to legislation and patent practice.
While the present invention has been described in considerable detail and with particular reference to a few illustrative embodiments thereof, it is not intended to be limited to any such details or embodiments or any particular embodiments, but it is to be construed as effectively covering the intended scope of the invention by providing a broad, potential interpretation of such claims in view of the prior art with reference to the appended claims. Furthermore, the foregoing describes the invention in terms of embodiments foreseen by the inventor for which an enabling description was available, notwithstanding that insubstantial modifications of the invention, not presently foreseen, may nonetheless represent equivalent modifications thereto.
The above description is only a preferred embodiment of the present invention, and the present invention is not limited to the above embodiment, and the present invention shall fall within the protection scope of the present invention as long as the technical effects of the present invention are achieved by the same means. The invention is capable of other modifications and variations in its technical solution and/or its implementation, within the scope of protection of the invention.
Claims (10)
1. A detection method suitable for highlighting surface defects of various ceramic tiles is characterized by comprising the following steps:
acquiring a first image shot by a first camera and a second image shot by a second camera, wherein the first camera is a color line camera and is arranged perpendicular to the shooting plane, and the second camera is a black-and-white line camera and is arranged obliquely to the shooting plane;
respectively carrying out image preprocessing operation on the first image and the second image to obtain a positioned first image and a positioned second image;
inputting the positioned first image and the second image into a pre-established neural network model, outputting feature maps of three dimensions, wherein each feature map comprises the offset information of each anchor frame and the confidence coefficient of corresponding prediction, and selecting a boundary frame with the confidence coefficient larger than a first threshold value as the final output of the predicted defect.
2. The method for detecting the surface defect highlighting of the multi-variety ceramic tiles according to claim 1, wherein the image preprocessing operation comprises the following steps,
and respectively carrying out filtering and denoising processing on the first image and the second image, and then carrying out Harris corner detection to position the tile images in the first image and the second image to obtain the positioned first image and second image.
3. The method for detecting the surface defect highlighting of the multi-variety ceramic tiles according to claim 1, further comprising,
before inputting the first and second positioned images into the pre-established neural network model, the sizes of the first and second positioned images are uniformly adjusted to be integral multiples of 416x416 pixels, and then the first and second positioned images are cut into a plurality of small-size images of 416x416 pixels.
4. The method for detecting the highlighting of the surface defect of the multi-variety ceramic tile according to claim 1, wherein the pre-established neural network model comprises,
a feature extraction module, configured to extract feature information of the first and second images after positioning, where an output end of the feature extraction module is connected to an input end of a first RESn module, an output end of the first RESn module is connected to an input end of a second RESn module and an input end of a second concat module, an output end of the second RESn module is connected to an input end of a first DBL module and an input end of the first concat module, an output end of the first DBL module is connected to input ends of a first output convolution layer conv and a second DBL module respectively for up-sampling, an output end of the second DBL module is connected to an input end of the first concat module, an output end of the first concat module is connected to an input end of a third DBL module, and an output end of the third DBL module is connected to an input end of the second output convolution layer conv and an input end of a fourth DBL module respectively for up-sampling, the output end of the fourth DBL module is connected with the input end of the first concat module, the output end of the first concat module is connected with the input end of a fifth DBL module, the output end of the fifth DBL module is connected with the input end of a third output convolution layer conv, and finally the output ends of the first output convolution layer conv, the second output convolution layer conv and the third output convolution layer conv respectively output characteristic graphs with three dimensions;
the DBL module comprises a convolution layer, a normalization BN layer and a leak relu activation layer, the concat module is formed by splicing feature graphs, RESn is a jump layer structure, n represents a number and represents the number of res _ units contained in the res _ block, the res _ unit structure is formed by connecting two DBL modules and jump layers, the former DBL module adopts a convolution kernel of 1x1, and the latter DBL module adopts a convolution kernel of 3x 3.
5. The method for detecting the highlighting of the surface defect of the multi-variety ceramic tile according to claim 4, wherein the neural network model is obtained by training,
the loss function is the same as that of a YOLOv3 network, an Adam gradient descent method is used for training, the learning rate is set to be 0.001, the iteration times are 10 epochs, the verification precision of the verification set is recorded, when the recall rate of the tile verification set reaches over 97%, the training is stopped, the model is tested, and if the recall rate of the tile on the verification set reaches over 95%, the model training is finished.
6. The method for detecting the highlighting of the surface defect of the multi-variety tile according to claim 1, wherein the anchor frame is obtained a priori by a kmeans clustering algorithm according to a tile training data set, and the formula of the position transformation operation is shown as follows
bx＝σ(tx)+cx
by＝σ(ty)+cy
Wherein t isx,tyIs the predicted coordinate offset value, tw,thIs a scaling, pw、phIs a preset anchor box mapped to width and height in the feature map, cx，cyThe feature map output cells are shifted from the upper left corner of the cell of the corresponding image, and sigma (-) is a sigmoid function.
7. The method for detecting the surface defect highlighting of the multi-variety ceramic tiles as claimed in claim 6, wherein the sizes of the specific three-dimensional feature maps are respectively 13x13, 26x26 and 52x52, wherein the 13x13 feature map has a larger receptive field for predicting large defects; the 26 × 26 signature has a moderate receptive field for predicting moderate defects; the 52x52 signature has a smaller receptive field for predicting small defects.
8. A detection system suitable for highlighting surface defects of various ceramic tiles is characterized by comprising the following components:
the image acquisition module is used for acquiring a first image shot by a first camera and acquiring a second image shot by a second camera, wherein the first camera is a color linear array camera and is arranged perpendicular to the shooting plane, and the second camera is a black and white linear array camera and is arranged obliquely to the shooting plane;
the image preprocessing module is used for respectively carrying out image preprocessing operation on the first image and the second image to obtain a positioned first image and a positioned second image;
and the detection module is used for inputting the positioned first image and the second image into a pre-established neural network model, outputting three-dimensional feature maps, wherein each feature map comprises the offset information of each anchor frame and the confidence coefficient of the corresponding prediction, and selecting a boundary frame with the confidence coefficient larger than a first threshold value as the final output of the predicted defect.
9. The detection system for highlighting surface defects of various tiles according to claim 8, wherein the pre-established neural network model comprises,
a feature extraction module, configured to extract feature information of the first and second images after positioning, where an output end of the feature extraction module is connected to an input end of a first RESn module, an output end of the first RESn module is connected to an input end of a second RESn module and an input end of a second concat module, an output end of the second RESn module is connected to an input end of a first DBL module and an input end of the first concat module, an output end of the first DBL module is connected to input ends of a first output convolution layer conv and a second DBL module respectively for up-sampling, an output end of the second DBL module is connected to an input end of the first concat module, an output end of the first concat module is connected to an input end of a third DBL module, and an output end of the third DBL module is connected to an input end of the second output convolution layer conv and an input end of a fourth DBL module respectively for up-sampling, the output end of the fourth DBL module is connected with the input end of the first concat module, the output end of the first concat module is connected with the input end of a fifth DBL module, the output end of the fifth DBL module is connected with the input end of a third output convolution layer conv, and finally the output ends of the first output convolution layer conv, the second output convolution layer conv and the third output convolution layer conv respectively output characteristic graphs with three dimensions;
the DBL module comprises a convolution layer, a normalization BN layer and a leak relu activation layer, the concat module is formed by splicing feature graphs, RESn is a jump layer structure, n represents a number and represents the number of res _ units contained in the res _ block, the res _ unit structure is formed by connecting two DBL modules and jump layers, the former DBL module adopts a convolution kernel of 1x1, and the latter DBL module adopts a convolution kernel of 3x 3.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111301767.4A CN114202508A (en) | 2021-11-04 | 2021-11-04 | A detection method and system adapting to the highlighting of surface defects of multi-variety ceramic tiles |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111301767.4A CN114202508A (en) | 2021-11-04 | 2021-11-04 | A detection method and system adapting to the highlighting of surface defects of multi-variety ceramic tiles |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114202508A true CN114202508A (en) | 2022-03-18 |
Family
ID=80646823
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111301767.4A Pending CN114202508A (en) | 2021-11-04 | 2021-11-04 | A detection method and system adapting to the highlighting of surface defects of multi-variety ceramic tiles |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114202508A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114782418A (en) * | 2022-06-16 | 2022-07-22 | 深圳市信润富联数字科技有限公司 | Detection method and device for ceramic tile surface defects and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019104767A1 (en) * | 2017-11-28 | 2019-06-06 | 河海大学常州校区 | Fabric defect detection method based on deep convolutional neural network and visual saliency |
| CN109858389A (en) * | 2019-01-10 | 2019-06-07 | 浙江新再灵科技股份有限公司 | Vertical ladder demographic method and system based on deep learning |
| CN110728200A (en) * | 2019-09-23 | 2020-01-24 | 武汉大学 | Real-time pedestrian detection method and system based on deep learning |
| CN111504459A (en) * | 2020-04-09 | 2020-08-07 | 广东科达洁能股份有限公司 | Detection device for ceramic tile grading and color separation |
| CN112200764A (en) * | 2020-09-02 | 2021-01-08 | 重庆邮电大学 | Photovoltaic power station hot spot detection and positioning method based on thermal infrared image |
-
2021
- 2021-11-04 CN CN202111301767.4A patent/CN114202508A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019104767A1 (en) * | 2017-11-28 | 2019-06-06 | 河海大学常州校区 | Fabric defect detection method based on deep convolutional neural network and visual saliency |
| CN109858389A (en) * | 2019-01-10 | 2019-06-07 | 浙江新再灵科技股份有限公司 | Vertical ladder demographic method and system based on deep learning |
| CN110728200A (en) * | 2019-09-23 | 2020-01-24 | 武汉大学 | Real-time pedestrian detection method and system based on deep learning |
| CN111504459A (en) * | 2020-04-09 | 2020-08-07 | 广东科达洁能股份有限公司 | Detection device for ceramic tile grading and color separation |
| CN112200764A (en) * | 2020-09-02 | 2021-01-08 | 重庆邮电大学 | Photovoltaic power station hot spot detection and positioning method based on thermal infrared image |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114782418A (en) * | 2022-06-16 | 2022-07-22 | 深圳市信润富联数字科技有限公司 | Detection method and device for ceramic tile surface defects and storage medium |
| CN114782418B (en) * | 2022-06-16 | 2022-09-16 | 深圳市信润富联数字科技有限公司 | Detection method and device for ceramic tile surface defects and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106875381B (en) | Mobile phone shell defect detection method based on deep learning | |
| CN110544231B (en) | Surface defect detection method of lithium battery electrode based on background normalization and centralized compensation algorithm | |
| CN110210448B (en) | Intelligent face skin aging degree identification and evaluation method | |
| CN112233067A (en) | Hot rolled steel coil end face quality detection method and system | |
| JP2008171417A (en) | Method for detecting a substantially rectangular object in an image, method for estimating a background color in an image, computer readable medium, apparatus for detecting a substantially rectangular object in an image, and apparatus for estimating a background color in an image | |
| CN113240626A (en) | Neural network-based method for detecting and classifying concave-convex flaws of glass cover plate | |
| CN114241469B (en) | A method and device for identifying information during meter rotation process | |
| CN104089580A (en) | Concrete surface crack width measuring instrument and method based on smart phone | |
| CN114913134B (en) | Tunnel shotcrete roughness identification method, terminal equipment and storage medium | |
| CN112435214A (en) | Pollen detection method and device based on prior frame linear scaling and electronic equipment | |
| CN111353396A (en) | Concrete crack segmentation method based on SCSEOCUnet | |
| CN115588024A (en) | Artificial intelligence-based complex industrial image edge extraction method and device | |
| CN113486715A (en) | Image reproduction identification method, intelligent terminal and computer storage medium | |
| CN112085727B (en) | An intelligent identification method for oxide scale structure on the surface of hot-rolled steel | |
| CN115880209A (en) | Surface defect detection system, method, device and medium suitable for steel plate | |
| CN118229629A (en) | Edge detection method, system, equipment and medium for large-size liquid crystal display | |
| CN115546536B (en) | Ivory product identification method and system | |
| CN114202508A (en) | A detection method and system adapting to the highlighting of surface defects of multi-variety ceramic tiles | |
| CN110298347B (en) | Method for identifying automobile exhaust analyzer screen based on GrayWorld and PCA-CNN | |
| CN115937320B (en) | Visual positioning method for polishing mobile phone shell | |
| CN111768384A (en) | Cell counting method and system based on three-dimensional scanning imaging | |
| CN116823761A (en) | Information processing methods, devices, equipment and storage media based on cell segmentation | |
| CN115578333A (en) | Domestic ceramic surface defect detection method based on region division and yolo5 | |
| CN117169247A (en) | Metal surface defect multi-dimensional detection method and system based on machine vision | |
| CN115331209A (en) | Bronze inscription identification method and device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |