CN113066088A - Detection method, detection device and storage medium in industrial detection - Google Patents
Detection method, detection device and storage medium in industrial detection Download PDFInfo
- Publication number
- CN113066088A CN113066088A CN202110617151.1A CN202110617151A CN113066088A CN 113066088 A CN113066088 A CN 113066088A CN 202110617151 A CN202110617151 A CN 202110617151A CN 113066088 A CN113066088 A CN 113066088A
- Authority
- CN
- China
- Prior art keywords
- picture
- workpiece
- neural network
- example segmentation
- workpiece picture
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 92
- 238000003860 storage Methods 0.000 title claims abstract description 11
- 230000011218 segmentation Effects 0.000 claims abstract description 93
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 61
- 239000011159 matrix material Substances 0.000 claims abstract description 41
- 238000012549 training Methods 0.000 claims abstract description 34
- 238000001914 filtration Methods 0.000 claims abstract description 33
- 238000013528 artificial neural network Methods 0.000 claims abstract description 32
- 238000000034 method Methods 0.000 claims abstract description 23
- 238000012545 processing Methods 0.000 claims abstract description 17
- 238000012216 screening Methods 0.000 claims abstract description 17
- 230000001131 transforming effect Effects 0.000 claims abstract description 6
- 230000009466 transformation Effects 0.000 claims description 18
- 238000004590 computer program Methods 0.000 claims description 3
- 230000008901 benefit Effects 0.000 abstract description 2
- 230000000694 effects Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 6
- 230000006870 function Effects 0.000 description 4
- 238000009776 industrial production Methods 0.000 description 4
- 239000000463 material Substances 0.000 description 4
- 230000007547 defect Effects 0.000 description 3
- 238000003384 imaging method Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000004075 alteration Effects 0.000 description 2
- 238000007689 inspection Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/20—Image enhancement or restoration by the use of local operators
-
- G06T5/70—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/32—Determination of transform parameters for the alignment of images, i.e. image registration using correlation-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a detection method, a detection device and a storage medium in industrial detection, wherein the method comprises the following steps: acquiring a workpiece picture to be detected and screening out a fuzzy image; carrying out zooming and Gaussian filtering on the workpiece picture after the fuzzy image is screened out; the processed workpiece picture is taken as input and sent into a first convolution neural network to obtain an example segmentation graph of the workpiece picture; sending the example segmentation graph of the workpiece picture as an input into a second convolutional neural network to obtain a homography matrix H for transforming the example segmentation graph of the workpiece picture to a reference example segmentation graph; acquiring a corresponding registration image according to the homography matrix H; and carrying out image detection according to the registration image. The detection method adopts the convolutional neural network when image registration is carried out, does not depend on an operator with commercial use any more, can greatly reduce the software cost, and in addition, the network has the advantages of higher robustness and registration rate, less training time and lower requirements on resources of a CPU (Central processing Unit) and a GPU (graphics processing Unit).
Description
Technical Field
The invention relates to the technical field of detection, in particular to a detection method in industrial detection, a detection device in industrial detection and a non-transitory computer readable storage medium.
Background
In the field of industrial detection, whether defect detection or size detection is adopted, image registration of a picture of a workpiece after imaging is an important link, and if the workpiece on an industrial production line is directly subjected to defect detection or size detection without the image registration link, the detection result has great deviation, which leads to serious consequences. Therefore, it is necessary to perform image registration on the workpiece image to be detected before other subsequent detection works.
In the related art, the image registration method generally includes:
1. visual algorithms (such as feature-based matching algorithms), which mostly utilize operators in OpenCV (cross-platform computer vision and machine learning software library) to extract features and implement image registration through feature matching, but the feature extraction method has high resource requirement on CPU (Central Processing Unit) and GPU (Graphics Processing Unit) and has low Processing speed, and some operators are expensive in commercial use, and generally use one operator to perform one charge (i.e., perform one charge for image registration), and the cost is too high for industrial quality inspection with many workpiece pictures.
2. Template matching (e.g. registration algorithm based on gray scale image), however, the registration rate of such method is not high because the industrial camera light source is not always stable in industrial production.
3. Based on the registration of GAN (generic adaptive Networks) Networks, engineers need to spend much time training GAN Networks before image matching, and different models need to be trained for different optical surfaces of the same workpiece, which is time-consuming and labor-consuming.
Disclosure of Invention
In order to solve the above technical problems, a first objective of the present invention is to provide a detection method in industrial detection, which performs detection according to a registered image, and can significantly improve the detection effect, and a convolutional neural network is adopted during image registration, and is not dependent on an operator for commercial use, so that the software cost can be greatly reduced.
A second object of the present invention is to provide a detection device for industrial detection.
A third object of the invention is to propose a non-transitory computer-readable storage medium.
The technical scheme adopted by the invention is as follows:
an embodiment of the first aspect of the present invention provides a detection method in industrial detection, including the following steps: acquiring a workpiece picture to be detected and screening out a fuzzy image; carrying out zooming and Gaussian filtering on the workpiece picture after the fuzzy image is screened out; sending the workpiece picture subjected to scaling and Gaussian filtering into a first convolution neural network as input to obtain an example segmentation graph of the workpiece picture; sending the example segmentation graph of the workpiece picture as an input into a second convolutional neural network to obtain a homography matrix H for transforming the example segmentation graph of the workpiece picture to a reference example segmentation graph; performing homography transformation on the corresponding workpiece picture according to the homography matrix H to obtain a corresponding registration image; and detecting images according to the registration images.
The detection method in the industrial detection provided by the invention can also have the following additional technical characteristics:
according to one embodiment of the invention, the first convolutional neural network is trained in the following way: marking characteristic point positions of a workpiece picture to obtain an example segmentation picture of the workpiece picture, and simultaneously carrying out scaling and Gaussian filtering on the workpiece picture; and sending the workpiece picture subjected to scaling and Gaussian filtering into the first convolution neural network as input, and sending an example segmentation graph of the workpiece picture into the first convolution neural network as a label so as to perform supervision training on the first convolution neural network.
According to one embodiment of the invention, the second convolutional neural network is trained in the following way: marking feature point positions of the standard picture to obtain a reference example segmentation picture; inputting the example segmentation graph of the workpiece picture and the reference example segmentation graph into a homographic operator to obtain a homography matrix of the workpiece picture and a standard picture; and taking the example segmentation graph of the workpiece picture as input to be sent into the second convolutional neural network, and taking the homography matrix as a label to be sent into the second convolutional neural network so as to perform supervised training on the second convolutional neural network.
According to one embodiment of the invention, a LoG operator is used for screening out the blurred image in the picture of the workpiece to be detected.
An embodiment of the second aspect of the present invention provides a detection apparatus in industrial detection, including: the screening module is used for acquiring a workpiece picture to be detected and screening out a fuzzy image; the processing module is used for carrying out zooming and Gaussian filtering processing on the workpiece picture after the fuzzy image is screened out; the first acquisition module is used for sending the workpiece picture subjected to scaling and Gaussian filtering into a first convolution neural network as input so as to acquire an example segmentation graph of the workpiece picture; a second obtaining module, configured to send the example segmentation map of the workpiece picture as an input to a second convolutional neural network, so as to obtain a homography matrix H that transforms the example segmentation map of the workpiece picture to a reference example segmentation map; the transformation module is used for carrying out homography transformation on the corresponding workpiece picture according to the homography matrix H so as to obtain a corresponding registration image; a detection module for performing image detection based on the registration image.
The detection device in industrial detection provided by the invention can also have the following additional technical characteristics:
according to one embodiment of the invention, the first convolutional neural network is trained in the following way: marking characteristic point positions of a workpiece picture to obtain an example segmentation picture of the workpiece picture, and simultaneously carrying out scaling and Gaussian filtering on the workpiece picture; and sending the workpiece picture subjected to scaling and Gaussian filtering into the first convolution neural network as input, and sending an example segmentation graph of the workpiece picture into the first convolution neural network as a label so as to perform supervision training on the first convolution neural network.
According to one embodiment of the invention, the second convolutional neural network is trained in the following way: marking feature point positions of the standard picture to obtain a reference example segmentation picture; inputting the example segmentation graph of the workpiece picture and the reference example segmentation graph into a homographic operator to obtain a homography matrix of the workpiece picture and a standard picture; and taking the example segmentation graph of the workpiece picture as input to be sent into the second convolutional neural network, and taking the homography matrix as a label to be sent into the second convolutional neural network so as to perform supervised training on the second convolutional neural network.
According to one embodiment of the invention, a LoG operator is used for screening out the blurred image in the picture of the workpiece to be detected.
The invention has the beneficial effects that:
the invention can obviously improve the detection effect by detecting according to the registered image, and adopts the convolution neural network when the image registration is carried out, thereby not depending on the operator of commercial use, greatly reducing the software cost.
Drawings
FIG. 1 is a flow diagram of a detection method in industrial detection according to one embodiment of the invention;
FIG. 2 is a schematic diagram of a detection method in industrial detection according to one embodiment of the present invention;
FIG. 3 is a schematic diagram of a training process for a first convolutional neural network, according to one embodiment of the present invention;
FIG. 4 is a schematic diagram of a training process for a second convolutional neural network, in accordance with one embodiment of the present invention;
FIG. 5 is a block schematic diagram of a detection device in industrial detection according to one embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
FIG. 1 is a flow chart of a detection method in industrial detection according to one embodiment of the present invention. As shown in fig. 1, the method comprises the steps of:
and S1, acquiring the picture of the workpiece to be detected and screening out the blurred image.
According to one embodiment of the invention, the blurred image in the picture of the workpiece to be detected can be screened out by using the LoG operator.
In particular, the blurred image is first screened out, which can improve the accuracy of the subsequent model (convolutional neural network) and can improve the speed of the overall operation.
And S2, carrying out zooming and Gaussian filtering processing on the workpiece picture after the fuzzy image is screened out.
Specifically, the zoom factor may be 1/2 (i.e. 2 times smaller) of the original image size, where too small a factor may reduce the speed of the subsequent model, and too large a factor may reduce the accuracy of the subsequent registration. In addition, aiming at different projects, the scaling factor can be adjusted, and the optimal value can be found according to corresponding experimental verification.
And (4) carrying out one layer of Gaussian filtering processing on the zoomed workpiece picture, wherein the purpose is to smooth the image so as to remove Gaussian noise in the image, thereby improving the accuracy of the model. The formula of gaussian filtering is:
wherein, (x, y) is the point coordinate of the workpiece picture.
And S3, inputting the workpiece picture after being subjected to the scaling and Gaussian filtering processing into a first convolution neural network A to obtain an example segmentation graph of the workpiece picture.
And S4, feeding the example segmentation graph of the workpiece picture as an input into a second convolutional neural network B to obtain a homography matrix H for transforming the example segmentation graph of the workpiece picture into a reference example segmentation graph.
And S5, performing homography transformation on the corresponding workpiece picture according to the homography matrix H to obtain a corresponding registration image.
And S6, detecting the image according to the registration image.
Specifically, the workpiece picture is an original imaging picture of a workpiece to be detected on an industrial production line, the standard picture is a picture of a standard workpiece, namely a registration template of all workpiece pictures, and the image registration is to convert the workpiece picture (the original imaging picture before registration) into a new workpiece picture (a registration image after registration) close to the standard picture through affine transformation. It can be understood that the workpiece picture of the workpiece to be detected on the industrial production line is a data set.
As shown in fig. 2, a workpiece picture to be detected is obtained, and then subjected to blur image screening, scaling and gaussian filtering, and then sent into a first convolution neural network a as an input, an example segmentation map (with feature points) of the workpiece picture is obtained by using the first convolution neural network a, and then sent into a second convolution neural network B as an input, and the example segmentation map of the workpiece picture is transformed into a homography matrix H of a reference example segmentation map by using the second convolution neural network B, and then the workpiece picture is subjected to homography transformation according to the homography matrix H, wherein the homography transformation process is as follows:
where H is a homography matrix, typically a 3 x 3 matrix,
the coordinate position of any point in the workpiece picture before homography transformation (the characteristic point position of the workpiece picture);
is the corresponding coordinate position in the registered image after the homography transformation.
And finally, carrying out image detection according to the registered image, such as defect detection or size detection, wherein the detection according to the registered image can reduce the deviation of the detection result and improve the detection effect.
Therefore, the method can obviously improve the detection effect by detecting according to the registered images, and adopts the convolutional neural network when the images are registered, so that the method does not depend on an operator with commercial use any more, the software cost can be greatly reduced, in addition, the network has higher robustness and registration rate, less training time and lower requirements on resources of a CPU and a GPU.
According to one embodiment of the invention, the first convolutional neural network a is trained in the following way: marking characteristic point positions of the workpiece picture to obtain an example segmentation picture of the workpiece picture, and simultaneously carrying out scaling and Gaussian filtering on the workpiece picture; and sending the workpiece picture subjected to scaling and Gaussian filtering into a first convolution neural network A as an input, and sending an example segmentation picture of the workpiece picture into the first convolution neural network A as a label so as to perform supervision training on the first convolution neural network A.
Specifically, as shown in fig. 3, a preset number of workpiece pictures are obtained as a training sample, then the workpiece pictures in the training sample are scaled and gaussian filtered to improve subsequent model representation, and meanwhile, obvious feature points of the workpiece pictures in the training sample are found and labeled to obtain an example segmentation graph of the workpiece pictures. Such as: the front surface of one workpiece is provided with 5 threaded holes in different positions, and the threaded holes can be used as obvious characteristic points to mark the 5 threaded holes. Then, the example segmentation graph of the workpiece picture is sent into the first convolution neural network A as label for supervised training until all sample training is completed. After a plurality of rounds of training, the final model of the first convolution neural network A can predict example segmentation information of obvious feature points, namely an example segmentation graph, according to the input image.
According to one embodiment of the invention, the second convolutional neural network is trained in the following way: marking feature point positions of the standard picture to obtain a reference example segmentation picture; inputting the example segmentation chart and the reference example segmentation chart of the workpiece picture into a homographic operator to obtain a homography matrix of the workpiece picture and the standard picture; and sending the example segmentation graph of the workpiece picture as input into a second convolutional neural network, and sending the homography matrix as a label into the second convolutional neural network so as to perform supervision training on the second convolutional neural network.
Specifically, the training of the second convolutional neural network B needs to be based on the training of the first convolutional neural network a. As shown in fig. 4, feature point locations are labeled on a standard picture to obtain an example segmentation map (reference example segmentation map) of the standard picture, and then the example segmentation map of the workpiece picture of the training sample and the reference example segmentation map are input into a homograph operator together, so that a homography matrix H of each workpiece picture and the standard picture in the sample can be obtained, where H is a transformation matrix required for transforming the workpiece example segmentation map to the reference example segmentation map. The homography matrix H is then fed into a second convolutional neural network B as label and the example segmentation map of the workpiece picture is fed into the second convolutional neural network B as input. After multiple rounds of supervised training, the second convolutional neural network B can predict a homography matrix H which can transform the workpiece example segmentation graph to the reference example segmentation graph according to the input workpiece example segmentation graph by the final model.
Therefore, the second convolutional neural network B can replace a commercial operator to directly predict the homography matrix, even if the homography operator needs to be applied in the training process, the application frequency is only the number of training samples, the number of the training samples is far lower than the number of pictures in an industrial quality inspection field, and the commercial operator generally adopts a charge-by-time system, so that the software cost can be greatly reduced by adopting the second convolutional neural network B, the cost advantage is more obvious when more work pieces are used, the robustness of the convolutional neural network is high, the effect is also high, the training time is short, the resource requirements on a CPU and a GPU are low, and the registration rate and the registration speed of image registration can be greatly improved by adopting the trained second convolutional neural network B to perform image registration.
In summary, according to the detection method in industrial detection of the embodiment of the present invention, the workpiece picture to be detected is obtained, the blurred image is screened out, the workpiece picture from which the blurred image is screened out is scaled and gaussian-filtered, the workpiece picture after scaling and gaussian-filtering is input into the first convolutional neural network to obtain the example segmentation map of the workpiece picture, the example segmentation map of the workpiece picture is input into the second convolutional neural network to obtain the homography matrix H that transforms the example segmentation map of the workpiece picture to the reference example segmentation map, the corresponding workpiece picture is homography transformed according to the homography matrix H to obtain the corresponding registration image, and finally, the image detection is performed according to the registration image. Therefore, the method can obviously improve the detection effect by detecting according to the registered images, and adopts the convolutional neural network when the images are registered, so that the method does not depend on an operator with commercial use any more, the software cost can be greatly reduced, in addition, the network has higher robustness and registration rate, less training time and lower requirements on resources of a CPU and a GPU.
Corresponding to the detection method in the industrial detection, the invention also provides a detection device in the industrial detection. Since the device embodiment of the present invention corresponds to the method embodiment described above, details that are not disclosed in the device embodiment may refer to the method embodiment described above, and are not described again in the present invention.
FIG. 5 is a block schematic diagram of a detection device in industrial detection according to one embodiment of the present invention. As shown in fig. 5, the apparatus includes: the screening module 1, the processing module 2, the first obtaining module 3, the second obtaining module 4, the transforming module 5 and the detecting module 6.
The screening module 1 is used for acquiring a workpiece picture to be detected and screening out a fuzzy image; the processing module 2 is used for carrying out zooming and Gaussian filtering processing on the workpiece picture after the fuzzy image is screened out; the first acquisition module 3 is used for inputting the workpiece picture subjected to scaling and Gaussian filtering into a first convolution neural network to acquire an example segmentation graph of the workpiece picture; the second obtaining module 4 is configured to send the example segmentation map of the workpiece picture as an input to a second convolutional neural network, so as to obtain a homography matrix H that transforms the example segmentation map of the workpiece picture to the reference example segmentation map; the transformation module 5 is used for performing homography transformation on the corresponding workpiece picture according to the homography matrix H to obtain a corresponding registration image; the detection module 6 is used for detecting images according to the registration images.
According to one embodiment of the invention, the first convolutional neural network is trained in the following way: marking characteristic point positions of the workpiece picture to obtain an example segmentation picture of the workpiece picture, and simultaneously carrying out scaling and Gaussian filtering on the workpiece picture; and sending the workpiece picture subjected to scaling and Gaussian filtering into a first convolution neural network as input, and sending an example segmentation picture of the workpiece picture into the first convolution neural network as a label so as to perform supervision training on the first convolution neural network.
According to one embodiment of the invention, the second convolutional neural network is trained in the following way: marking feature point positions of the standard picture to obtain a reference example segmentation picture; inputting the example segmentation chart and the reference example segmentation chart of the workpiece picture into a homographic operator to obtain a homography matrix of the workpiece picture and the standard picture; and sending the example segmentation graph of the workpiece picture as input into a second convolutional neural network, and sending the homography matrix as a label into the second convolutional neural network so as to perform supervision training on the second convolutional neural network.
According to one embodiment of the invention, a LoG operator is used for screening out fuzzy images in the picture of the workpiece to be detected.
In summary, according to the detection apparatus in industrial detection in the embodiment of the present invention, the screening module is used to obtain a workpiece picture to be detected and screen out a blurred image, the processing module performs scaling and gaussian filtering on the workpiece picture after screening out the blurred image, the first obtaining module sends the workpiece picture after scaling and gaussian filtering as an input to the first convolutional neural network to obtain an example segmentation map of the workpiece picture, the second obtaining module sends the example segmentation map of the workpiece picture as an input to the second convolutional neural network to obtain a homography matrix H that transforms the example segmentation map of the workpiece picture to a reference example segmentation map, the transformation module performs homography transformation on the corresponding workpiece picture according to the homography matrix H to obtain a corresponding registration image, and the detection module performs image detection according to the registration image. Therefore, the device carries out detection according to the registered images, the detection effect can be obviously improved, a convolutional neural network is adopted during image registration, operators for commercial use are not relied on, the software cost can be greatly reduced, in addition, the network has high robustness and registration rate, the training time is short, and the resource requirements on a CPU and a GPU are low.
Furthermore, the present invention also proposes a non-transitory computer-readable storage medium, on which a computer program is stored, which when executed by a processor implements the detection method in the industrial detection described above.
According to the non-transitory computer-readable storage medium of the embodiment of the present invention, when a computer program stored thereon is executed by a processor, a workpiece picture to be detected is acquired and a blurred image is screened out, then the workpiece picture from which the blurred image is screened out is subjected to scaling and gaussian filtering, then the workpiece picture subjected to scaling and gaussian filtering is input into a first convolutional neural network to acquire an example segmentation map of the workpiece picture, then the example segmentation map of the workpiece picture is input into a second convolutional neural network to acquire a homography matrix H that transforms the example segmentation map of the workpiece picture to a reference example segmentation map, a corresponding workpiece picture is subjected to homography transformation according to the homography matrix H to acquire a corresponding registered image, and finally, image detection is performed according to the registered image, thereby detection is performed according to the registered image, the detection effect can be obviously improved, the convolutional neural network is adopted during image registration, operators for commercial use are not depended on, the software cost can be greatly reduced, in addition, the network has high robustness and registration rate, the training time is short, and the resource requirements on a CPU and a GPU are low.
In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implying any number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. The meaning of "plurality" is two or more unless specifically limited otherwise.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction. In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing steps of a custom logic function or process, and alternate implementations are included within the scope of the preferred embodiment of the present invention in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present invention.
The logic and/or steps represented in the flowcharts or otherwise described herein, e.g., an ordered listing of executable instructions that can be considered to implement logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium could even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via for instance optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.
It should be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
It will be understood by those skilled in the art that all or part of the steps carried by the method for implementing the above embodiments may be implemented by hardware related to instructions of a program, which may be stored in a computer readable storage medium, and when the program is executed, the program includes one or a combination of the steps of the method embodiments.
In addition, functional units in the embodiments of the present invention may be integrated into one processing module, or each unit may exist alone physically, or two or more units are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a stand-alone product, may also be stored in a computer readable storage medium.
The storage medium mentioned above may be a read-only memory, a magnetic or optical disk, etc. Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention, and that variations, modifications, substitutions and alterations can be made to the above embodiments by those of ordinary skill in the art within the scope of the present invention.
Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.
Claims (9)
1. A detection method in industrial detection is characterized by comprising the following steps:
acquiring a workpiece picture to be detected and screening out a fuzzy image;
carrying out zooming and Gaussian filtering on the workpiece picture after the fuzzy image is screened out;
sending the workpiece picture subjected to scaling and Gaussian filtering into a first convolution neural network as input to obtain an example segmentation graph of the workpiece picture;
sending the example segmentation graph of the workpiece picture as an input into a second convolutional neural network to obtain a homography matrix H for transforming the example segmentation graph of the workpiece picture to a reference example segmentation graph;
performing homography transformation on the corresponding workpiece picture according to the homography matrix H to obtain a corresponding registration image;
and detecting images according to the registration images.
2. The method of detection in industrial detection according to claim 1, characterized in that the first convolutional neural network is trained in the following way:
marking characteristic point positions of a workpiece picture to obtain an example segmentation picture of the workpiece picture, and simultaneously carrying out scaling and Gaussian filtering on the workpiece picture;
and sending the workpiece picture subjected to scaling and Gaussian filtering into the first convolution neural network as input, and sending an example segmentation graph of the workpiece picture into the first convolution neural network as a label so as to perform supervision training on the first convolution neural network.
3. The detection method in industrial detection according to claim 2, characterized in that the second convolutional neural network is trained in the following way:
marking feature point positions of the standard picture to obtain a reference example segmentation picture;
inputting the example segmentation graph of the workpiece picture and the reference example segmentation graph into a homographic operator to obtain a homography matrix of the workpiece picture and a standard picture;
and taking the example segmentation graph of the workpiece picture as input to be sent into the second convolutional neural network, and taking the homography matrix as a label to be sent into the second convolutional neural network so as to perform supervised training on the second convolutional neural network.
4. The method as claimed in claim 1, wherein a LoG operator is used to screen out the blurred image in the image of the workpiece to be detected.
5. A detection device in industrial detection, characterized by comprising:
the screening module is used for acquiring a workpiece picture to be detected and screening out a fuzzy image;
the processing module is used for carrying out zooming and Gaussian filtering processing on the workpiece picture after the fuzzy image is screened out;
the first acquisition module is used for sending the workpiece picture subjected to scaling and Gaussian filtering into a first convolution neural network as input so as to acquire an example segmentation graph of the workpiece picture;
a second obtaining module, configured to send the example segmentation map of the workpiece picture as an input to a second convolutional neural network, so as to obtain a homography matrix H that transforms the example segmentation map of the workpiece picture to a reference example segmentation map;
the transformation module is used for carrying out homography transformation on the corresponding workpiece picture according to the homography matrix H so as to obtain a corresponding registration image;
a detection module for performing image detection based on the registration image.
6. The detecting device in industrial detection according to claim 5, characterized in that the first convolutional neural network is trained in the following way:
marking characteristic point positions of a workpiece picture to obtain an example segmentation picture of the workpiece picture, and simultaneously carrying out scaling and Gaussian filtering on the workpiece picture;
and sending the workpiece picture subjected to scaling and Gaussian filtering into the first convolution neural network as input, and sending an example segmentation graph of the workpiece picture into the first convolution neural network as a label so as to perform supervision training on the first convolution neural network.
7. The detecting device in industrial detection according to claim 6, characterized in that the second convolutional neural network is trained in the following way:
marking feature point positions of the standard picture to obtain a reference example segmentation picture;
inputting the example segmentation graph of the workpiece picture and the reference example segmentation graph into a homographic operator to obtain a homography matrix of the workpiece picture and a standard picture;
and taking the example segmentation graph of the workpiece picture as input to be sent into the second convolutional neural network, and taking the homography matrix as a label to be sent into the second convolutional neural network so as to perform supervised training on the second convolutional neural network.
8. The detecting device in industrial detection according to claim 5, wherein the blurred image in the picture of the workpiece to be detected is screened out by using a LoG operator.
9. A non-transitory computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, implements a detection method in industrial detection according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110617151.1A CN113066088A (en) | 2021-06-03 | 2021-06-03 | Detection method, detection device and storage medium in industrial detection |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110617151.1A CN113066088A (en) | 2021-06-03 | 2021-06-03 | Detection method, detection device and storage medium in industrial detection |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113066088A true CN113066088A (en) | 2021-07-02 |
Family
ID=76568562
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110617151.1A Pending CN113066088A (en) | 2021-06-03 | 2021-06-03 | Detection method, detection device and storage medium in industrial detection |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113066088A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113469299A (en) * | 2021-09-06 | 2021-10-01 | 常州微亿智造科技有限公司 | Defect detection method and defect detection device in industrial detection |
| CN113516697A (en) * | 2021-07-19 | 2021-10-19 | 北京世纪好未来教育科技有限公司 | Image registration method and device, electronic equipment and computer-readable storage medium |
| CN114399505A (en) * | 2022-03-25 | 2022-04-26 | 江苏智云天工科技有限公司 | Detection method and detection device in industrial detection |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107689035A (en) * | 2017-08-30 | 2018-02-13 | 广州华多网络科技有限公司 | A kind of homography matrix based on convolutional neural networks determines method and device |
| CN110263918A (en) * | 2019-06-17 | 2019-09-20 | 北京字节跳动网络技术有限公司 | Method, apparatus, electronic equipment and the computer readable storage medium of training convolutional neural networks |
| CN111739029A (en) * | 2020-06-12 | 2020-10-02 | 鞍钢集团矿业有限公司 | Electric bucket tooth falling detection method based on deep learning convolutional neural network |
| CN111833237A (en) * | 2020-01-19 | 2020-10-27 | 宁波大学 | Image registration method based on convolutional neural network and local homography transformation |
-
2021
- 2021-06-03 CN CN202110617151.1A patent/CN113066088A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107689035A (en) * | 2017-08-30 | 2018-02-13 | 广州华多网络科技有限公司 | A kind of homography matrix based on convolutional neural networks determines method and device |
| CN110263918A (en) * | 2019-06-17 | 2019-09-20 | 北京字节跳动网络技术有限公司 | Method, apparatus, electronic equipment and the computer readable storage medium of training convolutional neural networks |
| CN111833237A (en) * | 2020-01-19 | 2020-10-27 | 宁波大学 | Image registration method based on convolutional neural network and local homography transformation |
| CN111739029A (en) * | 2020-06-12 | 2020-10-02 | 鞍钢集团矿业有限公司 | Electric bucket tooth falling detection method based on deep learning convolutional neural network |
Non-Patent Citations (2)
| Title |
|---|
| 佟喜峰等: "《图像处理与识别技术 应用与实践》", 30 April 2019 * |
| 舒程珣等: "基于卷积神经网络的点云配准方法", 《激光与光电子学进展》 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113516697A (en) * | 2021-07-19 | 2021-10-19 | 北京世纪好未来教育科技有限公司 | Image registration method and device, electronic equipment and computer-readable storage medium |
| CN113516697B (en) * | 2021-07-19 | 2024-02-02 | 北京世纪好未来教育科技有限公司 | Image registration method, device, electronic equipment and computer readable storage medium |
| CN113469299A (en) * | 2021-09-06 | 2021-10-01 | 常州微亿智造科技有限公司 | Defect detection method and defect detection device in industrial detection |
| CN113469299B (en) * | 2021-09-06 | 2021-12-07 | 常州微亿智造科技有限公司 | Defect detection method and defect detection device in industrial detection |
| CN114399505A (en) * | 2022-03-25 | 2022-04-26 | 江苏智云天工科技有限公司 | Detection method and detection device in industrial detection |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110414507B (en) | License plate recognition method and device, computer equipment and storage medium | |
| CN113066088A (en) | Detection method, detection device and storage medium in industrial detection | |
| CN110189285B (en) | Multi-frame image fusion method and device | |
| CN108229475B (en) | Vehicle tracking method, system, computer device and readable storage medium | |
| CN109632808B (en) | Edge defect detection method and device, electronic equipment and storage medium | |
| CN111008961B (en) | Transmission line equipment defect detection method and system, equipment and medium thereof | |
| CN111476758B (en) | Defect detection method and device for AMOLED display screen, computer equipment and storage medium | |
| TWI743837B (en) | Training data increment method, electronic apparatus and computer-readable medium | |
| CN113469971B (en) | Image matching method, detection device and storage medium | |
| CN113628179B (en) | PCB surface defect real-time detection method, device and readable medium | |
| CN114049499A (en) | Target object detection method, apparatus and storage medium for continuous contour | |
| CN117094975A (en) | Method and device for detecting surface defects of steel and electronic equipment | |
| CN116152166A (en) | Defect detection method and related device based on feature correlation | |
| CN114399505A (en) | Detection method and detection device in industrial detection | |
| CN112330618B (en) | Image offset detection method, device and storage medium | |
| CN115937059A (en) | Part inspection system with generative training models | |
| CN111340040B (en) | Paper character recognition method and device, electronic equipment and storage medium | |
| CN113469944A (en) | Product quality inspection method and device and electronic equipment | |
| CN110211105B (en) | Blurred image detection method, device and system | |
| CN115512381A (en) | Text recognition method, text recognition device, text recognition equipment, storage medium and working machine | |
| CN101150651A (en) | A pre-processing method for obtaining differential image | |
| CN114841943A (en) | Part detection method, device, equipment and storage medium | |
| CN112950598A (en) | Method, device and equipment for detecting flaw of workpiece and storage medium | |
| CN113469299B (en) | Defect detection method and defect detection device in industrial detection | |
| CN113450316B (en) | Method, system and device for detecting defects of metal surface characters and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210702 |