CN111929311B - One-stop intelligent defect detection system - Google Patents

One-stop intelligent defect detection system Download PDF

Info

Publication number
CN111929311B
CN111929311B CN202011099840.XA CN202011099840A CN111929311B CN 111929311 B CN111929311 B CN 111929311B CN 202011099840 A CN202011099840 A CN 202011099840A CN 111929311 B CN111929311 B CN 111929311B
Authority
CN
China
Prior art keywords
image
defect
defect detection
detection
stop
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202011099840.XA
Other languages
Chinese (zh)
Other versions
CN111929311A (en
Inventor
郑秀征
叶振飞
王英利
冯龙申
梁长国
朱超平
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
BEIJING ZODNGOC AUTOMATIC TECHNOLOGY CO LTD
Original Assignee
BEIJING ZODNGOC AUTOMATIC TECHNOLOGY CO LTD
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by BEIJING ZODNGOC AUTOMATIC TECHNOLOGY CO LTD filed Critical BEIJING ZODNGOC AUTOMATIC TECHNOLOGY CO LTD
Priority to CN202011099840.XA priority Critical patent/CN111929311B/en
Publication of CN111929311A publication Critical patent/CN111929311A/en
Application granted granted Critical
Publication of CN111929311B publication Critical patent/CN111929311B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G01N2021/8887—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges based on image processing techniques

Landscapes

  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Chemical & Material Sciences (AREA)
  • Analytical Chemistry (AREA)
  • Biochemistry (AREA)
  • General Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Immunology (AREA)
  • Pathology (AREA)
  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Signal Processing (AREA)
  • Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)

Abstract

The invention relates to a one-stop intelligent defect detection system, which comprises an image acquisition part, an intelligent mode self-adaption part, a one-stop defect detection part, a parallel acceleration part, a defect storage and communication part, a control part and a defect identification execution part. The invention can detect all defect types covered in the visual field range in a one-stop mode and give the position information of the defect area; and a complex defect detection tool is not required to be used for carrying out a combined and superposed defect detection means. The invention can carry out self-adaptive detection mode matching according to the material quality and the detection requirement of the detected material so as to achieve the optimal defect detection effect. The one-stop defect detection algorithm is suitable for parallel acceleration calculation, and can easily perform one-stop online, rapid and real-time defect detection by introducing a parallel acceleration part. The one-stop defect detection algorithm has very strong robustness to environmental factors such as illumination conditions and the like. The most painful environmental problem for industrial visual inspection is light exposure.

Description

One-stop intelligent defect detection system
Technical Field
The invention relates to a one-stop intelligent defect detection system, and belongs to the technical field of intelligent defect detection.
Background
At present, most of the market share of the demand of defect detection is distributed in the industrial production field, and the defect detection of most industries in the industrial production field is still in the artificial naked eye detection stage, so that enterprises need to invest a large amount of manpower, material resources and financial resources to ensure the quality of delivered products. At present, the mainstream visual defect detection system in the market mostly adopts a complex defect detection tool to implement a combined and superposed detection means; or a special detection tool is developed aiming at a single industry, and the detection tool is poor in universality.
Therefore, the patent provides a one-stop intelligent defect detection system, which can carry out intelligent detection mode matching according to the material to be detected, and can easily carry out one-stop online, quick and real-time defect detection by introducing a parallel acceleration part.
Disclosure of Invention
Aiming at the defects in the prior art, the invention provides a one-stop intelligent defect detection system, which has the following specific technical scheme:
a one-stop intelligent defect detection system, comprising:
the image acquisition part consists of a camera, a lens and a lighting subsystem, and is used for acquiring an original image of the detected material;
the intelligent mode self-adaptive part is realized by a detection mode self-adaptive algorithm, and the detection mode self-adaptive algorithm realizes closed-loop control by constructing a defect detection result quality evaluation function;
the one-stop defect detection part is realized by a one-stop defect detection algorithm, and the one-stop defect detection algorithm is used for realizing all defect types and defect position information covered in the camera view field;
the parallel acceleration part comprises a GPU or FPGA chip and a parallel acceleration algorithm module;
the defect storage and communication part is realized by a database and a communication module and is used for storing the defect detection result in real time;
a control section for receiving a defect detection result;
and the defect identification executing part is used for receiving the control signal sent by the control part and making a corresponding action to carry out defect identification.
As an improvement of the above technical solution, the operation flow of the one-stop intelligent defect detecting system includes the following steps:
step 1, collecting a plurality of images of good products;
step 2, collecting a plurality of defective images;
step 3, deploying defect detection and starting on line, and intelligently matching the detection mode through the good product image and the defective product image by the intelligent mode self-adaptive part;
step 4, one-stop normal defect detection and parallel calculation acceleration are carried out;
step 5, storing the defect detection result and sending detection information to the control part in a communication way;
step 6, the control part receives the detection information and sends a related control signal;
and 7, the defect identification executing part receives the control signal sent by the control part and makes a corresponding executing action.
As an improvement of the above technical solution, the resolution of the images involved in the one-stop defect detection algorithm must be consistent, otherwise normal defect detection cannot be performed; if the resolution ratios of the collected images are not consistent in practical application, defect detection is carried out through advanced resolution ratio unified conversion, and the method comprises the following steps:
Figure 55493DEST_PATH_IMAGE001
，
Figure 379158DEST_PATH_IMAGE002
，
wherein the content of the first and second substances,
Figure 955633DEST_PATH_IMAGE003
is the height of the acquired image;
Figure 6634DEST_PATH_IMAGE004
width of the acquired image;
Figure 555427DEST_PATH_IMAGE005
to obtain the image height;
Figure 174627DEST_PATH_IMAGE006
to obtain the image width;
Figure 848185DEST_PATH_IMAGE007
to input good images.
As an improvement of the above technical solution, the detection mode adaptive algorithm includes the following steps:
step 1.1, CollectionnOpening a good product image of the detected material,nthe collected good product images are weighted and fused to obtain a good product template image as a positive integer
Figure 578244DEST_PATH_IMAGE008
，
CollectingmOpening a defective product image of the detected material,mis a positive integer, and the acquired defective product images are weighted and fused to obtain a defective product template image
Figure 981543DEST_PATH_IMAGE009
，
Figure 896279DEST_PATH_IMAGE010
，
Figure 385029DEST_PATH_IMAGE011
，
Wherein the content of the first and second substances,irepresenting the number of the collected images;ncollecting the total number of good images;mthe total number of the collected defective images is counted;
Figure 918778DEST_PATH_IMAGE012
the images are collected for different good products;
Figure 848688DEST_PATH_IMAGE013
the collected images of different defective products;
Figure 12953DEST_PATH_IMAGE008
obtaining a good product template image through weighted fusion;
Figure 51316DEST_PATH_IMAGE014
obtaining a defective product template image through weighted fusion; g () is a weighted fusion operator;
step 1.2, selecting a detection mode
Figure 60861DEST_PATH_IMAGE015
To good product template image
Figure 625703DEST_PATH_IMAGE008
According to a mode
Figure 492028DEST_PATH_IMAGE015
Performing feature extraction operation to obtain feature images of good-quality template images
Figure 955370DEST_PATH_IMAGE016
(ii) a For defective product template image
Figure 706289DEST_PATH_IMAGE014
According to a mode
Figure 938687DEST_PATH_IMAGE015
Performing feature extraction operation to obtain feature image of defective product template image
Figure 507071DEST_PATH_IMAGE017
(ii) a To be detected image
Figure 192131DEST_PATH_IMAGE018
According to a mode
Figure 996007DEST_PATH_IMAGE015
Performing feature extraction operation to obtain feature image of the detected image
Figure 82912DEST_PATH_IMAGE019
；
Figure 822198DEST_PATH_IMAGE020
，
Figure 994553DEST_PATH_IMAGE021
，
Figure 87274DEST_PATH_IMAGE022
，
Wherein the content of the first and second substances,
Figure 294265DEST_PATH_IMAGE023
indicating a detection mode
Figure 204452DEST_PATH_IMAGE015
Corresponding feature extraction operators;
step 1.3, detecting mode
Figure 864103DEST_PATH_IMAGE015
Characteristic image of
Figure 744204DEST_PATH_IMAGE016
,
Figure 602438DEST_PATH_IMAGE019
Carrying out matrix subtraction and solving the absolute value of the matrix subtraction to obtain a difference characteristic image
Figure 621210DEST_PATH_IMAGE024
(ii) a For difference characteristic image
Figure 705840DEST_PATH_IMAGE024
Performing small kernel convolution operation to obtain defect position, and marking the defect position to the detected image
Figure 468260DEST_PATH_IMAGE018
Corresponding to the spatial position to obtain a detection identification image
Figure 181001DEST_PATH_IMAGE025
；
Figure 292045DEST_PATH_IMAGE026
，
Figure 926289DEST_PATH_IMAGE027
，
Wherein the content of the first and second substances,Conv.() To representThe convolution identification operator is used for carrying out convolution operation on the difference value characteristics to obtain a defect position and restoring the defect position information to the detected image for identification;
step 1.3, exhausting all detection modes
Figure 289137DEST_PATH_IMAGE015
And repeating the steps 1.1 to 1.3 to obtain an optimal mode
Figure 325226DEST_PATH_IMAGE028
；
Figure 357905DEST_PATH_IMAGE029
，
Figure 276182DEST_PATH_IMAGE030
，
Figure 380404DEST_PATH_IMAGE031
，
Figure 215809DEST_PATH_IMAGE032
，
Wherein the content of the first and second substances,
Figure 747284DEST_PATH_IMAGE015
in order to detect the mode of the light,
Figure 152858DEST_PATH_IMAGE033
represents an optimal quality assessment value;f 1()、f 2() Representing a quality assessment operator;
Figure 60771DEST_PATH_IMAGE034
、
Figure 743556DEST_PATH_IMAGE035
in order to be the weighting coefficients,
Figure 445933DEST_PATH_IMAGE036
；
Figure 338803DEST_PATH_IMAGE037
representing a positive integer.
As an improvement of the above technical solution, the one-stop defect detection algorithm includes the following steps:
step 2.1, deploying actual detection on line and using an optimal mode
Figure 784827DEST_PATH_IMAGE028
Calculating the images of the good templates
Figure 571387DEST_PATH_IMAGE008
Characteristic image of
Figure 444665DEST_PATH_IMAGE038
For real-time single detected image
Figure 824831DEST_PATH_IMAGE039
According to the optimal mode
Figure 808967DEST_PATH_IMAGE028
Performing feature extraction to obtain image
Figure 466345DEST_PATH_IMAGE040
，
Figure 510524DEST_PATH_IMAGE041
，
Figure 377986DEST_PATH_IMAGE042
，
Wherein the content of the first and second substances,
Figure 165813DEST_PATH_IMAGE043
is an optimal mode
Figure 926965DEST_PATH_IMAGE028
Corresponding feature is providedTaking a descriptor;
step 2.2, optimizing the mode
Figure 142045DEST_PATH_IMAGE028
Characteristic image of
Figure 231224DEST_PATH_IMAGE038
,
Figure 517029DEST_PATH_IMAGE040
Carrying out matrix subtraction and solving the absolute value of the matrix subtraction to obtain a difference characteristic image
Figure 840694DEST_PATH_IMAGE044
(ii) a For difference characteristic image
Figure 620431DEST_PATH_IMAGE044
Performing small kernel convolution and identification operation to obtain defect detection result image
Figure 812378DEST_PATH_IMAGE045
(ii) a Defect detection result image
Figure 95592DEST_PATH_IMAGE045
The method comprises the steps of containing position information of all defect areas in an image, and carrying out differentiated color highlight identification on the defect positions;
Figure 105005DEST_PATH_IMAGE046
，
Figure 106459DEST_PATH_IMAGE047
。
as an improvement of the technical scheme, the parallel acceleration algorithm module is accelerated by adopting GPU hardware and a CUDA acceleration algorithm library in an industrial personal computer environment; the parallel acceleration algorithm module performs parallel acceleration by using an FPGA chip under an embedded system environment.
As an improvement of the technical scheme, the step of using the CUDA acceleration algorithm library comprises the following steps:
step 3.1, allocating host memories and initializing data;
step 3.2, allocating device memories, and copying data to the devices from the host;
3.3, calling a kernel function of the CUDA to complete specified operation on the device;
step 3.4, copying the operation result on the device to a host;
and 3.5, releasing the memory allocated on the device and the host.
The invention has the beneficial effects that:
1) one-stop detection: the invention can detect all defect types covered in the visual field range in a one-stop mode and give the position information of the defect area; and a complex defect detection tool is not required to be used for carrying out a combined and superposed defect detection means.
2) Intelligent mode adaptation: the invention can carry out self-adaptive detection mode matching according to the material quality and the detection requirement of the detected material so as to achieve the optimal defect detection effect. The user can also carry out secondary development and add a user-defined detection mode, and the user-defined detection mode can also support the matching of the self-adaptive detection mode.
3) And quick: the one-stop defect detection algorithm is suitable for parallel acceleration calculation, and can easily perform one-stop online, rapid and real-time defect detection by introducing a parallel acceleration part.
4) High robustness: the one-stop defect detection algorithm has very strong robustness to environmental factors such as illumination conditions and the like. The most painful environmental problem for industrial visual inspection is light exposure.
Drawings
FIG. 1 is a flow chart of a one-stop intelligent defect detection system according to the present invention;
FIG. 2 is a block diagram of an algorithm module of the one-stop intelligent defect detection system according to the present invention;
FIG. 3 is an original image for product inspection;
FIG. 4 is a graph of high intensity illumination condition defect detection;
fig. 5 is a diagram of the detection effect of low-brightness lighting conditions.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
Example 1
As shown in fig. 1, the one-stop intelligent defect detection system includes:
the image acquisition part consists of a camera, a lens and a lighting subsystem, and is used for acquiring an original image of the detected material; the method can acquire the high-quality and high-contrast original image of the detected material.
The intelligent mode self-adaptive part is realized by a detection mode self-adaptive algorithm, the detection mode self-adaptive algorithm realizes closed-loop control by constructing a defect detection result quality evaluation function, and the intelligent selection and switching of the detection modes can be realized aiming at different detected materials.
The one-stop defect detection part is realized by a one-stop defect detection algorithm, and the one-stop defect detection algorithm is used for realizing all defect types and defect position information covered in the camera view field; meanwhile, the one-stop defect detection algorithm can intelligently select different detection modes, including a user-defined detection mode added by secondary development of a user.
The parallel acceleration part comprises a GPU or FPGA chip and a parallel acceleration algorithm module; the real-time defect detection requirement of the high-resolution image can be easily met through parallel computing and acceleration.
The defect storage and communication part is realized by a database and a communication module and is used for storing the defect detection result in real time; the detection result may also be communicated to the control section.
And a control section for receiving the defect detection result.
And the defect identification executing part is used for receiving the control signal sent by the control part and making a corresponding action to carry out defect identification.
Example 2
The operation flow of the one-stop intelligent defect detection system comprises the following steps:
step 1, collecting a plurality of images of good products;
step 2, collecting a plurality of defective images;
step 3, deploying defect detection and starting on line, and intelligently matching the detection mode through the good product image and the defective product image by the intelligent mode self-adaptive part;
step 4, one-stop normal defect detection and parallel calculation acceleration are carried out;
step 5, storing the defect detection result and sending detection information to the control part in a communication way;
step 6, the control part receives the detection information and sends a related control signal;
and 7, the defect identification executing part receives the control signal sent by the control part and makes a corresponding executing action.
The defect mark executing part can be specifically a cylinder with a quick-dry marking pen, an ink jet printer, a coding machine and other mechanisms. The actions performed may be: and marking characters such as 'NG' or 'defective product' and 'unqualified product' on the defective product.
Example 3
The one-stop defect detection algorithm is realized by the following steps:
the one-stop defect detection algorithm can select different detection modes to adapt to detected materials of different materials, the detection mode self-adaptive algorithm can automatically select an optimal detection mode, and the parallel acceleration algorithm module provides strong calculation support for the one-stop defect detection algorithm.
Because the image acquisition part is determined, the acquired image resolutions are consistent, the image resolutions (height and width) related in the one-stop defect detection algorithm are required to be consistent, otherwise, normal defect detection cannot be carried out; if the resolution ratios of the collected images are not consistent in practical application, defect detection is carried out through advanced resolution ratio unified conversion, and the method comprises the following steps:
Figure 102097DEST_PATH_IMAGE001
，
Figure 239817DEST_PATH_IMAGE002
，
wherein the content of the first and second substances,
Figure 170864DEST_PATH_IMAGE003
is the height of the acquired image;
Figure 659614DEST_PATH_IMAGE004
width of the acquired image;
Figure 193364DEST_PATH_IMAGE005
to obtain the image height;
Figure 451170DEST_PATH_IMAGE006
to obtain the image width;
Figure 536806DEST_PATH_IMAGE007
to input good images.
Example 4
The detection mode self-adaptive algorithm intelligently selects and switches the optimal detection mode of the detected material.
As shown in fig. 2, the detection mode adaptive algorithm includes the following steps:
step 1.1, CollectionnOpening a good product image of the detected material,nthe collected good product images are weighted and fused to obtain a good product template image as a positive integer
Figure 309590DEST_PATH_IMAGE008
，
CollectingmOpening a defective product image of the detected material,mis a positive integer, and the acquired defective product images are weighted and fused to obtain a defective product template image
Figure 584714DEST_PATH_IMAGE009
，
Figure 900289DEST_PATH_IMAGE010
，
Figure 969876DEST_PATH_IMAGE011
，
Wherein the content of the first and second substances,irepresenting the number of the collected images;ncollecting the total number of good images;mthe total number of the collected defective images is counted;
Figure 495535DEST_PATH_IMAGE012
the images are collected for different good products;
Figure 308770DEST_PATH_IMAGE013
the collected images of different defective products;
Figure 728119DEST_PATH_IMAGE008
obtaining a good product template image through weighted fusion;
Figure 968608DEST_PATH_IMAGE014
obtaining a defective product template image through weighted fusion; g () is a weighted fusion operator;
step 1.2, selecting a detection mode
Figure 981563DEST_PATH_IMAGE048
To good product template image
Figure 598489DEST_PATH_IMAGE008
According to a mode
Figure 623077DEST_PATH_IMAGE048
Performing feature extraction operation to obtain feature images of good-quality template images
Figure 300046DEST_PATH_IMAGE016
(ii) a For defective product template image
Figure 534718DEST_PATH_IMAGE014
According to a mode
Figure 689756DEST_PATH_IMAGE048
Performing feature extraction operation to obtain feature image of defective product template image
Figure 83697DEST_PATH_IMAGE017
(ii) a To be detected image
Figure 665988DEST_PATH_IMAGE018
According to a mode
Figure 387957DEST_PATH_IMAGE048
Performing feature extraction operation to obtain feature image of the detected image
Figure 346685DEST_PATH_IMAGE019
；
Figure 345865DEST_PATH_IMAGE020
，
Figure 161375DEST_PATH_IMAGE021
，
Figure 573901DEST_PATH_IMAGE022
，
Wherein the content of the first and second substances,
Figure 70742DEST_PATH_IMAGE023
indicating a detection mode
Figure 173696DEST_PATH_IMAGE048
Corresponding feature extraction operators;
step 1.3, detecting mode
Figure 363369DEST_PATH_IMAGE015
Characteristic image of
Figure 59929DEST_PATH_IMAGE016
,
Figure 360461DEST_PATH_IMAGE019
Carrying out matrix subtraction and solving the absolute value of the matrix subtraction to obtain a difference characteristic image
Figure 68654DEST_PATH_IMAGE024
(ii) a For difference characteristic image
Figure 225966DEST_PATH_IMAGE024
Performing small kernel convolution operation to obtain defect position, and marking the defect position to the detected image
Figure 347505DEST_PATH_IMAGE018
Corresponding to the spatial position to obtain a detection identification image
Figure 373099DEST_PATH_IMAGE025
；
Figure 263695DEST_PATH_IMAGE026
，
Figure 857487DEST_PATH_IMAGE027
，
Wherein the content of the first and second substances,Conv.() Representing a convolution identification operator, carrying out convolution operation on the difference value characteristics to obtain a defect position, and restoring the defect position information to the detected image to carry out identification;
step 1.3, exhausting all detection modes
Figure 200744DEST_PATH_IMAGE015
And repeating the steps 1.1 to 1.3 to obtain an optimal mode
Figure 46340DEST_PATH_IMAGE028
；
Figure 791442DEST_PATH_IMAGE029
，
Figure 556136DEST_PATH_IMAGE030
，
Figure 386689DEST_PATH_IMAGE031
，
Figure 754085DEST_PATH_IMAGE032
，
Wherein the content of the first and second substances,
Figure 619273DEST_PATH_IMAGE015
for the detection mode, different detection modes correspond to different feature extraction methods, here, an equivalent image feature descriptor extraction function can be used, and by constructing a quality evaluation function, an optimal detection mode is selected from a plurality of detection modes
Figure 554868DEST_PATH_IMAGE049
。
Wherein the content of the first and second substances,
Figure 872716DEST_PATH_IMAGE033
represents an optimal quality assessment value;f 1()、f 2() Representing a quality assessment operator;
Figure 794536DEST_PATH_IMAGE034
、
Figure 514230DEST_PATH_IMAGE035
for weighting factors, empirical values may be taken
Figure 620727DEST_PATH_IMAGE036
；
Figure 425872DEST_PATH_IMAGE037
Representing a positive integer.
Example 5
The one-stop defect detection algorithm comprises the following steps:
step 2.1, deploying actual detection on line and using an optimal mode
Figure 400650DEST_PATH_IMAGE028
Calculating the images of the good templates
Figure 974851DEST_PATH_IMAGE008
Characteristic image of
Figure 252248DEST_PATH_IMAGE038
For real-time single detected image
Figure 279110DEST_PATH_IMAGE039
According to the optimal mode
Figure 808311DEST_PATH_IMAGE028
Performing feature extraction to obtain image
Figure 237019DEST_PATH_IMAGE040
，
Figure 685318DEST_PATH_IMAGE041
，
Figure 465055DEST_PATH_IMAGE042
，
Wherein the content of the first and second substances,
Figure 781636DEST_PATH_IMAGE043
is an optimal mode
Figure 64849DEST_PATH_IMAGE028
Corresponding feature extraction descriptors;
step 2.2, optimizing the mode
Figure 949629DEST_PATH_IMAGE028
Characteristic image of
Figure 951083DEST_PATH_IMAGE038
,
Figure 556508DEST_PATH_IMAGE040
Carrying out matrix subtraction and solving the absolute value of the matrix subtraction to obtain a difference characteristic image
Figure 959807DEST_PATH_IMAGE044
(ii) a For difference characteristic image
Figure 15488DEST_PATH_IMAGE044
Performing small kernel convolution and identification operation to obtain defect detection result image
Figure 238659DEST_PATH_IMAGE045
(ii) a Defect detection result image
Figure 162621DEST_PATH_IMAGE045
The method comprises the steps of containing position information of all defect areas in an image, and carrying out differentiated color highlight identification on the defect positions;
Figure 420427DEST_PATH_IMAGE046
，
Figure 381430DEST_PATH_IMAGE047
。
since the algorithm is based on the same detection mode
Figure 357476DEST_PATH_IMAGE028
Extracting features, and performing matrix subtraction to obtain difference feature image
Figure 570283DEST_PATH_IMAGE044
Therefore, the algorithm has natural high robustness to interference of environmental factors such as light.
The robustness of the algorithm to illumination can be seen from the overall flow of image processing (PipeLine). 3-5 are graphs providing a comparison of the detection effect of the algorithm before and after a change in ambient light; as can be seen from fig. 3-5: the algorithm is proved to be applicable to different illumination conditions.
Example 6
The one-stop defect detection algorithm applies a large number of matrix addition, subtraction, multiplication and division operations, and is suitable for parallel accelerated calculation. The parallel acceleration algorithm module adopts GPU hardware and a CUDA acceleration algorithm library to accelerate in an industrial personal computer environment; the parallel acceleration algorithm module performs parallel acceleration by using an FPGA chip under an embedded system environment.
The steps of using the CUDA acceleration algorithm library are as follows:
step 3.1, allocating host memories and initializing data;
step 3.2, allocating device memories, and copying data to the devices from the host;
3.3, calling a kernel function of the CUDA to complete specified operation on the device;
step 3.4, copying the operation result on the device to a host;
and 3.5, releasing the memory allocated on the device and the host.
Wherein, host can be regarded as a logic control unit, usually a CPU; device can be thought of as a parallel computing acceleration device, commonly referred to as a GPU.
In the above embodiments, the present invention provides a one-stop intelligent defect inspection system, which can inspect all defect types and defect position information covered in the field of view in one-stop manner without using the complex defect inspection tools to perform the combined and overlapped inspection means. The system can intelligently switch the detection modes according to the material attribute and the detection requirement of the detected material, achieves the requirement of being compatible with different material defect detection, and can also support a user to carry out secondary development and add a self-defined detection mode. The one-stop defect detection method is suitable for parallel accelerated computation, and the speed bottleneck problem of high-resolution image online deployment real-time detection is perfectly solved by adding a parallel computation part. For example: 500 ten thousand color images, wherein 500ms is needed for processing one image under the condition of no acceleration, the frame rate is 2fps, the product jump distance (the length of one product) is assumed to be 160mm, and the running speed is 19.2 m/min; acceleration was 200ms, 5fps, assuming a product jump (length of one piece of product) of 160mm, and a running speed of 48 m/min.
The invention has the following advantages:
1) one-stop detection: the invention can detect all defect types (defect types comprise hole blockage, incomplete waste discharge, scratch, dirt, foreign matters, glue deficiency, glue overflow, bubbles, folds, sheet overlapping and the like, most defect types have the characteristics of random positions, different sizes and shapes and the like) covered in a visual field in a one-stop mode, and provides position information of a defect area. And a complex defect detection tool is not required to be used for carrying out a combined and superposed defect detection means.
The core detection idea of the invention is to adopt a full-width characteristic image comparison mode to carry out 'full-width' comparison on a standard qualified product and a detected product, so that all defects covered by the whole image can be detected in a one-stop mode. (full comparison: the meaning of the full characteristic image is different according to the different detection modes, the full characteristic image can be simply understood as a characteristic diagram, the characteristic extraction methods under different modes are different, developers can add the characteristic extraction methods, and the method can provide a brand-new image processing flow of surface defect detection, namely standard sample collection, positioning, full characteristic diagram comparison and small convolution kernel identification of defect positions).
2) Intelligent mode adaptation: the invention can carry out self-adaptive detection mode matching according to the material quality and the detection requirement of the detected material so as to achieve the optimal defect detection effect. The user can also carry out secondary development and add a user-defined detection mode, and the user-defined detection mode can also support the matching of the self-adaptive detection mode.
3) And quick: the one-stop defect detection algorithm is suitable for parallel acceleration calculation, and can easily perform one-stop online, rapid and real-time defect detection by introducing a parallel acceleration part.
4) High robustness: the one-stop defect detection algorithm has very strong robustness to environmental factors such as illumination conditions and the like. The most painful environmental problem for industrial visual inspection is light exposure.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.

Claims (5)

1. A one-stop intelligent defect detection system, comprising:
the image acquisition part consists of a camera, a lens and a lighting subsystem, and is used for acquiring an original image of the detected material;
the intelligent mode self-adaptive part is realized by a detection mode self-adaptive algorithm, and the detection mode self-adaptive algorithm realizes closed-loop control by constructing a defect detection result quality evaluation function;
the one-stop defect detection part is realized by a one-stop defect detection algorithm, and the one-stop defect detection algorithm is used for realizing all defect types and defect position information covered in the camera view field;
the parallel acceleration part comprises a GPU or FPGA chip and a parallel acceleration algorithm module;
the defect storage and communication part is realized by a database and a communication module and is used for storing the defect detection result in real time;
a control section for receiving a defect detection result;
the defect identification executing part is used for receiving the control signal sent by the control part and making a corresponding action to carry out defect identification;
the detection mode adaptive algorithm comprises the following steps:
step 1.1, CollectionnOpening a good product image of the detected material,nthe collected good product images are weighted and fused to obtain a good product template image as a positive integer
Figure DEST_PATH_IMAGE001
,
CollectingmOpening a defective product image of the detected material,mis a positive integer, and carries out weighted fusion on the collected defective product images to obtain the defective productTemplate image
Figure 819509DEST_PATH_IMAGE002
，
Figure DEST_PATH_IMAGE003
，
Figure 873790DEST_PATH_IMAGE004
，
Wherein the content of the first and second substances,irepresenting the number of the collected images;ncollecting the total number of good images;mthe total number of the collected defective images is counted;
Figure DEST_PATH_IMAGE005
the images are collected for different good products;
Figure 19601DEST_PATH_IMAGE006
the collected images of different defective products;
Figure 191956DEST_PATH_IMAGE001
obtaining a good product template image through weighted fusion;
Figure DEST_PATH_IMAGE007
obtaining a defective product template image through weighted fusion; g () is a weighted fusion operator;
step 1.2, selecting a detection mode
Figure 81415DEST_PATH_IMAGE008
To good product template image
Figure 22826DEST_PATH_IMAGE001
According to a mode
Figure 870696DEST_PATH_IMAGE008
Performing feature extraction operation to obtain a good product template diagramCharacteristic image of image
Figure DEST_PATH_IMAGE009
(ii) a For defective product template image
Figure 999189DEST_PATH_IMAGE007
According to a mode
Figure 957918DEST_PATH_IMAGE008
Performing feature extraction operation to obtain feature image of defective product template image
Figure 753836DEST_PATH_IMAGE010
(ii) a To be detected image
Figure DEST_PATH_IMAGE011
According to a mode
Figure 739984DEST_PATH_IMAGE008
Performing feature extraction operation to obtain feature image of the detected image
Figure 886932DEST_PATH_IMAGE012
；
Figure DEST_PATH_IMAGE013
，
Figure 118193DEST_PATH_IMAGE014
，
Figure DEST_PATH_IMAGE015
，
Wherein the content of the first and second substances,
Figure 768617DEST_PATH_IMAGE016
indicating a detection mode
Figure 427132DEST_PATH_IMAGE008
Corresponding feature extraction operators;
step 1.3, detecting mode
Figure 61375DEST_PATH_IMAGE008
Characteristic image of
Figure 361907DEST_PATH_IMAGE009
,
Figure 132416DEST_PATH_IMAGE012
Carrying out matrix subtraction and solving the absolute value of the matrix subtraction to obtain a difference characteristic image
Figure DEST_PATH_IMAGE017
(ii) a For difference characteristic image
Figure 961832DEST_PATH_IMAGE017
Performing small kernel convolution operation to obtain defect position, and marking the defect position to the detected image
Figure 581907DEST_PATH_IMAGE011
Corresponding to the spatial position to obtain a detection identification image
Figure 420550DEST_PATH_IMAGE018
；
Figure DEST_PATH_IMAGE019
，
Figure 779987DEST_PATH_IMAGE020
，
Wherein the content of the first and second substances,Conv.() Representing a convolution identification operator, carrying out convolution operation on the difference value characteristics to obtain a defect position, and restoring the defect position information to the detected image to carry out identification;
step 1.3, exhausting all detection modes
Figure 45884DEST_PATH_IMAGE008
And repeating the steps 1.1 to 1.3 to obtain an optimal mode
Figure DEST_PATH_IMAGE021
；
Figure 123561DEST_PATH_IMAGE022
，
Figure DEST_PATH_IMAGE023
，
Figure 500316DEST_PATH_IMAGE024
，
Figure DEST_PATH_IMAGE025
，
Wherein the content of the first and second substances,
Figure 714259DEST_PATH_IMAGE008
in order to detect the mode of the light,
Figure 416636DEST_PATH_IMAGE026
represents an optimal quality assessment value;f 1()、f 2() Representing a quality assessment operator;
Figure DEST_PATH_IMAGE027
、
Figure 214566DEST_PATH_IMAGE028
in order to be the weighting coefficients,
Figure DEST_PATH_IMAGE029
；
Figure 129432DEST_PATH_IMAGE030
represents a positive integer;
the one-stop defect detection algorithm comprises the following steps:
step 2.1, deploying actual detection on line and using an optimal mode
Figure 729041DEST_PATH_IMAGE021
Calculating the images of the good templates
Figure 602319DEST_PATH_IMAGE001
Characteristic image of
Figure DEST_PATH_IMAGE031
For real-time single detected image
Figure 389009DEST_PATH_IMAGE032
According to the optimal mode
Figure 373146DEST_PATH_IMAGE021
Performing feature extraction to obtain image
Figure DEST_PATH_IMAGE033
，
Figure 561681DEST_PATH_IMAGE034
，
Figure DEST_PATH_IMAGE035
，
Wherein the content of the first and second substances,
Figure 573238DEST_PATH_IMAGE036
is an optimal mode
Figure 378383DEST_PATH_IMAGE021
Corresponding feature extraction descriptors;
step 2.2, optimizing the mode
Figure 166210DEST_PATH_IMAGE021
Characteristic image of
Figure 740411DEST_PATH_IMAGE031
,
Figure 689912DEST_PATH_IMAGE033
Carrying out matrix subtraction and solving the absolute value of the matrix subtraction to obtain a difference characteristic image
Figure DEST_PATH_IMAGE037
(ii) a For difference characteristic image
Figure 451195DEST_PATH_IMAGE037
Performing small kernel convolution and identification operation to obtain defect detection result image
Figure 777134DEST_PATH_IMAGE038
(ii) a Defect detection result image
Figure 205841DEST_PATH_IMAGE038
The method comprises the steps of containing position information of all defect areas in an image, and carrying out differentiated color highlight identification on the defect positions;
Figure DEST_PATH_IMAGE039
，
Figure 60665DEST_PATH_IMAGE040
。
2. the one-stop intelligent defect detecting system of claim 1, wherein the operation flow of the one-stop intelligent defect detecting system comprises the following steps:
step 1, collecting a plurality of images of good products;
step 2, collecting a plurality of defective images;
step 3, deploying defect detection and starting on line, and intelligently matching the detection mode through the good product image and the defective product image by the intelligent mode self-adaptive part;
step 4, one-stop normal defect detection and parallel calculation acceleration are carried out;
step 5, storing the defect detection result and sending detection information to the control part in a communication way;
step 6, the control part receives the detection information and sends a related control signal;
and 7, the defect identification executing part receives the control signal sent by the control part and makes a corresponding executing action.
3. The system of claim 1, wherein the resolution of the images involved in the one-stop defect detection algorithm must be consistent, otherwise normal defect detection is not possible; if the resolution ratios of the collected images are not consistent in practical application, defect detection is carried out through advanced resolution ratio unified conversion, and the method comprises the following steps:
Figure DEST_PATH_IMAGE041
，
Figure 73358DEST_PATH_IMAGE042
，
wherein the content of the first and second substances,
Figure DEST_PATH_IMAGE043
is the height of the acquired image;
Figure 671829DEST_PATH_IMAGE044
width of the acquired image;
Figure DEST_PATH_IMAGE045
to obtain the image height;
Figure 423885DEST_PATH_IMAGE046
to obtain the image width;
Figure DEST_PATH_IMAGE047
to input good images.
4. The one-stop intelligent defect detection system according to claim 1, wherein the parallel acceleration algorithm module is accelerated by adopting GPU hardware and a CUDA acceleration algorithm library in an industrial personal computer environment; the parallel acceleration algorithm module performs parallel acceleration by using an FPGA chip under an embedded system environment.
5. The one-stop intelligent defect detection system of claim 4, wherein the step of using the CUDA acceleration algorithm library comprises:
step 3.1, allocating host memories and initializing data;
step 3.2, allocating device memories, and copying data to the devices from the host;
3.3, calling a kernel function of the CUDA to complete specified operation on the device;
step 3.4, copying the operation result on the device to a host;
and 3.5, releasing the memory allocated on the device and the host.
CN202011099840.XA 2020-10-15 2020-10-15 One-stop intelligent defect detection system Active CN111929311B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011099840.XA CN111929311B (en) 2020-10-15 2020-10-15 One-stop intelligent defect detection system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011099840.XA CN111929311B (en) 2020-10-15 2020-10-15 One-stop intelligent defect detection system

Publications (2)

Publication Number Publication Date
CN111929311A CN111929311A (en) 2020-11-13
CN111929311B true CN111929311B (en) 2021-01-05

Family

ID=73334789

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011099840.XA Active CN111929311B (en) 2020-10-15 2020-10-15 One-stop intelligent defect detection system

Country Status (1)

Country Link
CN (1) CN111929311B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116773528B (en) * 2023-05-16 2024-05-10 安徽省高速公路试验检测科研中心有限公司 Visual defect detection method and system for candidate region

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102981467A (en) * 2012-11-13 2013-03-20 上海交通大学 One-stop general information integration monitoring system
CN104156913A (en) * 2014-07-15 2014-11-19 电子科技大学 Automatic optic inspection method for surface defects of metal cylindrical workpieces
WO2016092640A1 (en) * 2014-12-10 2016-06-16 株式会社 日立ハイテクノロジーズ Defect observation device and defect observation method
JP2018205005A (en) * 2017-05-31 2018-12-27 株式会社キーエンス Image inspection device
CN110766664A (en) * 2019-09-29 2020-02-07 杭州电子科技大学 Method for detecting appearance defective products of electronic components based on deep learning
CN110895716A (en) * 2018-09-12 2020-03-20 发那科株式会社 Inspection apparatus and machine learning method
CN111476306A (en) * 2020-04-10 2020-07-31 腾讯科技（深圳）有限公司 Object detection method, device, equipment and storage medium based on artificial intelligence

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9726617B2 (en) * 2013-06-04 2017-08-08 Kla-Tencor Corporation Apparatus and methods for finding a best aperture and mode to enhance defect detection

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102981467A (en) * 2012-11-13 2013-03-20 上海交通大学 One-stop general information integration monitoring system
CN104156913A (en) * 2014-07-15 2014-11-19 电子科技大学 Automatic optic inspection method for surface defects of metal cylindrical workpieces
WO2016092640A1 (en) * 2014-12-10 2016-06-16 株式会社 日立ハイテクノロジーズ Defect observation device and defect observation method
JP2018205005A (en) * 2017-05-31 2018-12-27 株式会社キーエンス Image inspection device
CN110895716A (en) * 2018-09-12 2020-03-20 发那科株式会社 Inspection apparatus and machine learning method
CN110766664A (en) * 2019-09-29 2020-02-07 杭州电子科技大学 Method for detecting appearance defective products of electronic components based on deep learning
CN111476306A (en) * 2020-04-10 2020-07-31 腾讯科技（深圳）有限公司 Object detection method, device, equipment and storage medium based on artificial intelligence

Also Published As

Publication number Publication date
CN111929311A (en) 2020-11-13

Similar Documents

Publication Publication Date Title
CN111598861B (en) Improved Faster R-CNN model-based non-uniform texture small defect detection method
CN114283111A (en) Qualitative or quantitative characterization of the surface of a coating
CN109492638A (en) Method for text detection, device and electronic equipment
CN110598609A (en) Weak supervision target detection method based on significance guidance
CN103439348B (en) Remote controller key defect detection method based on difference image method
CN108305243A (en) A kind of magnetic tile surface defect detection method based on deep learning
CN111080622A (en) Neural network training method, workpiece surface defect classification and detection method and device
JP7316731B2 (en) Systems and methods for detecting and classifying patterns in images in vision systems
CN112507815A (en) Artificial intelligence image recognition algorithm and system for pointer instrument panel scale
CN113034488B (en) Visual inspection method for ink-jet printed matter
CN111929311B (en) One-stop intelligent defect detection system
CN110930356A (en) Industrial two-dimensional code reference-free quality evaluation system and method
CN114266764A (en) Character integrity detection method and device for printed label
CN114372980A (en) Industrial defect detection method and system
CN114743102A (en) Furniture board oriented flaw detection method, system and device
CN115908988A (en) Defect detection model generation method, device, equipment and storage medium
CN113780484B (en) Industrial product defect detection method and device
CN109087292B (en) Thermal transfer film glue deviation detection method based on machine vision
CN110508510A (en) A kind of plastic pump defect inspection method, apparatus and system
CN114078127A (en) Object defect detection and counting method, device, equipment and storage medium
Ma et al. A hierarchical attention detector for bearing surface defect detection
CN110533660B (en) Method for detecting silk-screen defect of electronic product shell
CN112101060A (en) Two-dimensional code positioning method based on translation invariance and small-area template matching
CN116681677A (en) Lithium battery defect detection method, device and system
Xu et al. Highlight detection and removal method based on bifurcated-CNN

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant