CN111929311A - One-stop intelligent defect detection system - Google Patents

One-stop intelligent defect detection system Download PDF

Info

Publication number
CN111929311A
CN111929311A CN202011099840.XA CN202011099840A CN111929311A CN 111929311 A CN111929311 A CN 111929311A CN 202011099840 A CN202011099840 A CN 202011099840A CN 111929311 A CN111929311 A CN 111929311A
Authority
CN
China
Prior art keywords
image
defect
defect detection
detection
stop
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202011099840.XA
Other languages
Chinese (zh)
Other versions
CN111929311B (en
Inventor
郑秀征
叶振飞
王英利
冯龙申
梁长国
朱超平
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
BEIJING ZODNGOC AUTOMATIC TECHNOLOGY CO LTD
Original Assignee
BEIJING ZODNGOC AUTOMATIC TECHNOLOGY CO LTD
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by BEIJING ZODNGOC AUTOMATIC TECHNOLOGY CO LTD filed Critical BEIJING ZODNGOC AUTOMATIC TECHNOLOGY CO LTD
Priority to CN202011099840.XA priority Critical patent/CN111929311B/en
Publication of CN111929311A publication Critical patent/CN111929311A/en
Application granted granted Critical
Publication of CN111929311B publication Critical patent/CN111929311B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G01N2021/8887—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges based on image processing techniques

Landscapes

  • Chemical & Material Sciences (AREA)
  • Biochemistry (AREA)
  • Pathology (AREA)
  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Immunology (AREA)
  • Analytical Chemistry (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Signal Processing (AREA)
  • Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)

Abstract

The invention relates to a one-stop intelligent defect detection system, which comprises an image acquisition part, an intelligent mode self-adaption part, a one-stop defect detection part, a parallel acceleration part, a defect storage and communication part, a control part and a defect identification execution part. The invention can detect all defect types covered in the visual field range in a one-stop mode and give the position information of the defect area; and a complex defect detection tool is not required to be used for carrying out a combined and superposed defect detection means. The invention can carry out self-adaptive detection mode matching according to the material quality and the detection requirement of the detected material so as to achieve the optimal defect detection effect. The one-stop defect detection algorithm is suitable for parallel acceleration calculation, and can easily perform one-stop online, rapid and real-time defect detection by introducing a parallel acceleration part. The one-stop defect detection algorithm has very strong robustness to environmental factors such as illumination conditions and the like. The most painful environmental problem for industrial visual inspection is light exposure.

Description

One-stop intelligent defect detection system
Technical Field
The invention relates to a one-stop intelligent defect detection system, and belongs to the technical field of intelligent defect detection.
Background
At present, most of the market share of the demand of defect detection is distributed in the industrial production field, and the defect detection of most industries in the industrial production field is still in the artificial naked eye detection stage, so that enterprises need to invest a large amount of manpower, material resources and financial resources to ensure the quality of delivered products. At present, the mainstream visual defect detection system in the market mostly adopts a complex defect detection tool to implement a combined and superposed detection means; or a special detection tool is developed aiming at a single industry, and the detection tool is poor in universality.
Therefore, the patent provides a one-stop intelligent defect detection system, which can carry out intelligent detection mode matching according to the material to be detected, and can easily carry out one-stop online, quick and real-time defect detection by introducing a parallel acceleration part.
Disclosure of Invention
Aiming at the defects in the prior art, the invention provides a one-stop intelligent defect detection system, which has the following specific technical scheme:
a one-stop intelligent defect detection system, comprising:
the image acquisition part consists of a camera, a lens and a lighting subsystem, and is used for acquiring an original image of the detected material;
the intelligent mode self-adaptive part is realized by a detection mode self-adaptive algorithm, and the detection mode self-adaptive algorithm realizes closed-loop control by constructing a defect detection result quality evaluation function;
the one-stop defect detection part is realized by a one-stop defect detection algorithm, and the one-stop defect detection algorithm is used for realizing all defect types and defect position information covered in the camera view field;
the parallel acceleration part comprises a GPU or FPGA chip and a parallel acceleration algorithm module;
the defect storage and communication part is realized by a database and a communication module and is used for storing the defect detection result in real time;
a control section for receiving a defect detection result;
and the defect identification executing part is used for receiving the control signal sent by the control part and making a corresponding action to carry out defect identification.
As an improvement of the above technical solution, the operation flow of the one-stop intelligent defect detecting system includes the following steps:
step 1, collecting a plurality of images of good products;
step 2, collecting a plurality of defective images;
step 3, deploying defect detection and starting on line, and intelligently matching the detection mode through the good product image and the defective product image by the intelligent mode self-adaptive part;
step 4, one-stop normal defect detection and parallel calculation acceleration are carried out;
step 5, storing the defect detection result and sending detection information to the control part in a communication way;
step 6, the control part receives the detection information and sends a related control signal;
and 7, the defect identification executing part receives the control signal sent by the control part and makes a corresponding executing action.
As an improvement of the above technical solution, the resolution of the images involved in the one-stop defect detection algorithm must be consistent, otherwise normal defect detection cannot be performed; if the resolution ratios of the collected images are not consistent in practical application, defect detection is carried out through advanced resolution ratio unified conversion, and the method comprises the following steps:
Figure 55493DEST_PATH_IMAGE001
，
Figure 379158DEST_PATH_IMAGE002
，
wherein the content of the first and second substances,
Figure 955633DEST_PATH_IMAGE003
is the height of the acquired image;
Figure 6634DEST_PATH_IMAGE004
width of the acquired image;
Figure 555427DEST_PATH_IMAGE005
to obtain the image height;
Figure 174627DEST_PATH_IMAGE006
to obtain the image width;
Figure 848185DEST_PATH_IMAGE007
to input good images.
As an improvement of the above technical solution, the detection mode adaptive algorithm includes the following steps:
step 1.1, CollectionnOpening a good product image of the detected material,nthe collected good product images are weighted and fused to obtain a good product template image as a positive integer
Figure 578244DEST_PATH_IMAGE008
，
CollectingmOpening a defective product image of the detected material,mis a positive integer, and the acquired defective product images are weighted and fused to obtain a defective product template image
Figure 981543DEST_PATH_IMAGE009
，
Figure 896279DEST_PATH_IMAGE010
，
Figure 385029DEST_PATH_IMAGE011
，
Wherein the content of the first and second substances,irepresenting the number of the collected images;ncollecting the total number of good images;mthe total number of the collected defective images is counted;
Figure 918778DEST_PATH_IMAGE012
the images are collected for different good products;
Figure 848688DEST_PATH_IMAGE013
the collected images of different defective products;
Figure 12953DEST_PATH_IMAGE008
obtaining a good product template image through weighted fusion;
Figure 51316DEST_PATH_IMAGE014
obtaining a defective product template image through weighted fusion; g () is a weighted fusion operator;
step 1.2, selecting a detection mode
Figure 60861DEST_PATH_IMAGE015
To good product template image
Figure 625703DEST_PATH_IMAGE008
According to a mode
Figure 492028DEST_PATH_IMAGE015
Performing feature extraction operation to obtain feature images of good-quality template images
Figure 955370DEST_PATH_IMAGE016
(ii) a For defective product template image
Figure 706289DEST_PATH_IMAGE014
According to a mode
Figure 938687DEST_PATH_IMAGE015
Performing feature extraction operation to obtain feature image of defective product template image
Figure 507071DEST_PATH_IMAGE017
(ii) a To be detected image
Figure 192131DEST_PATH_IMAGE018
According to a mode
Figure 996007DEST_PATH_IMAGE015
Performing feature extraction operation to obtain feature image of the detected image
Figure 82912DEST_PATH_IMAGE019
；
Figure 822198DEST_PATH_IMAGE020
，
Figure 994553DEST_PATH_IMAGE021
，
Figure 87274DEST_PATH_IMAGE022
，
Wherein the content of the first and second substances,
Figure 294265DEST_PATH_IMAGE023
indicating a detection mode
Figure 204452DEST_PATH_IMAGE015
Corresponding feature extraction operators;
step 1.3, detecting mode
Figure 864103DEST_PATH_IMAGE015
Characteristic image of
Figure 744204DEST_PATH_IMAGE016
,
Figure 602438DEST_PATH_IMAGE019
Carrying out matrix subtraction and solving the absolute value of the matrix subtraction to obtain a difference characteristic image
Figure 621210DEST_PATH_IMAGE024
(ii) a For difference characteristic image
Figure 705840DEST_PATH_IMAGE024
Performing small kernel convolution operation to obtainObtaining the defect position, and marking the defect position to the detected image
Figure 468260DEST_PATH_IMAGE018
Corresponding to the spatial position to obtain a detection identification image
Figure 181001DEST_PATH_IMAGE025
；
Figure 292045DEST_PATH_IMAGE026
，
Figure 926289DEST_PATH_IMAGE027
，
Wherein the content of the first and second substances,Conv.() Representing a convolution identification operator, carrying out convolution operation on the difference value characteristics to obtain a defect position, and restoring the defect position information to the detected image to carry out identification;
step 1.3, exhausting all detection modes
Figure 289137DEST_PATH_IMAGE015
And repeating the steps 1.1 to 1.3 to obtain an optimal mode
Figure 325226DEST_PATH_IMAGE028
；
Figure 357905DEST_PATH_IMAGE029
，
Figure 276182DEST_PATH_IMAGE030
，
Figure 380404DEST_PATH_IMAGE031
，
Figure 215809DEST_PATH_IMAGE032
，
Wherein the content of the first and second substances,
Figure 747284DEST_PATH_IMAGE015
in order to detect the mode of the light,
Figure 152858DEST_PATH_IMAGE033
represents an optimal quality assessment value;f 1()、f 2() Representing a quality assessment operator;
Figure 60771DEST_PATH_IMAGE034
、
Figure 743556DEST_PATH_IMAGE035
in order to be the weighting coefficients,
Figure 445933DEST_PATH_IMAGE036
；
Figure 338803DEST_PATH_IMAGE037
representing a positive integer.
As an improvement of the above technical solution, the one-stop defect detection algorithm includes the following steps:
step 2.1, deploying actual detection on line and using an optimal mode
Figure 784827DEST_PATH_IMAGE028
Calculating the images of the good templates
Figure 571387DEST_PATH_IMAGE008
Characteristic image of
Figure 444665DEST_PATH_IMAGE038
For real-time single detected image
Figure 824831DEST_PATH_IMAGE039
According to the optimal mode
Figure 808967DEST_PATH_IMAGE028
Performing feature extraction to obtain image
Figure 466345DEST_PATH_IMAGE040
，
Figure 510524DEST_PATH_IMAGE041
，
Figure 377986DEST_PATH_IMAGE042
，
Wherein the content of the first and second substances,
Figure 165813DEST_PATH_IMAGE043
is an optimal mode
Figure 926965DEST_PATH_IMAGE028
Corresponding feature extraction descriptors;
step 2.2, optimizing the mode
Figure 142045DEST_PATH_IMAGE028
Characteristic image of
Figure 231224DEST_PATH_IMAGE038
,
Figure 517029DEST_PATH_IMAGE040
Carrying out matrix subtraction and solving the absolute value of the matrix subtraction to obtain a difference characteristic image
Figure 840694DEST_PATH_IMAGE044
(ii) a For difference characteristic image
Figure 620431DEST_PATH_IMAGE044
Performing small kernel convolution and identification operation to obtain defect detection result image
Figure 812378DEST_PATH_IMAGE045
(ii) a Defect detection result image
Figure 95592DEST_PATH_IMAGE045
Including the position information of all the defective areas in the image, andcarrying out differentiated color highlight identification on the defect position;
Figure 105005DEST_PATH_IMAGE046
，
Figure 106459DEST_PATH_IMAGE047
。
as an improvement of the technical scheme, the parallel acceleration algorithm module is accelerated by adopting GPU hardware and a CUDA acceleration algorithm library in an industrial personal computer environment; the parallel acceleration algorithm module performs parallel acceleration by using an FPGA chip under an embedded system environment.
As an improvement of the technical scheme, the step of using the CUDA acceleration algorithm library comprises the following steps:
step 3.1, allocating host memories and initializing data;
step 3.2, allocating device memories, and copying data to the devices from the host;
3.3, calling a kernel function of the CUDA to complete specified operation on the device;
step 3.4, copying the operation result on the device to a host;
and 3.5, releasing the memory allocated on the device and the host.
The invention has the beneficial effects that:
1) one-stop detection: the invention can detect all defect types covered in the visual field range in a one-stop mode and give the position information of the defect area; and a complex defect detection tool is not required to be used for carrying out a combined and superposed defect detection means.
2) Intelligent mode adaptation: the invention can carry out self-adaptive detection mode matching according to the material quality and the detection requirement of the detected material so as to achieve the optimal defect detection effect. The user can also carry out secondary development and add a user-defined detection mode, and the user-defined detection mode can also support the matching of the self-adaptive detection mode.
3) And quick: the one-stop defect detection algorithm is suitable for parallel acceleration calculation, and can easily perform one-stop online, rapid and real-time defect detection by introducing a parallel acceleration part.
4) High robustness: the one-stop defect detection algorithm has very strong robustness to environmental factors such as illumination conditions and the like. The most painful environmental problem for industrial visual inspection is light exposure.
Drawings
FIG. 1 is a flow chart of a one-stop intelligent defect detection system according to the present invention;
FIG. 2 is a block diagram of an algorithm module of the one-stop intelligent defect detection system according to the present invention;
FIG. 3 is an original image for product inspection;
FIG. 4 is a graph of high intensity illumination condition defect detection;
fig. 5 is a diagram of the detection effect of low-brightness lighting conditions.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
Example 1
As shown in fig. 1, the one-stop intelligent defect detection system includes:
the image acquisition part consists of a camera, a lens and a lighting subsystem, and is used for acquiring an original image of the detected material; the method can acquire the high-quality and high-contrast original image of the detected material.
The intelligent mode self-adaptive part is realized by a detection mode self-adaptive algorithm, the detection mode self-adaptive algorithm realizes closed-loop control by constructing a defect detection result quality evaluation function, and the intelligent selection and switching of the detection modes can be realized aiming at different detected materials.
The one-stop defect detection part is realized by a one-stop defect detection algorithm, and the one-stop defect detection algorithm is used for realizing all defect types and defect position information covered in the camera view field; meanwhile, the one-stop defect detection algorithm can intelligently select different detection modes, including a user-defined detection mode added by secondary development of a user.
The parallel acceleration part comprises a GPU or FPGA chip and a parallel acceleration algorithm module; the real-time defect detection requirement of the high-resolution image can be easily met through parallel computing and acceleration.
The defect storage and communication part is realized by a database and a communication module and is used for storing the defect detection result in real time; the detection result may also be communicated to the control section.
And a control section for receiving the defect detection result.
And the defect identification executing part is used for receiving the control signal sent by the control part and making a corresponding action to carry out defect identification.
Example 2
The operation flow of the one-stop intelligent defect detection system comprises the following steps:
step 1, collecting a plurality of images of good products;
step 2, collecting a plurality of defective images;
step 3, deploying defect detection and starting on line, and intelligently matching the detection mode through the good product image and the defective product image by the intelligent mode self-adaptive part;
step 4, one-stop normal defect detection and parallel calculation acceleration are carried out;
step 5, storing the defect detection result and sending detection information to the control part in a communication way;
step 6, the control part receives the detection information and sends a related control signal;
and 7, the defect identification executing part receives the control signal sent by the control part and makes a corresponding executing action.
The defect mark executing part can be specifically a cylinder with a quick-dry marking pen, an ink jet printer, a coding machine and other mechanisms. The actions performed may be: and marking characters such as 'NG' or 'defective product' and 'unqualified product' on the defective product.
Example 3
The one-stop defect detection algorithm is realized by the following steps:
the one-stop defect detection algorithm can select different detection modes to adapt to detected materials of different materials, the detection mode self-adaptive algorithm can automatically select an optimal detection mode, and the parallel acceleration algorithm module provides strong calculation support for the one-stop defect detection algorithm.
Because the image acquisition part is determined, the acquired image resolutions are consistent, the image resolutions (height and width) related in the one-stop defect detection algorithm are required to be consistent, otherwise, normal defect detection cannot be carried out; if the resolution ratios of the collected images are not consistent in practical application, defect detection is carried out through advanced resolution ratio unified conversion, and the method comprises the following steps:
Figure 102097DEST_PATH_IMAGE001
，
Figure 239817DEST_PATH_IMAGE002
，
wherein the content of the first and second substances,
Figure 170864DEST_PATH_IMAGE003
is the height of the acquired image;
Figure 659614DEST_PATH_IMAGE004
width of the acquired image;
Figure 193364DEST_PATH_IMAGE005
to obtain the image height;
Figure 451170DEST_PATH_IMAGE006
to obtain the image width;
Figure 536806DEST_PATH_IMAGE007
to input good images.
Example 4
The detection mode self-adaptive algorithm intelligently selects and switches the optimal detection mode of the detected material.
As shown in fig. 2, the detection mode adaptive algorithm includes the following steps:
step 1.1, CollectionnOpening a good product image of the detected material,nthe collected good product images are weighted and fused to obtain a good product template image as a positive integer
Figure 309590DEST_PATH_IMAGE008
，
CollectingmOpening a defective product image of the detected material,mis a positive integer, and the acquired defective product images are weighted and fused to obtain a defective product template image
Figure 584714DEST_PATH_IMAGE009
，
Figure 900289DEST_PATH_IMAGE010
，
Figure 969876DEST_PATH_IMAGE011
，
Wherein the content of the first and second substances,irepresenting the number of the collected images;ncollecting the total number of good images;mthe total number of the collected defective images is counted;
Figure 495535DEST_PATH_IMAGE012
the images are collected for different good products;
Figure 308770DEST_PATH_IMAGE013
the collected images of different defective products;
Figure 728119DEST_PATH_IMAGE008
obtaining a good product template image through weighted fusion;
Figure 968608DEST_PATH_IMAGE014
obtaining a defective product template image through weighted fusion; g () is a weighted fusion operator;
step 1.2,Selecting a detection mode
Figure 981563DEST_PATH_IMAGE048
To good product template image
Figure 598489DEST_PATH_IMAGE008
According to a mode
Figure 623077DEST_PATH_IMAGE048
Performing feature extraction operation to obtain feature images of good-quality template images
Figure 300046DEST_PATH_IMAGE016
(ii) a For defective product template image
Figure 534718DEST_PATH_IMAGE014
According to a mode
Figure 689756DEST_PATH_IMAGE048
Performing feature extraction operation to obtain feature image of defective product template image
Figure 83697DEST_PATH_IMAGE017
(ii) a To be detected image
Figure 665988DEST_PATH_IMAGE018
According to a mode
Figure 387957DEST_PATH_IMAGE048
Performing feature extraction operation to obtain feature image of the detected image
Figure 346685DEST_PATH_IMAGE019
；
Figure 345865DEST_PATH_IMAGE020
，
Figure 161375DEST_PATH_IMAGE021
，
Figure 573901DEST_PATH_IMAGE022
，
Wherein the content of the first and second substances,
Figure 70742DEST_PATH_IMAGE023
indicating a detection mode
Figure 173696DEST_PATH_IMAGE048
Corresponding feature extraction operators;
step 1.3, detecting mode
Figure 363369DEST_PATH_IMAGE015
Characteristic image of
Figure 59929DEST_PATH_IMAGE016
,
Figure 360461DEST_PATH_IMAGE019
Carrying out matrix subtraction and solving the absolute value of the matrix subtraction to obtain a difference characteristic image
Figure 68654DEST_PATH_IMAGE024
(ii) a For difference characteristic image
Figure 225966DEST_PATH_IMAGE024
Performing small kernel convolution operation to obtain defect position, and marking the defect position to the detected image
Figure 347505DEST_PATH_IMAGE018
Corresponding to the spatial position to obtain a detection identification image
Figure 373099DEST_PATH_IMAGE025
；
Figure 263695DEST_PATH_IMAGE026
，
Figure 857487DEST_PATH_IMAGE027
，
Wherein the content of the first and second substances,Conv.() Representing a convolution identification operator, carrying out convolution operation on the difference value characteristics to obtain a defect position, and restoring the defect position information to the detected image to carry out identification;
step 1.3, exhausting all detection modes
Figure 200744DEST_PATH_IMAGE015
And repeating the steps 1.1 to 1.3 to obtain an optimal mode
Figure 46340DEST_PATH_IMAGE028
；
Figure 791442DEST_PATH_IMAGE029
，
Figure 556136DEST_PATH_IMAGE030
，
Figure 386689DEST_PATH_IMAGE031
，
Figure 754085DEST_PATH_IMAGE032
，
Wherein the content of the first and second substances,
Figure 619273DEST_PATH_IMAGE015
for the detection mode, different detection modes correspond to different feature extraction methods, here, an equivalent image feature descriptor extraction function can be used, and by constructing a quality evaluation function, an optimal detection mode is selected from a plurality of detection modes
Figure 554868DEST_PATH_IMAGE049
。
Wherein the content of the first and second substances,
Figure 872716DEST_PATH_IMAGE033
represents an optimal quality assessment value;f 1()、f 2() Representing a quality assessment operator;
Figure 794536DEST_PATH_IMAGE034
、
Figure 514230DEST_PATH_IMAGE035
for weighting factors, empirical values may be taken
Figure 620727DEST_PATH_IMAGE036
；
Figure 425872DEST_PATH_IMAGE037
Representing a positive integer.
Example 5
The one-stop defect detection algorithm comprises the following steps:
step 2.1, deploying actual detection on line and using an optimal mode
Figure 400650DEST_PATH_IMAGE028
Calculating the images of the good templates
Figure 974851DEST_PATH_IMAGE008
Characteristic image of
Figure 252248DEST_PATH_IMAGE038
For real-time single detected image
Figure 279110DEST_PATH_IMAGE039
According to the optimal mode
Figure 808311DEST_PATH_IMAGE028
Performing feature extraction to obtain image
Figure 237019DEST_PATH_IMAGE040
，
Figure 685318DEST_PATH_IMAGE041
，
Figure 465055DEST_PATH_IMAGE042
，
Wherein the content of the first and second substances,
Figure 781636DEST_PATH_IMAGE043
is an optimal mode
Figure 64849DEST_PATH_IMAGE028
Corresponding feature extraction descriptors;
step 2.2, optimizing the mode
Figure 949629DEST_PATH_IMAGE028
Characteristic image of
Figure 951083DEST_PATH_IMAGE038
,
Figure 556508DEST_PATH_IMAGE040
Carrying out matrix subtraction and solving the absolute value of the matrix subtraction to obtain a difference characteristic image
Figure 959807DEST_PATH_IMAGE044
(ii) a For difference characteristic image
Figure 15488DEST_PATH_IMAGE044
Performing small kernel convolution and identification operation to obtain defect detection result image
Figure 238659DEST_PATH_IMAGE045
(ii) a Defect detection result image
Figure 162621DEST_PATH_IMAGE045
The method comprises the steps of containing position information of all defect areas in an image, and carrying out differentiated color highlight identification on the defect positions;
Figure 420427DEST_PATH_IMAGE046
，
Figure 381430DEST_PATH_IMAGE047
。
since the algorithm is based on the same detection mode
Figure 357476DEST_PATH_IMAGE028
Extracting features, and performing matrix subtraction to obtain difference feature image
Figure 570283DEST_PATH_IMAGE044
Therefore, the algorithm has natural high robustness to interference of environmental factors such as light.
The robustness of the algorithm to illumination can be seen from the overall flow of image processing (PipeLine). 3-5 are graphs providing a comparison of the detection effect of the algorithm before and after a change in ambient light; as can be seen from fig. 3-5: the algorithm is proved to be applicable to different illumination conditions.
Example 6
The one-stop defect detection algorithm applies a large number of matrix addition, subtraction, multiplication and division operations, and is suitable for parallel accelerated calculation. The parallel acceleration algorithm module adopts GPU hardware and a CUDA acceleration algorithm library to accelerate in an industrial personal computer environment; the parallel acceleration algorithm module performs parallel acceleration by using an FPGA chip under an embedded system environment.
The steps of using the CUDA acceleration algorithm library are as follows:
step 3.1, allocating host memories and initializing data;
step 3.2, allocating device memories, and copying data to the devices from the host;
3.3, calling a kernel function of the CUDA to complete specified operation on the device;
step 3.4, copying the operation result on the device to a host;
and 3.5, releasing the memory allocated on the device and the host.
Wherein, host can be regarded as a logic control unit, usually a CPU; device can be thought of as a parallel computing acceleration device, commonly referred to as a GPU.
In the above embodiments, the present invention provides a one-stop intelligent defect inspection system, which can inspect all defect types and defect position information covered in the field of view in one-stop manner without using the complex defect inspection tools to perform the combined and overlapped inspection means. The system can intelligently switch the detection modes according to the material attribute and the detection requirement of the detected material, achieves the requirement of being compatible with different material defect detection, and can also support a user to carry out secondary development and add a self-defined detection mode. The one-stop defect detection method is suitable for parallel accelerated computation, and the speed bottleneck problem of high-resolution image online deployment real-time detection is perfectly solved by adding a parallel computation part. For example: 500 ten thousand color images, wherein 500ms is needed for processing one image under the condition of no acceleration, the frame rate is 2fps, the product jump distance (the length of one product) is assumed to be 160mm, and the running speed is 19.2 m/min; acceleration was 200ms, 5fps, assuming a product jump (length of one piece of product) of 160mm, and a running speed of 48 m/min.
The invention has the following advantages:
1) one-stop detection: the invention can detect all defect types (defect types comprise hole blockage, incomplete waste discharge, scratch, dirt, foreign matters, glue deficiency, glue overflow, bubbles, folds, sheet overlapping and the like, most defect types have the characteristics of random positions, different sizes and shapes and the like) covered in a visual field in a one-stop mode, and provides position information of a defect area. And a complex defect detection tool is not required to be used for carrying out a combined and superposed defect detection means.
The core detection idea of the invention is to adopt a full-width characteristic image comparison mode to carry out 'full-width' comparison on a standard qualified product and a detected product, so that all defects covered by the whole image can be detected in a one-stop mode. (full comparison: the meaning of the full characteristic image is different according to the different detection modes, the full characteristic image can be simply understood as a characteristic diagram, the characteristic extraction methods under different modes are different, developers can add the characteristic extraction methods, and the method can provide a brand-new image processing flow of surface defect detection, namely standard sample collection, positioning, full characteristic diagram comparison and small convolution kernel identification of defect positions).
2) Intelligent mode adaptation: the invention can carry out self-adaptive detection mode matching according to the material quality and the detection requirement of the detected material so as to achieve the optimal defect detection effect. The user can also carry out secondary development and add a user-defined detection mode, and the user-defined detection mode can also support the matching of the self-adaptive detection mode.
3) And quick: the one-stop defect detection algorithm is suitable for parallel acceleration calculation, and can easily perform one-stop online, rapid and real-time defect detection by introducing a parallel acceleration part.
4) High robustness: the one-stop defect detection algorithm has very strong robustness to environmental factors such as illumination conditions and the like. The most painful environmental problem for industrial visual inspection is light exposure.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.

Claims (7)

1. A one-stop intelligent defect detection system, comprising:
the image acquisition part consists of a camera, a lens and a lighting subsystem, and is used for acquiring an original image of the detected material;
the intelligent mode self-adaptive part is realized by a detection mode self-adaptive algorithm, and the detection mode self-adaptive algorithm realizes closed-loop control by constructing a defect detection result quality evaluation function;
the one-stop defect detection part is realized by a one-stop defect detection algorithm, and the one-stop defect detection algorithm is used for realizing all defect types and defect position information covered in the camera view field;
the parallel acceleration part comprises a GPU or FPGA chip and a parallel acceleration algorithm module;
the defect storage and communication part is realized by a database and a communication module and is used for storing the defect detection result in real time;
a control section for receiving a defect detection result;
and the defect identification executing part is used for receiving the control signal sent by the control part and making a corresponding action to carry out defect identification.
2. The one-stop intelligent defect detecting system of claim 1, wherein the operation flow of the one-stop intelligent defect detecting system comprises the following steps:
step 1, collecting a plurality of images of good products;
step 2, collecting a plurality of defective images;
step 3, deploying defect detection and starting on line, and intelligently matching the detection mode through the good product image and the defective product image by the intelligent mode self-adaptive part;
step 4, one-stop normal defect detection and parallel calculation acceleration are carried out;
step 5, storing the defect detection result and sending detection information to the control part in a communication way;
step 6, the control part receives the detection information and sends a related control signal;
and 7, the defect identification executing part receives the control signal sent by the control part and makes a corresponding executing action.
3. The system of claim 1, wherein the resolution of the images involved in the one-stop defect detection algorithm must be consistent, otherwise normal defect detection is not possible; if the resolution ratios of the collected images are not consistent in practical application, defect detection is carried out through advanced resolution ratio unified conversion, and the method comprises the following steps:
Figure 667726DEST_PATH_IMAGE001
，
Figure 514459DEST_PATH_IMAGE002
，
wherein the content of the first and second substances,
Figure 334778DEST_PATH_IMAGE003
is the height of the acquired image;
Figure 532542DEST_PATH_IMAGE004
width of the acquired image;
Figure 345777DEST_PATH_IMAGE005
to obtain the image height;
Figure 578175DEST_PATH_IMAGE006
to obtain the image width;
Figure 553084DEST_PATH_IMAGE007
to input good images.
4. The one-stop intelligent defect detection system of claim 1, wherein said detection mode adaptive algorithm comprises the steps of:
step 1.1, CollectionnOpening a good product image of the detected material,nthe collected good product images are weighted and fused to obtain a good product template image as a positive integer
Figure 487411DEST_PATH_IMAGE008
,
CollectingmOpening a defective product image of the detected material,mis a positive integer, and the acquired defective product images are weighted and fused to obtain a defective product template image
Figure 104337DEST_PATH_IMAGE009
，
Figure 925663DEST_PATH_IMAGE010
，
Figure 71473DEST_PATH_IMAGE011
，
Wherein the content of the first and second substances,irepresenting the number of the collected images;ncollecting the total number of good images;mthe total number of the collected defective images is counted;
Figure 243829DEST_PATH_IMAGE012
the images are collected for different good products;
Figure 149599DEST_PATH_IMAGE013
the collected images of different defective products;
Figure 825431DEST_PATH_IMAGE008
obtaining a good product template image through weighted fusion;
Figure 673301DEST_PATH_IMAGE014
obtaining a defective product template image through weighted fusion; g () is a weighted fusion operator;
step 1.2, selecting a detection mode
Figure 67373DEST_PATH_IMAGE015
To good product template image
Figure 760523DEST_PATH_IMAGE008
According to a mode
Figure 71287DEST_PATH_IMAGE015
Performing feature extraction operation to obtain feature images of good-quality template images
Figure 824480DEST_PATH_IMAGE016
(ii) a For defective product template image
Figure 705848DEST_PATH_IMAGE014
According to a mode
Figure 202688DEST_PATH_IMAGE015
Performing feature extraction operation to obtain feature image of defective product template image
Figure 118692DEST_PATH_IMAGE017
(ii) a To be detected image
Figure 793518DEST_PATH_IMAGE018
According to a mode
Figure 427762DEST_PATH_IMAGE015
Performing feature extraction operation to obtain feature image of the detected image
Figure 728293DEST_PATH_IMAGE019
；
Figure 233224DEST_PATH_IMAGE020
，
Figure 328219DEST_PATH_IMAGE021
，
Figure 184179DEST_PATH_IMAGE022
，
Wherein the content of the first and second substances,
Figure 272090DEST_PATH_IMAGE023
indicating a detection mode
Figure 162685DEST_PATH_IMAGE015
Corresponding feature extraction operators;
step 1.3, detecting mode
Figure 428582DEST_PATH_IMAGE015
Characteristic image of
Figure 771838DEST_PATH_IMAGE016
,
Figure 148593DEST_PATH_IMAGE019
Carrying out matrix subtraction and solving the absolute value of the matrix subtraction to obtain a difference characteristic image
Figure 656146DEST_PATH_IMAGE024
(ii) a For difference characteristic image
Figure 92944DEST_PATH_IMAGE024
Performing small kernel convolution operation to obtain defect position, and marking the defect position to the detected image
Figure 657917DEST_PATH_IMAGE018
Corresponding to the spatial position to obtain a detection identification image
Figure 103942DEST_PATH_IMAGE025
；
Figure 703551DEST_PATH_IMAGE026
，
Figure 826096DEST_PATH_IMAGE027
，
Wherein the content of the first and second substances,Conv.() Representing a convolution identification operator, carrying out convolution operation on the difference value characteristics to obtain a defect position, and restoring the defect position information to the detected image to carry out identification;
step 1.3, exhausting all detection modes
Figure 612787DEST_PATH_IMAGE015
And repeating the steps 1.1 to 1.3 to obtain an optimal mode
Figure 596923DEST_PATH_IMAGE028
；
Figure 316618DEST_PATH_IMAGE029
，
Figure 95218DEST_PATH_IMAGE030
，
Figure 385516DEST_PATH_IMAGE031
，
Figure 907764DEST_PATH_IMAGE032
，
Wherein the content of the first and second substances,
Figure 481965DEST_PATH_IMAGE015
in order to detect the mode of the light,
Figure 431466DEST_PATH_IMAGE033
represents an optimal quality assessment value;f 1()、f 2() Representing a quality assessment operator;
Figure 458328DEST_PATH_IMAGE034
、
Figure 33535DEST_PATH_IMAGE035
in order to be the weighting coefficients,
Figure 462242DEST_PATH_IMAGE036
；
Figure 848224DEST_PATH_IMAGE037
representing a positive integer.
5. The one-stop intelligent defect detection system of claim 4, wherein the one-stop defect detection algorithm comprises the steps of:
step 2.1, deploying actual detection on line and using an optimal mode
Figure 96803DEST_PATH_IMAGE028
Calculating the images of the good templates
Figure 226433DEST_PATH_IMAGE008
Characteristic image of
Figure 260379DEST_PATH_IMAGE038
For real-time single detected image
Figure 551683DEST_PATH_IMAGE039
According to the optimal mode
Figure 553137DEST_PATH_IMAGE028
Performing feature extraction to obtain image
Figure 220879DEST_PATH_IMAGE040
，
Figure 624179DEST_PATH_IMAGE041
，
Figure 335652DEST_PATH_IMAGE042
，
Wherein the content of the first and second substances,
Figure 824402DEST_PATH_IMAGE043
is an optimal mode
Figure 295834DEST_PATH_IMAGE028
Corresponding feature extraction descriptors;
step 2.2, optimizing the mode
Figure 288061DEST_PATH_IMAGE028
Characteristic image of
Figure 186747DEST_PATH_IMAGE038
,
Figure 647946DEST_PATH_IMAGE040
Carrying out matrix subtraction and solving the absolute value of the matrix subtraction to obtain a difference characteristic image
Figure 923070DEST_PATH_IMAGE044
(ii) a For difference characteristic image
Figure 769803DEST_PATH_IMAGE044
Performing small kernel convolution and identification operation to obtain defect detection result image
Figure 104970DEST_PATH_IMAGE045
(ii) a Defect detection result image
Figure 37154DEST_PATH_IMAGE045
The method comprises the steps of containing position information of all defect areas in an image, and carrying out differentiated color highlight identification on the defect positions;
Figure 115968DEST_PATH_IMAGE046
，
Figure 66475DEST_PATH_IMAGE047
。
6. the one-stop intelligent defect detection system according to claim 1, wherein the parallel acceleration algorithm module is accelerated by adopting GPU hardware and a CUDA acceleration algorithm library in an industrial personal computer environment; the parallel acceleration algorithm module performs parallel acceleration by using an FPGA chip under an embedded system environment.
7. The one-stop intelligent defect detection system of claim 6, wherein the step of using the CUDA acceleration algorithm library comprises:
step 3.1, allocating host memories and initializing data;
step 3.2, allocating device memories, and copying data to the devices from the host;
3.3, calling a kernel function of the CUDA to complete specified operation on the device;
step 3.4, copying the operation result on the device to a host;
and 3.5, releasing the memory allocated on the device and the host.
CN202011099840.XA 2020-10-15 2020-10-15 One-stop intelligent defect detection system Active CN111929311B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011099840.XA CN111929311B (en) 2020-10-15 2020-10-15 One-stop intelligent defect detection system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011099840.XA CN111929311B (en) 2020-10-15 2020-10-15 One-stop intelligent defect detection system

Publications (2)

Publication Number Publication Date
CN111929311A true CN111929311A (en) 2020-11-13
CN111929311B CN111929311B (en) 2021-01-05

Family

ID=73334789

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011099840.XA Active CN111929311B (en) 2020-10-15 2020-10-15 One-stop intelligent defect detection system

Country Status (1)

Country Link
CN (1) CN111929311B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116773528A (en) * 2023-05-16 2023-09-19 安徽省高速公路试验检测科研中心有限公司 Visual defect detection method and system for candidate region

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102981467A (en) * 2012-11-13 2013-03-20 上海交通大学 One-stop general information integration monitoring system
CN104156913A (en) * 2014-07-15 2014-11-19 电子科技大学 Automatic optic inspection method for surface defects of metal cylindrical workpieces
US20140354983A1 (en) * 2013-06-04 2014-12-04 Kla-Tencor Corporation Apparatus and methods for finding a best aperture and mode to enhance defect detection
WO2016092640A1 (en) * 2014-12-10 2016-06-16 株式会社 日立ハイテクノロジーズ Defect observation device and defect observation method
JP2018205005A (en) * 2017-05-31 2018-12-27 株式会社キーエンス Image inspection device
CN110766664A (en) * 2019-09-29 2020-02-07 杭州电子科技大学 Method for detecting appearance defective products of electronic components based on deep learning
CN110895716A (en) * 2018-09-12 2020-03-20 发那科株式会社 Inspection apparatus and machine learning method
CN111476306A (en) * 2020-04-10 2020-07-31 腾讯科技（深圳）有限公司 Object detection method, device, equipment and storage medium based on artificial intelligence

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102981467A (en) * 2012-11-13 2013-03-20 上海交通大学 One-stop general information integration monitoring system
US20140354983A1 (en) * 2013-06-04 2014-12-04 Kla-Tencor Corporation Apparatus and methods for finding a best aperture and mode to enhance defect detection
CN104156913A (en) * 2014-07-15 2014-11-19 电子科技大学 Automatic optic inspection method for surface defects of metal cylindrical workpieces
WO2016092640A1 (en) * 2014-12-10 2016-06-16 株式会社 日立ハイテクノロジーズ Defect observation device and defect observation method
JP2018205005A (en) * 2017-05-31 2018-12-27 株式会社キーエンス Image inspection device
CN110895716A (en) * 2018-09-12 2020-03-20 发那科株式会社 Inspection apparatus and machine learning method
CN110766664A (en) * 2019-09-29 2020-02-07 杭州电子科技大学 Method for detecting appearance defective products of electronic components based on deep learning
CN111476306A (en) * 2020-04-10 2020-07-31 腾讯科技（深圳）有限公司 Object detection method, device, equipment and storage medium based on artificial intelligence

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116773528A (en) * 2023-05-16 2023-09-19 安徽省高速公路试验检测科研中心有限公司 Visual defect detection method and system for candidate region
CN116773528B (en) * 2023-05-16 2024-05-10 安徽省高速公路试验检测科研中心有限公司 Visual defect detection method and system for candidate region

Also Published As

Publication number Publication date
CN111929311B (en) 2021-01-05

Similar Documents

Publication Publication Date Title
CN111598861B (en) Improved Faster R-CNN model-based non-uniform texture small defect detection method
EP3971557A1 (en) Qualitative or quantitative characterization of a coating surface
CN108918536B (en) Tire mold surface character defect detection method, device, equipment and storage medium
CN111693534B (en) Surface defect detection method, model training method, device, equipment and medium
CN110598609A (en) Weak supervision target detection method based on significance guidance
Benedek et al. Solder paste scooping detection by multilevel visual inspection of printed circuit boards
CN109544522A (en) A kind of Surface Defects in Steel Plate detection method and system
CN103439348B (en) Remote controller key defect detection method based on difference image method
JP7316731B2 (en) Systems and methods for detecting and classifying patterns in images in vision systems
CN105894514A (en) Printed matter defect detection method and system based on GPU parallel operation
CN104200478B (en) Low-resolution touch screen image defect detection method based on sparse representation
CN112507815A (en) Artificial intelligence image recognition algorithm and system for pointer instrument panel scale
CN114240885B (en) Cloth flaw detection method based on improved Yolov4 network
CN114549507B (en) Improved Scaled-YOLOv fabric flaw detection method
CN111929311B (en) One-stop intelligent defect detection system
CN110599453A (en) Panel defect detection method and device based on image fusion and equipment terminal
CN114372980A (en) Industrial defect detection method and system
CN114266764A (en) Character integrity detection method and device for printed label
CN113034488A (en) Visual detection method of ink-jet printed matter
CN112819756A (en) PCB surface defect detection device and method
CN113705673A (en) Character detection method, device, equipment and storage medium
CN109087292B (en) Thermal transfer film glue deviation detection method based on machine vision
CN113780484A (en) Industrial product defect detection method and device
CN110508510A (en) A kind of plastic pump defect inspection method, apparatus and system
Ma et al. A hierarchical attention detector for bearing surface defect detection

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant