CN113894055A - Hardware surface defect detection and classification system and method based on machine vision - Google Patents
Hardware surface defect detection and classification system and method based on machine vision Download PDFInfo
- Publication number
- CN113894055A CN113894055A CN202111042160.9A CN202111042160A CN113894055A CN 113894055 A CN113894055 A CN 113894055A CN 202111042160 A CN202111042160 A CN 202111042160A CN 113894055 A CN113894055 A CN 113894055A
- Authority
- CN
- China
- Prior art keywords
- hardware
- area
- defect
- detection
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000007547 defect Effects 0.000 title claims abstract description 126
- 238000001514 detection method Methods 0.000 title claims abstract description 89
- 238000000034 method Methods 0.000 title claims abstract description 31
- 230000005540 biological transmission Effects 0.000 claims abstract description 8
- 238000006243 chemical reaction Methods 0.000 claims description 22
- 238000004422 calculation algorithm Methods 0.000 claims description 13
- 239000011159 matrix material Substances 0.000 claims description 9
- 238000012545 processing Methods 0.000 claims description 9
- 238000004364 calculation method Methods 0.000 claims description 6
- 239000000284 extract Substances 0.000 claims description 5
- 230000002950 deficient Effects 0.000 claims description 4
- 239000002131 composite material Substances 0.000 claims description 3
- 239000000047 product Substances 0.000 description 7
- 230000008569 process Effects 0.000 description 5
- 230000032258 transport Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 230000006872 improvement Effects 0.000 description 3
- 238000004519 manufacturing process Methods 0.000 description 3
- 238000005259 measurement Methods 0.000 description 3
- 238000007670 refining Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 208000019914 Mental Fatigue Diseases 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 239000012467 final product Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
- B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
- B07C5/00—Sorting according to a characteristic or feature of the articles or material being sorted, e.g. by control effected by devices which detect or measure such characteristic or feature; Sorting by manually actuated devices, e.g. switches
- B07C5/04—Sorting according to size
- B07C5/10—Sorting according to size measured by light-responsive means
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
- B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
- B07C5/00—Sorting according to a characteristic or feature of the articles or material being sorted, e.g. by control effected by devices which detect or measure such characteristic or feature; Sorting by manually actuated devices, e.g. switches
- B07C5/02—Measures preceding sorting, e.g. arranging articles in a stream orientating
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
- B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
- B07C5/00—Sorting according to a characteristic or feature of the articles or material being sorted, e.g. by control effected by devices which detect or measure such characteristic or feature; Sorting by manually actuated devices, e.g. switches
- B07C5/34—Sorting according to other particular properties
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/02—Measuring arrangements characterised by the use of optical techniques for measuring length, width or thickness
- G01B11/022—Measuring arrangements characterised by the use of optical techniques for measuring length, width or thickness by means of tv-camera scanning
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/22—Measuring arrangements characterised by the use of optical techniques for measuring depth
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/28—Measuring arrangements characterised by the use of optical techniques for measuring areas
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
- G01N2021/8854—Grading and classifying of flaws
- G01N2021/8874—Taking dimensions of defect into account
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
- G01N2021/8887—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges based on image processing techniques
Abstract
The invention discloses a hardware surface defect detection and classification system and method based on machine vision, relates to the technical field of hardware detection by machine vision, and aims to solve the problems of low efficiency, low automation degree and low accuracy of the conventional hardware which adopts manual detection. The method mainly comprises the following steps: the system comprises a transmission module, an acquisition module, a terminal computer, a control module and a sorting module; the conveying module is used for conveying the hardware from the feeding belt to the detection area and the sorting area; the acquisition module is used for acquiring hardware images; the terminal computer is used for receiving hardware arrival signals sent by the PLC control module, transmitting the signals to the camera, receiving hardware images returned by the camera, sending acquired image signals to the PLC control module, extracting and classifying defects of the hardware images by using the provided detection and classification method, and sending results to the PLC control module.
Description
Technical Field
The invention relates to the field of machine vision detection of hardware, in particular to a hardware surface defect detection and classification system and method based on machine vision.
Background
Machine vision detection is a non-contact detection technology which is based on a machine vision method and integrates image processing, precision measurement, pattern recognition, artificial intelligence and the like. The basic principle is to analyze the measured target image obtained by a machine vision system so as to obtain the required measurement information, and judge whether the measured target meets the standard (i.e. is qualified or unqualified) according to the prior knowledge. With the continuous and deep cross-discipline basic research, the rapid improvement of the computer performance and the improvement of the cost performance of other vision measurement peripheral components, the precision detection technology based on the machine vision has wider application prospect.
In the manufacturing process of the hardware, the edge of the hardware is easy to generate notches or deformation, and the surface of the hardware is easy to generate defects of pits, pockmarks and scratches. In the process of mass production of small hardware, strict and efficient detection links are inevitably needed, and the final product quality is ensured. The surface defects cause difficulty in detection due to randomness and irregularity, and the traditional surface defect detection means utilizes manual detection, so that a detector observes the defects by naked eyes. However, the visual detection capability is limited, so that the tiny defects are difficult to observe, the manual detection result is greatly influenced by subjective factors, the detection results of different detectors are probably different, and the long-time detection can cause mental fatigue of the detectors to influence the detection results. Compared with the prior art, the visual detection can ensure the consistency of product detection, can detect whether the product is qualified or not, and can also detect out specific deviation values, thereby realizing monitoring, analyzing and unifying the quality problems in the production process of the product.
Disclosure of Invention
The invention aims to: in order to solve the technical problems, the invention provides a hardware surface defect detection and classification system and method based on machine vision.
The invention specifically adopts the following technical scheme for realizing the purpose:
the hardware surface defect detection and classification system based on machine vision comprises a transmission module, an acquisition module, a terminal computer, a PLC control module and a sorting module, wherein,
the conveying module is used for conveying the hardware from the feeding belt to a first detection area of the conveying belt at first and detecting the hardware by the acquisition module;
the acquisition module is used for starting the camera when receiving a signal sent by the terminal computer, acquiring a hardware image on the first detection area, transmitting the acquired hardware image to the terminal computer for further processing, and closing the camera after transmission is finished;
the terminal computer hardware image is used for extracting and classifying defects and sending results to the PLC control module;
the PLC control module receives a defect detection classification result sent by the terminal computer and restarts the conveyor belt; if the hardware is qualified, continuing to move after reaching a second detection area; if the hardware with the defects reaches the second detection area, the hardware with the defects stops moving, and the hardware with the defects is pushed into the corresponding defect classification area by the sorting module.
As an optional technical scheme: and the conveying module controls the conveyor belt to work according to the stop/start signal sent by the PLC control module.
Specifically, the conveying module is used for conveying the hardware from the feeding belt to the detection area, and when a stop signal sent by the PLC control module is received, the conveying module stops working until a start signal sent by the PLC control module is received to convey the hardware again; when the hardware is transmitted to the sorting area by the transmission module, the hardware is stopped in front of the sorting device according to a signal transmitted by the PLC control module;
as an optional technical scheme: the device comprises a plurality of sensors, wherein a first sensor is arranged in a first detection area; the remaining sensors are disposed in the second detection region.
As an optional technical scheme: the number of sensors arranged in the second detection area is 4, evenly distributed in the transport direction of the conveyor belt.
As an optional technical scheme: the sorting module comprises 4 defect sorting areas, and the defect sorting areas are sequentially provided with a perimeter defect sorting device, an area defect sorting device, a depth defect sorting device (603) and a comprehensive defect sorting device.
The PLC control module is used for receiving a signal that the hardware transmitted by the first sensor reaches the first detection area, sending the signal to the conveyor belt, controlling the conveyor belt to stop running, then sending the hardware reaching signal to the terminal computer, receiving an image acquisition completion signal returned by the terminal computer, sending the signal to the conveyor belt, and restarting the conveyor belt; receiving a defect detection classification result sent by a terminal computer, wherein if the hardware is qualified, the conveyor belt does not stop, and if the hardware is not defective, a signal of a stopping position is sent to the conveyor belt; and the defect hardware receives signals of other sensors and sends signals to the defect sorting device at the corresponding position.
As an optional technical scheme: the method comprises the following steps that the terminal computer extracts and classifies defects of hardware images, and the specific method comprises the following steps;
step 201: selecting and intercepting an area of the hardware needing to be detected, and removing a shooting background;
step 202: analyzing and processing the intercepted picture by using an image processing algorithm to obtain the number of pixels occupied by the perimeter and the area of the defect, and calculating the actual depth;
step 203: and judging whether the hardware is qualified or not by using a weighting algorithm.
As an optional technical scheme: firstly, the number of pixels occupied by the perimeter, the number of pixels occupied by the area and the actual depth value H of the defect obtained in step 2020While setting the perimeter threshold T0An area threshold T1, a depth threshold T2 and a comprehensive threshold T3, and the number of pixels occupied by the perimeter of the defect, the number of pixels occupied by the area and the actual depth value H are calculated0Inputting a perimeter threshold T0, an area threshold T1, a depth threshold T2 and a comprehensive threshold T3; the weighting algorithm in step 203 comprises the following specific steps:
step 301: calculating the actual circumference L by using a circumference conversion formula0；
Step 302: determine the actual circumference L0Whether the circumference is larger than or equal to the circumference threshold value T0 or not is judged, if yes, the circumference of the hardware is unqualified, and the judgment is finished; if not, continuing the following steps;
step 303: calculating the actual area S by using an area conversion formula0；
Step 304: determining the actual area S0Whether the area is larger than or equal to an area threshold value T1 or not is judged, if yes, the area of the hardware is unqualified, and the judgment is finished; if not, continuing the following steps;
step 305: judging the actual depth H0Whether the depth is greater than or equal to a depth threshold value T2 or not is judged, if yes, the depth of the hardware is unqualified, and the judgment is finished; if not, continuing the following steps;
step 306: using a weighting formula to obtain a composite value Z0；
Step 307: judging the comprehensive value Z0Whether the current time is greater than or equal to the comprehensive threshold value T3 is judged, if yes, the hardware is unqualified, and the judgment is finished; if not, finishing the judgment.
As an optional technical scheme: the conversion formula specifically includes a circumference conversion formula:
the circumference conversion formula is as follows:
L0＝tli*L
wherein L is0Is the actual perimeter of the defect, L is the length of the defect on the image, tli is a legend of image length and actual length;
tli is calculated as follows:
wherein u is1，u2Is the abscissa, X, of a point on the image with the same ordinate1，X2The horizontal coordinates of points with the same vertical coordinate on the image are taken;
the world coordinate system coordinate u and the image coordinate system coordinate X are in the following conversion relation:
wherein u and v are coordinates of a point in a pixel coordinate system, Xw，YwAs coordinates of points in the world coordinate system, M1Is an internal reference matrix, M2Is an external reference matrix;
M1the calculation formula of (a) is as follows:
where f is the focal length, dpi is the resolution, u0，v0Coordinates of the origin of the image coordinate system;
M2the calculation formula of (a) is as follows:
wherein R is a rotation matrix, and T is an offset vector;
as an optional technical scheme: the conversion formula specifically includes an area conversion formula:
the area conversion formula is as follows:
S0＝tli2*S
wherein S is0Is the actual area of the defect, and S is the area of the defect on the image; further, the weighting formula is as follows:
Z0＝a1*L0+a2*S0+a3*H0
wherein, a1、a2、a3Is a weighting coefficient, and a1+a2+a3＝1，a1、a2、a3Can be set according to requirements, and is generally 0.1, 0.2, 0.7, L0Is the actual perimeter of the defect, S0Is the actual area of the defect, H0Is the actual depth of the defect.
The invention has the following beneficial effects:
1. the conveying module can convey the hardware to the detection area and the sorting area from the feeding belt, the terminal computer extracts and classifies defects of the hardware images by using the provided detection and classification method according to the hardware images returned by the acquisition module and sends the results to the PLC control module, and the control module can control the sorting module to sort the hardware according to the classification results so as to distinguish qualified products and different types of defective products, so that the automation of hardware defect detection and classification is completed;
2. compared with the existing manual detection technology, the detection efficiency and accuracy are improved.
3. Meanwhile, the provided detection classification method judges whether the hardware is qualified or not according to the perimeter, the area and the depth of the hardware defect, and improves the detection accuracy through four-layer judgment
Drawings
FIG. 1 is a schematic structural diagram of a hardware surface defect detecting and classifying system according to a first embodiment of the present invention;
FIG. 2 is a schematic flowchart of a hardware surface defect detecting and classifying method according to a second embodiment of the present invention;
FIG. 3 is a flow chart of a weighting algorithm;
FIG. 4 is a schematic flowchart of a hardware surface defect detecting and classifying method according to a third embodiment of the present invention;
FIG. 5 is a flow chart illustrating a detailed procedure of step 401;
FIG. 6 is a flow chart illustrating a refinement step of step 402;
FIG. 7 is a flowchart illustrating a refinement step of step 403;
FIG. 8 is a flow chart illustrating a refinement step of step 404;
reference numerals: 1-conveying module, 101-feeding belt, 102-first detection area, 103-second detection area, 2-hardware, 3-acquisition module, 301-camera, 302-light source, 303-supporting frame, 4-terminal computer, 5 PLC-control module, 6-sorting module, 601-perimeter defect sorting device, 602-area defect sorting device, 603-depth defect sorting device, 604-comprehensive defect sorting device, 701-first sensor, 702-second sensor, 703-third sensor, 704-fourth sensor and 705-fifth sensor.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. The components of embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations.
Thus, the following detailed description of the embodiments of the present invention, presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Example 1
Fig. 1 is a schematic structural diagram of a hardware surface defect detecting and classifying system provided in this embodiment, where the system includes: the system comprises a conveying module 1, an acquisition module 3, a computer terminal 4, a PLC control module 5 and a sorting module 6. The conveying module 1, the sorting module 6, the collecting module 3, the first sensor 701, the second sensor 702, the third sensor 703, the fourth sensor 704 and the fifth sensor 705 are connected with the control module 5, the terminal computer 4 is connected with the control module 5, and the collecting module 3 is connected with the terminal computer 4.
Further, the conveying module 1 conveys the hardware 2 to the first detection area 102, the first sensor 701 detects that the hardware 2 reaches a detection position and sends a signal to the PLC control module 5, the PLC control module 5 sends the signal to the terminal computer 4, the terminal computer 4 sends the signal to the acquisition module 3, and meanwhile the conveyor belt is controlled to stop running.
Further, the conveying module 1 comprises a feeding belt 101 and a conveying belt, and a first detection area 102 and a second detection area 103 are arranged on the conveying belt.
Further, the acquisition module 3 includes a camera 301, a light source 302 and a support 303. The camera 301 and the light source 302 are fixed to a support frame 303. The light source 302 is turned on when the transport module 1 starts to operate and remains on until the end of the detection. The camera 301 receives signals sent by the terminal computer 4 and then is started and collects images, the collected images are sent to the terminal computer 4 after collection is completed, the terminal computer 4 receives hardware images sent back by the camera 301 and sends the collected image signals to the PLC control module 5, and the PLC control module 5 sends starting signals to the conveyor belt after receiving the signals.
Further, the terminal computer 4 extracts and classifies defects of the hardware image by using the proposed detection and classification method, and sends the classification result to the PLC control module 5. The classification result comprises five conditions of unqualified perimeter, unqualified area, unqualified depth, comprehensive unqualified and qualified. If the classification result is that the circumference is unqualified, the PLC control module 5 controls the conveyor belt to be in the second detection area 103, the hardware 2 is conveyed to a circumference defect sorting area, when the second sensor 702 detects that the hardware 2 reaches a specified position, a signal is sent to the PLC control module 5, the PLC control module 5 controls the conveyor belt to stop running, meanwhile, the circumference defect sorting device 601 is controlled to sort the hardware 2, and after sorting is completed, the PLC control module 5 sends a starting signal to the conveyor belt; if the classification result is that the area is unqualified, the PLC control module 5 controls the conveyor belt to convey the hardware 2 to an area defect sorting area, when the third sensor 703 detects that the hardware 2 reaches a specified position, a signal is sent to the PLC control module 5, the PLC control module 5 controls the conveyor belt to stop running, meanwhile, the area defect sorting device 602 is controlled to sort the hardware 2, and after sorting is completed, the PLC control module 5 sends a starting signal to the conveyor belt; if the classification result is that the depth is unqualified, the PLC control module 5 controls the conveyor belt to convey the hardware 2 to a depth defect sorting area, when the fourth sensor 704 detects that the hardware 2 reaches a specified position, a signal is sent to the PLC control module 5, the PLC control module 5 controls the conveyor belt to stop running, meanwhile, the depth defect sorting device 603 is controlled to sort the hardware 2, and after sorting is completed, the PLC control module 5 sends a starting signal to the conveyor belt; if the classification result is comprehensive unqualified, the PLC control module 5 controls the conveyor belt to convey the hardware 2 to a comprehensive defect sorting area, when the fifth sensor 705 detects that the hardware 2 reaches a specified position, a signal is sent to the PLC control module 5, the PLC control module 5 controls the conveyor belt to stop running, the comprehensive defect sorting device 604 is controlled to sort the hardware 2, and after sorting is completed, the PLC control module 5 sends a starting signal to the conveyor belt; if the classification result is qualified, the PLC control module 5 does not send signals to the conveying module 1, the sorting module 6 and the sensor, and the conveying belt conveys the hardware 2 to a qualified area at the tail of the conveying belt.
From the hardware surface defect detection classification system provided above, it can be known that: the invention provides a hardware surface defect detection and classification system based on machine vision, which comprises: the system comprises a conveying module 1, an acquisition module 3, a terminal computer 4, a PLC control module 5 and a sorting module 6. Conveying module 1 can convey hardware 2 to detection area and letter sorting region from pay-off area 101, terminal computer 4 is according to the hardware image that collection module 1 returned, utilize the detection classification method who proposes to carry out defect extraction and classification to the hardware image, and send the result to PLC control module 5, PLC control module 5 can control letter sorting module according to the classification result and sort hardware 2 in order to distinguish certified products and different types of defective products, the automation of hardware 2 defect detection classification has been accomplished, compare in present manual detection technique, detection efficiency and rate of accuracy have been improved. Meanwhile, the provided detection classification method judges whether the hardware 2 is qualified or not according to the perimeter, the area and the depth of the hardware defect, and improves the detection accuracy through four-layer judgment.
Example 2
Fig. 2 is a schematic flow chart of a method for detecting and classifying surface defects of hardware according to the present embodiment, and as shown in fig. 2, the process of detecting and classifying surface defects of hardware includes:
step 201: selecting and intercepting an area of the hardware needing to be detected, and removing a shooting background;
step 202: analyzing and processing the intercepted picture by using an image processing algorithm to obtain the number of pixels occupied by the perimeter and the area of the defect, and calculating the actual depth;
step 203: and judging whether the hardware is qualified or not by using a weighting algorithm.
Therefore, in the embodiment, the detection and classification method can detect and classify the defects of the hardware only according to the shot hardware picture without contacting the hardware, so that potential harm to the hardware caused by direct contact is avoided. Meanwhile, the hardware surface detection classification method judges whether the hardware is qualified or not according to the perimeter, the area and the depth of the hardware defect, and improves the detection accuracy through four-layer judgment.
Further, the flow chart of the weighting algorithm in step 203 is shown in fig. 3, and the process of the weighting algorithm includes:
firstly, the number of pixels occupied by the perimeter, the number of pixels occupied by the area and the actual depth value H of the defect obtained in step 2020Simultaneously setting a perimeter threshold T0, an area threshold T1, a depth threshold T2 and a comprehensive threshold T3, and setting the number of pixels occupied by the perimeter, the number of pixels occupied by the area and the actual depth value H of the defect0Inputting a perimeter threshold T0, an area threshold T1, a depth threshold T2 and a comprehensive threshold T3;
step 301: calculating the actual circumference L by using a circumference conversion formula0；
Step 302: determine the actual circumference L0Whether the circumference is larger than or equal to the circumference threshold value T0 or not is judged, if yes, the circumference of the hardware is unqualified, and the judgment is finished; if not, continuing the following steps;
step 303: calculating the actual area S by using an area conversion formula0；
Step 304: determining the actual area S0Whether the area is larger than or equal to an area threshold value T1 or not is judged, if yes, the area of the hardware is unqualified, and the judgment is finished; if not, continuing the following steps;
step 305: judging the actual depth H0Whether the depth is greater than or equal to a depth threshold value T2 or not is judged, if yes, the depth of the hardware is unqualified, and the judgment is finished; if not, continuing the following steps;
step 306: using a weighting formula to obtain a composite value Z0；
Step 307: judging the comprehensive value Z0Whether the current time is greater than or equal to the comprehensive threshold value T3 is judged, if yes, the hardware is unqualified, and the judgment is finished; if not, finishing the judgment.
Further, the formula of the perimeter transformation in step 301 is as follows:
L0＝tli*L
wherein L is0The actual perimeter of the defect, L the length of the defect on the image, and t L i the legend of image length and actual length.
tli is calculated as follows:
wherein u is1，u2Is the abscissa, X, of a point on the image with the same ordinate1，X2The abscissa is the abscissa of the same point on the image.
The world coordinate system coordinate u and the image coordinate system coordinate X are in the following conversion relation:
wherein u and v are coordinates of a point in a pixel coordinate system, Xw，YwAs coordinates of points in the world coordinate system, M1Is an internal reference matrix, M2Is a foreign reference matrix
M1The calculation formula of (a) is as follows:
where f is the focal length, dpi is the resolution, u0，v0Coordinates being the origin of the image coordinate system
M2The calculation formula of (a) is as follows:
wherein, R is a rotation matrix, and T is an offset vector.
Further, the area conversion formula in step 303 is as follows:
S0＝tli2*S
wherein S is0Is the actual area of the defect, S is the area of the defect on the image
Further, the weighting formula in step 306 is as follows:
Z0＝a1*L0+a2*S0+a3*H0
wherein, a1、a2、a3Is a weighting coefficient, and a1+a2+a3＝1，a1、a2、a3Can be set according to requirements, and is generally 0.1, 0.2, 0.7, L0Is the actual perimeter of the defect, S0Is the actual area of the defect, H0Is the actual depth of the defect.
As can be seen, in this embodiment, the actual perimeter, area, and integrated value of the hardware defect are calculated by using the input conditions of the number of pixels occupied by the perimeter, the number of pixels occupied by the area, the actual depth, and the like, and the obtained actual perimeter, area, depth, and integrated value of the hardware defect are compared with the perimeter threshold T0, the area threshold T1, the depth threshold T2, and the integrated threshold T3 step by step, so as to determine whether the hardware 2 is qualified. The gradual progressive judging mode can save certain judging time, save system resources, and meanwhile, can carefully judge the specific defects of the hardware and is convenient to classify.
Example 3
As shown in fig. 4, a schematic flow chart of a method for detecting and classifying surface defects of hardware according to this embodiment is used in the system for detecting and classifying surface defects of hardware according to embodiment 1, and the method includes steps 401 to 405:
step 401: preparing a system;
step 402: the hardware image is collected by the collection module and is sent to the terminal computer;
step 403: the terminal computer processes the hardware pictures to obtain a classification result and sends the classification result to the PLC control module;
step 404: and the PLC control module controls the sorting module to sort the hardware.
Further, the flowchart of the refining step of step 401 is shown in fig. 5, and includes steps 501 to 505:
step 501: starting a terminal computer;
step 502: starting the PLC;
step 503: starting a transmission module;
step 504: turning on a camera and a light source;
step 505: the sensor is turned on.
Specifically, the terminal computer, the PLC, the transmission module, the camera, the light source, the sensor and the like are started in advance, so that accidents of any link caused by the fact that the equipment is not started in time in the detection process are prevented, and the reliability of the detection classification result is guaranteed. The conveying module, the sorting module, the light source and the sensor are connected with the control module, the terminal computer is connected with the control module, and the camera is connected with the terminal computer.
Further, the flowchart of the refining step of step 402 is shown in fig. 6, and includes steps 601 to 602:
step 601: conveying hardware to a detection position by a conveying belt;
step 602: and the camera collects hardware images and transmits the hardware images to the terminal computer.
Specifically, arrange the hardware in the pay-off area, the pay-off area transports the hardware to the conveyer belt, and the conveyer belt transports the hardware to detection area again. The sensor detects that the hardware reaches a detection position and sends a signal to the PLC control module, the PLC control module sends the signal to the terminal computer, the terminal computer sends the signal to the acquisition module, and meanwhile the conveyor belt is controlled to stop running. The camera is started and collects images after receiving signals sent by the terminal computer, the collected images are sent to the terminal computer after collection is completed, the terminal computer receives hardware images sent back by the camera and sends collected image signals to the PLC control module, and the PLC control module sends starting signals to the conveyor belt after receiving the signals.
Further, a flowchart of the refining step of step 403 is shown in fig. 7, and includes steps 701 to 702:
step 701: the terminal computer inspects and classifies the defects according to the hardware surface defect detection and classification method;
step 702: and sending the classification result to a PLC control module.
Specifically, a flowchart of the hardware surface defect detection and classification method is shown in fig. 2, and according to the flowchart, defect detection and classification can be performed on the acquired hardware image.
A specific weighting algorithm flow chart is shown in fig. 3, the defect type of the hardware can be subjected to step judgment according to the weighting algorithm, and if the actual perimeter is greater than or equal to the set perimeter threshold, the perimeter of the hardware is unqualified; if the actual area is larger than or equal to the set area threshold value, the area of the sending hardware is unqualified; if the actual depth is larger than or equal to the set depth threshold value, the depth of the hardware sending is unqualified; if the comprehensive value is greater than or equal to the set comprehensive threshold value, the sending hardware is unqualified; when the hardware image passes through all 4 judging conditions, the hardware is sent to be qualified, and through 4 layers of judgment, the detection accuracy is improved, and the omission factor is reduced.
Further, the flowchart of the step 404 is shown in fig. 8, and includes steps 801 to 804:
step 801: judging whether the received signal is the hardware with unqualified perimeter, if so, controlling a perimeter defect sorting device by the PLC to sort the hardware, finishing the judgment, and if not, continuing;
step 802: judging whether the received signal is a hardware with unqualified area, if so, controlling an area defect sorting device by the PLC to sort the hardware, finishing the judgment, and if not, continuing;
step 803: judging whether the received signal is hardware with unqualified depth, if so, controlling a depth defect sorting device by the PLC to sort the hardware, finishing the judgment, and if not, continuing;
step 804: and judging whether the received signal is the hardware with unqualified comprehensive value, if so, controlling the comprehensive defect sorting device by the PLC to sort the hardware, finishing the judgment, and if not, ensuring that the hardware is qualified.
Specifically, the PLC receives classification result signals of the terminal computer, and the classification results comprise five conditions of unqualified perimeter, unqualified area, unqualified depth, comprehensive unqualified depth and qualified depth. If the classification result is that the circumference is unqualified, the PLC control module controls the conveyor belt to convey the hardware to a circumference defect sorting area, when the sensor detects that the hardware reaches a specified position, a signal is sent to the PLC control module, the PLC control module controls the conveyor belt to stop running, meanwhile, the circumference defect sorting device is controlled to sort the hardware, and after sorting is completed, the PLC control module sends a starting signal to the conveyor belt; if the classification result is that the area is unqualified, the PLC control module controls the conveyor belt to convey the hardware to an area defect sorting area, when the sensor detects that the hardware reaches a specified position, a signal is sent to the PLC control module, the PLC control module controls the conveyor belt to stop running, meanwhile, the area defect sorting device is controlled to sort the hardware, and after sorting is completed, the PLC control module sends a starting signal to the conveyor belt; if the classification result is that the depth is unqualified, the PLC control module controls the conveyor belt to convey the hardware to a depth defect sorting area, when the sensor detects that the hardware reaches a specified position, a signal is sent to the PLC control module, the PLC control module controls the conveyor belt to stop running, meanwhile, the depth defect sorting device is controlled to sort the hardware, and after sorting is completed, the PLC control module sends a starting signal to the conveyor belt; if the classification result is comprehensive unqualified, the PLC control module controls the conveyor belt to convey the hardware to a comprehensive defect sorting area, when the sensor detects that the hardware reaches a specified position, a signal is sent to the PLC control module, the PLC control module controls the conveyor belt to stop running, meanwhile, the comprehensive defect sorting device is controlled to sort the hardware, and after sorting is completed, the PLC control module sends a starting signal to the conveyor belt; if the classification result is qualified, the PLC control module does not send signals to the conveying module, the sorting module and the sensor, and the conveying belt conveys the hardware to a conveying belt tail qualified area.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and should not be taken as limiting the scope of the present invention, and any modifications, equivalents and improvements made by those skilled in the art within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (9)
1. The hardware surface defect detection and classification system based on machine vision comprises a transmission module (1), an acquisition module (3), a terminal computer (4), a PLC control module (5) and a sorting module (6),
the conveying module (1) is used for conveying the hardware (2) from the feeding belt (101) to a first detection area (102) of the conveying belt, and the hardware is detected by the acquisition module (3);
the acquisition module (3) is used for starting the camera when receiving a signal sent by the terminal computer (4), acquiring a hardware image on the first detection area (102), transmitting the acquired hardware image to the terminal computer (4) for further processing, and closing the camera after transmission is finished;
the terminal computer (4) extracts and classifies defects of the hardware image and sends the result to the PLC control module (5);
the PLC control module (5) receives a defect detection classification result sent by the terminal computer and restarts the conveyor belt; if the hardware is qualified, the hardware continues to move after reaching the second detection area (103); if the hardware is defective, the hardware stops moving after reaching the second detection area (103), and the hardware is pushed into the corresponding defect classification area by the sorting module (6).
2. The machine vision-based hardware surface defect detection and classification system is characterized in that the conveying module (1) controls the operation of a conveying belt according to a stop/start signal sent by a PLC (programmable logic controller) control module (5).
3. The machine-vision-based hardware surface defect detection and classification system of claim 1, characterized by comprising a plurality of sensors, wherein a first sensor (701) is arranged at a first detection area (102); the remaining sensors are arranged in a second detection area (103).
4. Machine vision based hardware surface defect detection and classification system according to claim 3, characterized in that the number of sensors arranged in the second detection area (103) is 4, evenly distributed in the transport direction of the conveyor belt.
5. The machine vision-based hardware surface defect detection and classification system is characterized in that the sorting module (6) comprises 4 defect sorting areas, and the defect sorting areas are sequentially provided with a perimeter defect sorting device (601), an area defect sorting device (602), a depth defect sorting device (603) and a comprehensive defect sorting device (604).
6. The hardware surface defect detection and classification system based on the machine vision is characterized in that the terminal computer (4) extracts and classifies the defects of the hardware image, and the specific method comprises the following steps;
step 201: selecting and intercepting an area of the hardware needing to be detected, and removing a shooting background;
step 202: analyzing and processing the intercepted picture by using an image processing algorithm to obtain the number of pixels occupied by the perimeter and the area of the defect, and calculating the actual depth;
step 203: and judging whether the hardware is qualified or not by using a weighting algorithm.
7. The machine-vision-based hardware surface defect detecting and classifying system according to claim 6, wherein the number of pixels occupied by the perimeter, the number of pixels occupied by the area and the actual depth value H of the defect obtained in step 202 are firstly used0Simultaneously setting a perimeter threshold T0, an area threshold T1, a depth threshold T2 and a comprehensive threshold T3, and setting the number of pixels occupied by the perimeter, the number of pixels occupied by the area and the actual depth value H of the defect0Inputting a perimeter threshold T0, an area threshold T1, a depth threshold T2 and a comprehensive threshold T3; the weighting algorithm in step 203 comprises the following specific steps:
step 301: calculating the actual circumference L by using a circumference conversion formula0；
Step 302: determine the actual circumference L0Whether the circumference is larger than or equal to the circumference threshold value T0 or not is judged, if yes, the circumference of the hardware is unqualified, and the judgment is finished; if not, continuing the following steps;
step 303: calculating the actual area S by using an area conversion formula0；
Step 304: determining the actual area S0Whether the area is larger than or equal to an area threshold value T1 or not is judged, if yes, the area of the hardware is unqualified, and the judgment is finished; if not, continuing the following steps;
step 305: judging the actual depth H0Whether the depth is greater than or equal to a depth threshold value T2 or not is judged, if yes, the depth of the hardware is unqualified, and the judgment is finished; if not, continuing the following steps;
step 306: using a weighting formula to obtain a composite value Z0；
Step 307: judging the comprehensive value Z0Whether the current time is greater than or equal to the comprehensive threshold value T3 is judged, if yes, the hardware is unqualified, and the judgment is finished; if not, finishing the judgment.
8. The machine-vision-based hardware surface defect detection and classification system of claim 7, wherein the conversion formula specifically comprises a circumference conversion formula:
the circumference conversion formula is as follows:
L0＝tli*L
wherein L is0Is the actual perimeter of the defect, L is the length of the defect on the image, tli is a legend of image length and actual length;
tli is calculated as follows:
wherein u is1，u2Is the abscissa, X, of a point on the image with the same ordinate1，X2The horizontal coordinates of points with the same vertical coordinate on the image are taken;
the world coordinate system coordinate u and the image coordinate system coordinate X are in the following conversion relation:
wherein u and v are coordinates of a point in a pixel coordinate system, Xw，YwAs coordinates of points in the world coordinate system, M1Is an internal reference matrix, M2Is an external reference matrix;
M1the calculation formula of (a) is as follows:
where f is the focal length, dpi is the resolution, u0，v0Coordinates of the origin of the image coordinate system;
M2the calculation formula of (a) is as follows:
wherein, R is a rotation matrix, and T is an offset vector.
9. The machine-vision-based hardware surface defect detection and classification system of claim 4, wherein the conversion formula specifically comprises an area conversion formula:
the area conversion formula is as follows:
S0＝tli2*S
wherein S is0Is the actual area of the defect, and S is the area of the defect on the image;
further, the weighting formula is as follows:
Z0＝a1*L0+a2*S0+a3*H0
wherein, a1、a2、a3Is a weighting coefficient, and a1+a2+a3＝1，a1、a2、a3Can be set according to requirements, and is generally 0.1, 0.2, 0.7, L0Is the actual perimeter of the defect, S0Is the actual area of the defect, H0Is the actual depth of the defect.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111042160.9A CN113894055A (en) | 2021-09-06 | 2021-09-06 | Hardware surface defect detection and classification system and method based on machine vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111042160.9A CN113894055A (en) | 2021-09-06 | 2021-09-06 | Hardware surface defect detection and classification system and method based on machine vision |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113894055A true CN113894055A (en) | 2022-01-07 |
Family
ID=79188613
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111042160.9A Pending CN113894055A (en) | 2021-09-06 | 2021-09-06 | Hardware surface defect detection and classification system and method based on machine vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113894055A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115836976A (en) * | 2023-02-23 | 2023-03-24 | 四川新荷花中药饮片股份有限公司 | Production method of intelligent control system for production of toxic decoction pieces in Araceae |
| CN115937199A (en) * | 2023-01-06 | 2023-04-07 | 山东济宁圣地电业集团有限公司 | Spraying quality detection method for insulating layer of power distribution cabinet |
| CN117606566A (en) * | 2024-01-22 | 2024-02-27 | 吉林省柏汇物联科技有限公司 | Iron and steel production evaluation system based on weight detection data |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2004066209A1 (en) * | 2003-01-14 | 2004-08-05 | Shuosen Robert Liu | Method for displaying and measuring objects in actual size |
| WO2007126260A1 (en) * | 2006-04-28 | 2007-11-08 | Won-Seob Kim | Electronic commerce system and method for providing real size image |
| CN103785622A (en) * | 2014-01-28 | 2014-05-14 | 浙江理工大学 | Machine-vision-based part sorting device and method |
| CN105588845A (en) * | 2016-01-04 | 2016-05-18 | 江苏科技大学 | Weld defect characteristic parameter extraction method |
| CN107486415A (en) * | 2017-08-09 | 2017-12-19 | 中国计量大学 | Thin bamboo strip defect on-line detecting system and detection method based on machine vision |
| CN207057050U (en) * | 2017-03-24 | 2018-03-02 | 桂林电子科技大学 | A kind of micro parts Appearance quality defect optical detection apparatus |
| CN108680109A (en) * | 2018-05-23 | 2018-10-19 | 山东农业大学 | A kind of wheat root measurement method based on image procossing |
| CN110472575A (en) * | 2019-08-15 | 2019-11-19 | 中国农业大学 | A kind of string tomato maturation detection method based on deep learning and computer vision |
| CN112308925A (en) * | 2019-08-02 | 2021-02-02 | 上海肇观电子科技有限公司 | Binocular calibration method and device of wearable device and storage medium |
| CN113070240A (en) * | 2021-03-25 | 2021-07-06 | 南京工业大学 | Copper plate surface defect detection and automatic classification method based on machine vision and deep learning |
-
2021
- 2021-09-06 CN CN202111042160.9A patent/CN113894055A/en active Pending
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2004066209A1 (en) * | 2003-01-14 | 2004-08-05 | Shuosen Robert Liu | Method for displaying and measuring objects in actual size |
| WO2007126260A1 (en) * | 2006-04-28 | 2007-11-08 | Won-Seob Kim | Electronic commerce system and method for providing real size image |
| CN103785622A (en) * | 2014-01-28 | 2014-05-14 | 浙江理工大学 | Machine-vision-based part sorting device and method |
| CN105588845A (en) * | 2016-01-04 | 2016-05-18 | 江苏科技大学 | Weld defect characteristic parameter extraction method |
| CN207057050U (en) * | 2017-03-24 | 2018-03-02 | 桂林电子科技大学 | A kind of micro parts Appearance quality defect optical detection apparatus |
| CN107486415A (en) * | 2017-08-09 | 2017-12-19 | 中国计量大学 | Thin bamboo strip defect on-line detecting system and detection method based on machine vision |
| CN108680109A (en) * | 2018-05-23 | 2018-10-19 | 山东农业大学 | A kind of wheat root measurement method based on image procossing |
| CN112308925A (en) * | 2019-08-02 | 2021-02-02 | 上海肇观电子科技有限公司 | Binocular calibration method and device of wearable device and storage medium |
| CN110472575A (en) * | 2019-08-15 | 2019-11-19 | 中国农业大学 | A kind of string tomato maturation detection method based on deep learning and computer vision |
| CN113070240A (en) * | 2021-03-25 | 2021-07-06 | 南京工业大学 | Copper plate surface defect detection and automatic classification method based on machine vision and deep learning |
Non-Patent Citations (1)
| Title |
|---|
| 李丽娟 等: ""基于图像处理技术的五金件表面缺陷检测研究"", 《工程设计学报》 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115937199A (en) * | 2023-01-06 | 2023-04-07 | 山东济宁圣地电业集团有限公司 | Spraying quality detection method for insulating layer of power distribution cabinet |
| CN115836976A (en) * | 2023-02-23 | 2023-03-24 | 四川新荷花中药饮片股份有限公司 | Production method of intelligent control system for production of toxic decoction pieces in Araceae |
| CN117606566A (en) * | 2024-01-22 | 2024-02-27 | 吉林省柏汇物联科技有限公司 | Iron and steel production evaluation system based on weight detection data |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113894055A (en) | Hardware surface defect detection and classification system and method based on machine vision | |
| CN104502356B (en) | A kind of automatic testing method of the sliding bearing defect in inner surface based on machine vision | |
| CN109724984B (en) | Defect detection and identification device and method based on deep learning algorithm | |
| CN107084992B (en) | Capsule detection method and system based on machine vision | |
| CN110403232B (en) | Cigarette quality detection method based on secondary algorithm | |
| CN107966454A (en) | A kind of end plug defect detecting device and detection method based on FPGA | |
| CN110246122A (en) | Small size bearing quality determining method, apparatus and system based on machine vision | |
| CN110490842B (en) | Strip steel surface defect detection method based on deep learning | |
| CN105388162A (en) | Raw material silicon wafer surface scratch detection method based on machine vision | |
| CN110310275B (en) | Chain conveyor belt defect detection method based on image processing | |
| CN105728339B (en) | A kind of ceramic cartridge vision automatic checkout system | |
| CN104101600A (en) | Method and apparatus for detecting fine cracks on cross section of continuous casting slab | |
| CN102901735B (en) | System for carrying out automatic detections upon workpiece defect, cracking, and deformation by using computer | |
| CN104483320A (en) | Digitized defect detection device and detection method of industrial denitration catalyst | |
| CN107154042A (en) | A kind of seed-coating machine visible detection method and device | |
| CN112819844B (en) | Image edge detection method and device | |
| CN106248680A (en) | A kind of engine commutator quality detecting system based on machine vision and detection method | |
| CN106018422A (en) | Matching-based visual outline defect inspection system and method for specially-shaped stamping parts | |
| CN111672773A (en) | Product surface defect detection system and method based on machine vision | |
| CN112881427B (en) | Electronic component defect detection device and method based on visible light and infrared thermal imaging | |
| CN107328781A (en) | A kind of columnar product detection method of surface flaw and device based on machine vision | |
| CN108776966A (en) | A kind of method and system of magnet appearance defect detection | |
| CN111551559A (en) | LCD (liquid Crystal display) liquid crystal screen defect detection method based on multi-view vision system | |
| CN106846294B (en) | Visual detection method, device and equipment | |
| CN205786374U (en) | Special-shaped stamping parts profile defects vision detection system based on coupling |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20220107 |
|
| RJ01 | Rejection of invention patent application after publication |