TW202100987A - Three-dimensional image surface defect detection system capable of performing an accurate, effective and simultaneous process of various defects including curvature defects, cavity defects and excessive material defects - Google Patents

Abstract

The present invention is a three-dimensional image surface defect detection system, which includes obtaining a standard object point cloud and a to-be-tested object point cloud by image capturing through a three-dimensional camera, and then using a rough comparison process of a distance structure feature comparison method to obtain a rough comparison conversion relationship, adjusting initial posture differences, and converting the to-be-tested object point cloud into a first converted point cloud, and then performing an accurate comparison process of an adaptive proximity point approximation method to obtain an accurate comparison conversion relationship, converting the first converted point cloud into a second converted point cloud, and finally, performing a defect detection process of a curved surface error comparison, identifying all defect points relative to the standard object point cloud from the second converted point cloud to obtain a defect detection result of a defect point cloud including all defect points, and displaying the defect point cloud with specific colors for easy identification.

Description

Three-dimensional image surface defect detection system

The present invention relates to a three-dimensional image surface defect detection system, in particular, the use of a three-dimensional camera to take pictures, capture three-dimensional point clouds, rough comparison processing, precise comparison processing, and defect detection processing to obtain a visualized point cloud containing flaws The defect detection result.

For a long time, in order to increase output and reduce product manufacturing costs, relevant manufacturers have continued to develop excellent manufacturing tools, machines, and equipment. With the continuous advancement of precision mechanical technology and electronic control technology, production automation has been successfully introduced into many industries. Fields, such as automobile manufacturing, pharmaceuticals, and warehousing.

Because the operational stability of production machinery and equipment will vary in actual operation, it is inevitable that mass-produced finished products will have a small number of defective products. Traditionally, the most commonly used screening method is to manually inspect the appearance of each finished product by visual inspection to check whether the appearance of each finished product is flawless, so as to eliminate defective products and ensure product quality. However, human vision is prone to fatigue and reduced sensitivity. Moreover, the long-term focus on inspecting defective products can also cause eye diseases, such as increased intraocular pressure, increased myopia, and even dizziness.

Moreover, the speed of human visual recognition is far lower than the speed of mass production of machines. Therefore, conventional technologies have mostly adopted machine vision recognition technology to replace traditional manpower, in which two-dimensional image recognition can detect the appearance, shape or color patches of the product. The three-dimensional image recognition can detect the surface curvature of the product, even the surface structure, texture, and so on. But whether it is 2D or 3D image recognition Identification requires a lot of calculations, which not only increases the burden on the software and hardware of the identification system, but also consumes considerable processing time, which affects the overall detection speed.

Take the three-dimensional image recognition system as an example. Currently, the more commonly used technologies include point cloud pre-processing, point cloud alignment, and defect detection. They mainly use point cloud pre-processing to remove possible impurities in the point cloud of the standard object and the object to be tested. Information, such as noise from the reflection of objects, uses point cloud alignment to generate rotation and translation effects to adjust the initial posture difference to achieve the goal of overlapping each other as much as possible, while flaw detection deals with different types of surface flaws.

For example, in the conventional technology, the Random Sample Consensus (RANSAC) algorithm is often used to achieve rough comparison and reduce the amount of subsequent processing operations. Iterative Closest Point (ICP) algorithm is often used. For accurate comparison.

However, the disadvantage of the conventional technology is that the calculation amount of the RANSAC algorithm and the ICP algorithm is too large, which seriously slows down the overall processing speed. In particular, the existing general defect detection methods cannot accurately and effectively deal with curvature defects and hole defects at the same time. , Multi-material defects and other different types of defects.

Therefore, there is a great need for a three-dimensional image surface defect detection system that uses a three-dimensional camera to take pictures, capture a three-dimensional point cloud, rough comparison processing, accurate comparison processing, and defect detection processing to obtain a visualized defect detection that includes a defect point cloud As a result, all the problems of the above-mentioned conventional technology can be effectively solved.

The main purpose of the present invention is to provide a three-dimensional image surface defect detection system, including steps S1, S10, S20, S30, S40, S50, S60, S70, S80, S90, and S100, which are used to use the three-dimensional image according to the standard object as the standard Technology to detect at least one object under test Facial flaws.

Specifically, step S1 is first performed to start the operation of the system, which mainly includes preparing standard objects and at least one object to be tested, and setting up a three-dimensional camera. Next, in step S10, the standard object is photographed with a three-dimensional camera to generate and transmit a standard object point cloud belonging to the three-dimensional point cloud, where the standard object point cloud includes multiple standard object clouds, and then in step S20, use the link to The point cloud comparison device of the 3D camera receives the point cloud of standard objects.

Next, proceed to step S30, use a three-dimensional camera to photograph the object to be tested to generate and transmit a point cloud of the object to be tested belonging to the three-dimensional point cloud, and in step S40, use a point cloud comparison device to receive the point cloud of the object to be tested.

Further, in step 50, the point cloud comparison device uses the distance structure feature comparison method to roughly compare the standard object point cloud and the object point cloud to be tested, so as to calculate the standard object point cloud and the object point to be tested. A rough comparison and conversion relationship between clouds, and the rough comparison and conversion relationship is used to convert the point cloud of the object to be tested into the first conversion point cloud.

After that, step S60 is executed, and the point cloud comparison device uses the adaptive proximity point approximation method to accurately compare the standard object point cloud and the first conversion point cloud to calculate the standard object point cloud and the first conversion point. The precise comparison conversion relationship between the clouds, and the precise comparison conversion relationship is used to convert the first conversion point cloud into the second conversion point cloud.

Then enter step S70, the point cloud comparison device uses the curved surface error comparison method to perform defect detection processing on the standard object point cloud and the second converted point cloud, and calculates the difference between the second converted point cloud and the standard object point cloud. A plurality of matching defect points constitute a defect point cloud, and then a defect detection result including the defect point cloud is generated.

Then, in step S80, it is determined whether all the objects to be tested have been completed, if If it is not completed yet, go back to step S30, if it is completed, go to step S90, the point cloud comparison device outputs all the defect detection results, and step S100 is executed to end the operation of the system.

In particular, the point cloud comparison device is an integrated circuit composed of digital circuits, and the rough comparison conversion relationship of the rough comparison processing is used to adjust the initial posture difference between the standard object and the object under test, such as Translation, rotation, so as to reduce the calculation amount of subsequent accurate comparison processing.

The distance between the two closest points in the 3D point cloud captured by the 3D camera is called the resolution distance.

More specifically, the above-mentioned distance structure feature comparison method includes: randomly selecting three points in the point cloud of the object to be tested as three comparison points; selecting three points in the standard object point cloud as corresponding Three selected points of the three comparison points; calculate the three side lengths of the three comparison points as the side length of the three comparison points; calculate the three side lengths of the three selected points as the three selected points side length; compare the three comparison points Whether the three differences between the side length and the side length of the corresponding three selected points are all less than the preset side length comparison value; if the three differences are all less than the side length comparison value, the three comparison points and the three selected points form a three matching point pair ; Calculate the conversion relationship between the three matching points and the three selected points in the three matching point pairs; use the conversion relationship to convert the point cloud of the object to be tested into the conversion point cloud; calculate the corresponding two points between the conversion point cloud and the standard object point cloud Distance, and calculate the root mean square value (Root-Mean-Square, RMS) of all the distances, and further calculate the total number of these distances that are less than the preset distance judgment value, so as to be regarded as the inner group (Inlier) Number; judge whether the root mean value and the number of inner groups meet the preset stopping conditions, and the stopping condition is that the root mean value is less than the preset mean root rough comparison judgment value and the number of inner groups is greater than the preset inner group Rough comparison judgment value for the number of groups; if the stop condition is reached, the conversion relationship is regarded as the rough comparison conversion relationship; and if the stop condition is not reached, all the above operations are repeated until all points of the object point cloud to be measured are completed until.

The aforementioned side length comparison value and distance judgment value are between 10-50% of the resolution distance.

In addition, the adaptive neighboring point approximation method includes: selecting a point in the first converted point cloud as the comparison point; finding the nearest point closest to the comparison point in the standard object point cloud; repeating the above two operations until Complete the first conversion point cloud corresponding to all the closest points; calculate and count the distance between all the comparison points and the corresponding closest points to generate a distance histogram (Distance Histogram), and the horizontal axis of the distance histogram The generation represents the distance, and the vertical axis of the distance histogram represents the number of the same distance; the minimum cross entropy method (or called the maximum between-class variance method, or Otsu method) is used to calculate the distance threshold corresponding to the distance histogram ; Determine whether the distance between each comparison point and the nearest neighbor point is less than the distance threshold; if the distance between the comparison point and the nearest neighbor point is less than the distance threshold, the comparison point and the nearest neighbor point are considered as matches Point pair; Calculate the conversion relationship between the nearest neighbor point and the comparison point in the matching point pair, as the nearest neighbor conversion relationship; use the nearest neighbor conversion relationship to convert the first conversion point cloud to generate the nearest neighbor point cloud, and calculate the nearest neighbor point The distance between the corresponding two points in the cloud and the standard point cloud, and calculate the Root-Mean-Square (RMS) of the nearest point cloud of all these distances, and further calculate the nearest point cloud to match the distance The total number less than the preset distance judgment value is regarded as the number of inliers in the nearest point cloud; judge whether the root mean value of the nearest point cloud and the number of groups in the nearest point cloud reach the preset Convergence condition, where the convergence condition means that the nearest point cloud mean root value is less than the preset mean root accurate comparison judgment value and the number of nearest neighbor point cloud inner groups is greater than the preset inner group number accurate comparison judgment value ; If the convergence condition is reached, the nearest neighbor conversion relationship is regarded as an accurate comparison conversion relationship; and if the convergence condition is not reached, all the above operations are repeated until all points of the first conversion point cloud are completed.

Especially, the distance judgment value is between 10-50% of the resolution distance.

Furthermore, the surface error comparison method includes: selecting a point in the second converted point cloud as the estimated point; using the estimated point to find the nearest 9 points in the standard object point cloud as the nearest point group; using the nearest point group To calculate a set of surface parameters, including real numbers a0, a1, a2, a3, a4, a5, a6, a7, a8 to define the surface equation, where the surface equation is expressed as z=a0+a1x+a2x ² +a3y +a4y ² +a5xy+a6x ² y+a7xy ² +a8x ² y ² , and (x, y, z) represents the spatial right-angle coordinates of each point in the nearest neighbor point group on the x-axis, y-axis, and z-axis; use The estimated points are substituted into the surface equation defined by the set of surface parameters to calculate the z-axis error, where the z-axis error (△z) is expressed as zk-a0+a1xk+a2xk ² +a3yk+a4yk ² +a5xkyk+a6xk ² yk+ a7xkyk ² +a8xk ² yk ² , and (xk, yk, zk) represents the spatial right-angle coordinates of the estimated point; judge whether the z-axis error is greater than the preset error threshold; if the z-axis error is greater than the error threshold, the estimated point is Treat it as a defect; and repeat all the above operations until each point of the second converted point cloud is processed.

In addition, the aforementioned error threshold is between 1-50% of the resolution distance.

In short, the present invention mainly uses rough comparison processing to adjust the initial posture difference, and overlaps the measured object point cloud with the standard object point cloud as much as possible to reduce the amount of subsequent processing calculations, which can greatly accelerate the overall system speed, and then use adaptive The linear proximity point approximation method is used for accurate comparison processing, and finally, the surface error comparison method is used for flaw detection processing to obtain the required flaw detection results. Therefore, the system of the present invention can greatly reduce the calculation time and improve the accuracy of detecting the surface defects of the three-dimensional object, and is very suitable for the field of real-time inspection on the production line during mass production of products, thereby realizing automatic inspection.

S1, S10, S20, S30, S40‧‧‧Step

S50, S60, S70, S80, S90, S100‧‧‧Step

10‧‧‧Shooting control unit

12‧‧‧Point cloud receiving unit

14‧‧‧Operation parameter storage unit

20‧‧‧Point Cloud Storage Unit

30‧‧‧Rough comparison unit

40‧‧‧Precise comparison unit

50‧‧‧Defect detection unit

60‧‧‧Defect detection result storage unit

C‧‧‧Three-dimensional camera

D‧‧‧Conveying direction

M‧‧‧Point cloud comparison device

P‧‧‧Display device

ST‧‧‧Standard Object

UUT‧‧‧Object to be tested

V‧‧‧Conveyor belt

The first figure shows the operation of the 3D image surface defect detection system according to the embodiment of the present invention. Make a flow chart.

The second figure shows a schematic diagram of a three-dimensional image surface defect detection system according to an embodiment of the present invention.

The third figure shows the functional block diagram of the point cloud comparison device in the 3D image surface defect detection system according to the embodiment of the present invention.

The following is a more detailed description of the implementation of the present invention in conjunction with the drawings and component symbols, so that those who are familiar with the art can implement it after studying this manual.

Please refer to the first figure and the second figure, which are respectively an operation flowchart and a schematic diagram of the 3D image surface defect detection system according to an embodiment of the present invention. As shown in the first and second figures, the three-dimensional image surface defect detection system of the present invention includes steps S1, S10, S20, S30, S40, S50, S60, S70, S80, S90, and S1000 for standard object ST To detect whether the UUT of the object under test has surface defects. In short, the three-dimensional image surface defect detection system of the present invention mainly performs photographing, capturing a three-dimensional point cloud, rough comparison processing, precise comparison processing, and defect detection processing.

Specifically, the 3D image surface defect detection system of the present invention first executes step S1 to start the operation of the system, mainly preparing a standard object ST and at least one object to be tested UUT, and setting up a three-dimensional camera C. For example, the standard object ST can be a model object, such as a hand-drawn object, and the object to be tested UUT can be a machine-made casting, plastic injection molding, semi-finished product, or final combined product, such as Dolls, especially objects with the characteristics of surface changes. In particular, the aforementioned three-dimensional camera C may include a binocular stereo camera, a single-eye time-of-flight (ToF) camera, or a single-structured light (Structured Light) camera, and the point cloud comparison device M is composed of An integrated circuit composed of digital circuits.

In step S10, use the three-dimensional camera C to photograph the standard object ST to generate and transmit Send a standard object point cloud belonging to a three-dimensional point cloud, where the standard object point cloud includes multiple standard object clouds, and in step S20, a point cloud comparison device M connected to the three-dimensional camera C is used to receive the standard object points cloud.

In step S30, the UUT of the object under test is photographed by the three-dimensional camera C to generate and transmit the object point cloud belonging to the three-dimensional point cloud, and in step S40, the point cloud comparison device M is used to receive the object point cloud under test.

In practical applications, the standard object ST can be first placed in a preset position and photographed by the 3D camera C, and all the objects to be tested UUT are placed on the conveyor belt V and proceed in the conveying direction D to be photographed by the 3D camera C one by one Preferably, the standard object ST is in an upright posture, and the object to be tested UUT is also placed on the conveyor belt V in an upright posture, or the standard object ST and all objects to be tested UUT are placed on the conveyor belt V in an upright posture The three-dimensional camera C will shoot one by one by moving forward in the conveying direction D. In particular, the standard object ST, the object under test UUT and the three-dimensional camera C can be kept as fixed distance as possible. Therefore, the standard object point cloud and the object point cloud The initial posture difference between them includes translation and rotation.

In step 50, the point cloud comparison device M uses the distance structure feature comparison method to roughly compare the standard object point cloud and the object point cloud to be measured to calculate the standard object point cloud and the object point cloud to be measured The rough comparison conversion relationship between the two is essentially a conversion matrix, and the rough comparison conversion relationship is used to convert the object point cloud to the first conversion point cloud.

More specifically, the above-mentioned distance structure feature comparison method includes the following processing steps.

First, randomly select three points in the point cloud of the object to be tested as the three comparison points, and then select three points in the standard object point cloud as the three selected points corresponding to the three comparison points.

Then calculate the three side lengths of the three comparison points as the side lengths of the three comparison points, and calculate the three side lengths of the three selected points as the side lengths of the three selected points, and compare the side lengths of the three comparison points and the corresponding three Whether the three differences of the side length of the selected point are all less than the preset side length comparison value. If the three differences are all less than the side length comparison value, the three comparison points and the three selected points form a three matching point pair.

The aforementioned side length comparison value can be between 10-50% of the resolution distance of the 3D camera C, where the resolution distance refers to the distance between the two nearest neighbors in the 3D point cloud captured by the 3D camera C, which is determined by The image sensor of the 3D camera C is determined.

Then, the conversion relationship between the three matching points and the three selected points in the three matching point pairs is calculated, and the conversion relationship is used to convert the point cloud of the object to be tested into a conversion point cloud.

Calculate the distance between the corresponding two points between the converted point cloud and the standard object point cloud, and calculate the root mean square value (Root-Mean-Square, RMS) of all these distances, and further calculate the distances less than the preset distance The total number of judgment values is used as the number of Inliers. The aforementioned distance judgment value may be between 10-50% of the resolution distance of the three-dimensional camera C.

After that, it is judged whether the root mean value and the number of inner groups meet the preset stopping conditions, and the stopping condition is that the root mean value is less than the preset mean root rough comparison judgment value and the number of inner groups is greater than the preset inner group. The number of groups is roughly compared to the judgment value.

If the stop condition is reached, the conversion relationship is regarded as a rough comparison of the conversion relationship, and if the stop condition is not reached, all the above operations are repeated until all points of the point cloud of the object to be measured are completed.

In addition, the rough comparison conversion relationship of the rough comparison processing is used to adjust the initial posture difference between the standard object and the object to be tested, which can reduce the calculation amount of the subsequent accurate comparison processing and speed up the processing.

After completing step S50, proceed to step S60. The point cloud comparison device M uses Adaptive Proximity Point Approximation to accurately compare the standard object point cloud and the first converted point cloud for calculation The precise comparison conversion relationship between the standard object point cloud and the first conversion point cloud is essentially a conversion matrix, and then the precise comparison conversion relationship is used to convert the first conversion point cloud into the second conversion point cloud.

Specifically, the adaptive neighboring point approximation method includes the following operations.

First, select a point in the first converted point cloud as the comparison point, and find the nearest point closest to the comparison point in the standard object point cloud, and repeat the above two operations until the first conversion point cloud is completed. Up to all corresponding nearest points.

Calculate and count the distances between all comparison points and the corresponding nearest points to generate a distance histogram (Distance Histogram), and use the horizontal axis of the distance histogram to represent the distance, and use the vertical axis of the distance histogram to indicate the same The number of distances. Then use the minimum cross entropy method (or called the maximum between-class variance method, or Otsu method) to calculate the distance threshold corresponding to the distance histogram.

After that, it is judged whether the distance between each comparison point and the corresponding nearest point is less than the distance threshold. If the distance between the comparison point and the corresponding nearest point is less than the distance threshold, the comparison point and the nearest point are regarded as For the matching point pair, the conversion relationship between the closest point and the comparison point in the matching point pair is calculated, as the closest conversion relationship.

Next, convert the first converted point cloud using the nearest neighbor conversion relationship to generate the nearest neighbor point cloud, calculate the distance between the nearest neighbor point cloud and the corresponding two points in the standard point cloud, and calculate all the nearest neighbor point clouds at the same distance Root-Mean-Square (RMS), further calculate the total number of the nearest point cloud that meets the distance less than the preset distance judgment value, and use it as the number of inliers in the nearest point cloud , And the distance judgment value is between 10-50% of the resolution distance.

Judge whether the root mean value of the nearest point cloud and the number of groups in the nearest point cloud reach the preset convergence condition, where the convergence condition means that the mean root value of the nearest point cloud is less than the preset mean root value. Value and the number of clusters in the nearest neighbor point cloud is greater than the preset number of clusters in the accurate comparison judgment value. If the convergence condition is reached, the nearest neighbor conversion relationship is regarded as an accurate comparison conversion relationship. If the convergence condition is not reached, all the above operations are repeated until all points of the first conversion point cloud are completed.

After completing step S60, proceed to step S70, the point cloud comparison device M uses the surface error comparison method to perform defect detection processing on the standard object point cloud and the second conversion point cloud, and calculates to obtain the second conversion Multiple flaws in the point cloud that do not match the standard object point cloud form a flawed point cloud, and then generate a flaw detection result that includes the flawed point cloud.

Furthermore, the point cloud comparison device M can be connected to the display device P, and the above-mentioned defect detection result is further transmitted to the display device P, and the display device P displays the content of the defect detection result. In addition, the three-dimensional camera C, the point cloud comparison device M and the display device P can be integrated into a single device, which not only has the portability of light, thin and short, and the convenience of operation at any time, but also provides the intelligent vision function of fully automatic identification of object defects .

Since the second conversion point cloud converted by the accurate comparison conversion relationship is already very coincident with the standard object point cloud, the surface error comparison method can effectively find the flaws.

Furthermore, the curved surface error comparison method includes the following processing.

First, select a point in the second conversion point cloud as the estimated point, and then use the estimated point to find the nearest 9 points in the standard object point cloud as the nearest point group.

Next, use the nearest neighbor point group to calculate a set of surface parameters, including real numbers a0, a1, a2, a3, a4, a5, a6, a7, and a8 to define the surface equation, where the surface square The formula is expressed as z=a0+a1x+a2x2+a3y+a4y2+a5xy+a6x2y+a7xy2+a8x2y2, and (x, y, z) means that each point in the nearest neighbor point group is on the x-axis, y-axis, and z-axis The right-angle coordinates of the space.

Then, the estimated points are substituted into the surface equation defined by the set of surface parameters to calculate the z-axis error, where the z-axis error (△z) is expressed as zk-a0+a1xk+a2xk2+a3yk+a4yk2+a5xkyk+a6xk2yk+a7xkyk2 +a8xk2yk2, and (xk, yk, zk) represents the space rectangular coordinates of the estimated point.

Then it is determined whether the z-axis error is greater than the preset error threshold. If the z-axis error is greater than the error threshold, the estimated point is regarded as a defect point. Repeat all the above operations until each point of the second converted point cloud is processed.

In addition, the aforementioned error threshold is between 1-50% of the resolution distance.

After completing step S70, proceed to step S80 to determine whether all the UUTs to be tested have been completed. If not, return to step S30. If completed, proceed to step S90. The point cloud comparison device M outputs all If the defect is detected, step S100 is executed to end the operation of the system.

Furthermore, the defect detection result displays the defect point cloud in the object point cloud under test in a first color, and displays the rest of the object point cloud under test except the defect point cloud in a second color different from the first color.

More specifically, referring to the third figure, the functional block diagram of the point cloud comparison device M in the 3D image surface defect detection system according to the embodiment of the present invention. As shown in the third figure, the point cloud comparison device M includes a shooting control unit 10, a point cloud receiving unit 12, a point cloud storage unit 20, a rough comparison unit 30, an accurate comparison unit 40, a flaw detection unit 50, and a flaw detection The result storage unit 60, wherein the shooting control unit 10 and the point cloud receiving unit 12 are connected to the three-dimensional camera C, and the point cloud receiving unit The unit 12, the rough comparison unit 30, the precise comparison unit 40, and the flaw detection unit 50 are connected to the point cloud storage unit 20, and the flaw detection result storage unit 60 is connected to the flaw detection unit 50 and the display device P.

In the present invention, the three-dimensional camera C can automatically photograph the object without being controlled by an external device, and generate and transmit a three-dimensional point cloud. At this time, the three-dimensional camera C can also be used when the object is not updated, for example, the object has been detected. Stop shooting immediately, and automatically achieve energy-saving and power-saving effects. For example, the specific method is to use the object detection technology to compare whether the content of the images taken twice before and after the preset time has moved. If the object does not move, it means that the object has not been updated and no more shooting is required. Of course, other detection technologies can also be used, such as ultrasonic range detection, infrared detection, proximity radar detection, etc. It should be noted that these conventional technologies should all fall within the scope of the present invention.

Alternatively, the three-dimensional camera C may photograph the object under the control of the photographing control unit 10, and generate and transmit a three-dimensional point cloud. At this time, the point cloud comparison device M may further include an operating parameter storage unit 14, which is connected to the photographing control unit 10. , Used to store the number of objects to be photographed, such as inputted from the outside through the I2C or SPI protocol of the conventional technology. Therefore, the photographing control unit 10 can read the number of objects stored in the operating parameter storage unit 14 to control The three-dimensional camera C shoots until the number of objects is reached, and then implements the operations of steps S10, S30, and S80 in the first figure.

The shooting control unit 10 may also control the point cloud receiving unit 12 to receive the three-dimensional point cloud photographed and transmitted by the three-dimensional camera C, and the point cloud receiving unit 12 further stores the three-dimensional point cloud to the point cloud storage unit 20 for reading . Therefore, the point cloud receiving unit 12 and the point cloud storage unit 20 are used to implement the operations of steps S20 and S40 in the first figure.

In addition, the above-mentioned rough comparison unit 30, precise comparison unit 40, and defect inspection The testing unit 50 respectively implements the operations of steps S50, S60 and S70 in the first figure, including rough comparison processing, rough comparison processing, and flaw detection processing, and the flaw detection unit 50 transmits the flaw detection results generated by the flaw detection processing The defect detection result storage unit 60 is stored. Further, the defect detection result storage unit 60 transmits the stored defect detection result to the display device P for display, so as to realize the output result operation of step S90 in the first figure.

It should be particularly noted that the point cloud comparison device M of the present invention is not a central processing unit (CPU) or microprocessor (MCU) of conventional technology, and does not need to store or execute any firmware programs, but belongs to a specific application. Integrated circuit (Application Specific Integrated Circuit, ASIC), and is composed of digital circuit (Digital Circuit), so the processing speed is faster, more power saving, and better operation stability.

In summary, the main feature of the present invention is to use rough comparison processing to adjust the initial posture difference, and to superimpose the measured object point cloud to the standard object point cloud as much as possible to reduce the amount of subsequent processing calculations and greatly accelerate the overall system speed. In addition, the adaptive neighboring point approximation method is used for accurate comparison processing, and finally, the surface error comparison method is used for defect detection processing to find all the defects and obtain the defect detection results.

Therefore, the system of the present invention can greatly reduce the calculation time and improve the accuracy of detecting surface defects of three-dimensional objects. It is very suitable for the field of real-time inspection on the production line during mass production of products, realizes comprehensive automatic inspection and eliminates defective products.

Since the technology of the present invention is not found in published publications, periodicals, magazines, media, and exhibition venues, it is novel, and can break through the current technical bottleneck and be implemented specifically, which is indeed progressive. In addition, the present invention can solve the problems of the conventional technology, improve the overall use efficiency, and achieve the value of industrial use.

The above descriptions are only used to explain the preferred embodiments of the present invention, and are not intended to limit the present invention in any form. Therefore, any modifications or changes related to the present invention made under the same spirit of the invention , Should still be included in the scope of the invention's intention to protect.

S1, S10‧‧‧Step

S20, S30, S40‧‧‧Step

S50, S60, S70‧‧‧Step

S80‧‧‧Step

S90, S100‧‧‧Step

Claims (9)

A three-dimensional image surface defect detection system is used to detect the surface defect of at least one object to be tested based on a standard object as a standard and using a three-dimensional image technology. The system includes: step S1, start, prepare the standard object and the at least An object to be tested, and a three-dimensional camera is set up; step S10, using the three-dimensional camera to photograph the standard object to generate and transmit a standard object point cloud belonging to a three-dimensional point cloud, the standard object point cloud including multiple standard object clouds; step S20, using a point cloud comparison device connected to the 3D camera to receive the standard object point cloud, and the point cloud comparison device is an integrated circuit (IC) composed of a digital circuit (Digital Circuit) Step S30, use the three-dimensional camera to photograph the object to be tested to generate and transmit a point cloud of the object to be tested belonging to the three-dimensional point cloud; step S40, use the point cloud comparison device to receive the point cloud of the object to be tested; step 50, The point cloud comparison device uses a distance structure feature comparison method to perform a rough comparison process on the standard object point cloud and the object point cloud to be tested, so as to calculate the standard object point cloud and the object point to be tested A rough comparison conversion relationship between clouds is used to adjust an initial posture difference between the standard object and the object to be tested, and the rough comparison conversion relationship is used to convert the point cloud of the test object into a first conversion Point cloud; step S60, the point cloud comparison device uses an adaptive proximity point approximation method (Adaptive Proximity Point Approximation), the standard object point cloud and the first conversion point cloud to perform an accurate comparison process, to Calculating an accurate comparison conversion relationship between the standard object point cloud and the first conversion point cloud, and using the accurate comparison conversion relationship to convert the first conversion point cloud into a second conversion point cloud; Step S70, the point cloud comparison device uses a surface error comparison method to perform a defect detection process on the standard object point cloud and the second conversion point cloud to calculate and obtain the second conversion point Multiple defect points in the cloud that do not match the standard object point cloud are used to form a defect point cloud, and a defect detection result including the defect point cloud is generated; step S80, whether all the at least one object to be tested is completed, if not When finished, go back to step S30; step S90, the point cloud comparison device outputs all the flaw detection results; and step S100, end, where the distance between the two closest points in the three-dimensional point cloud captured by the three-dimensional camera is called Draw a resolution distance, where the distance structure feature comparison method includes: randomly selecting three points in the point cloud of the object to be tested as three comparison points; selecting three points in the standard object point cloud as Three selected points corresponding to the three comparison points; calculate the three side lengths of the three comparison points as the side lengths of the three comparison points; calculate the three side lengths of the three selected points as the three selection points side length; Compare the side lengths of the three comparison points and the side lengths of the corresponding three selected points to see if the three differences are less than the preset side length comparison value, and the side length comparison value is between 10-50% of the resolution distance; If the three difference values are all less than the side length comparison value, the three matching points and the three selected points form a three matching point pair; calculate a conversion of the three matching points and the three selected points in the three matching point pair Relationship; use the conversion relationship to convert the point cloud of the object under test into a conversion point cloud; calculate the distance between the conversion point cloud and the standard object point cloud corresponding to two points, and calculate the root mean of all these distances Value (Root-Mean-Square, RMS), and further calculate a total number of the distances that is less than a preset distance judgment value, and use it as an inlier number. The distance judgment value is the three-dimensional Between 10-50% of the resolution distance of the camera; determine whether the root mean value and the number of inner groups reach a preset stop condition, and the stop condition is that the root mean value is less than a preset mean Rough comparison judgment value for divided roots and the number of inner groups is greater than a preset rough comparison judgment value for the number of inner groups; if the stop condition is reached, the conversion relationship is regarded as the rough comparison conversion relationship; And if the stop condition is not reached, all the above operations are repeated until all points of the point cloud of the object to be measured are completed, wherein the adaptive proximity point approximation method includes: selecting a point in the first converted point cloud as A comparison point; find the closest point closest to the comparison point in the standard object point cloud; repeat the above two operations until all the closest points corresponding to the first converted point cloud are completed; calculation And count all the distances between each comparison point and the corresponding nearest point to generate a distance histogram (Distance Histogram), and use a horizontal axis of the distance histogram to represent the distance, and use the distance histogram A vertical axis of represents the number with the same distance; using a minimum cross-entropy method (or called the maximum between-class variance method, or Otsu method), calculate a distance threshold corresponding to the distance histogram; judge each Whether the distance between the comparison point and the nearest neighbor point is less than the distance threshold; if the distance between the comparison point and the nearest neighbor point is less than the distance threshold, the comparison point and the nearest point are regarded as A pair of matching points; calculating a conversion relationship between the nearest neighbor point and the comparison point in the pair of matching points as a nearest conversion relationship; using the nearest conversion relationship to convert the first conversion point cloud to generate A nearest point cloud, calculate the nearest point cloud and the distance between two corresponding points in the standard point cloud, and calculate the root mean value of a nearest point cloud (Root-Mean-Square, RMS) to further calculate a total number of points in the nearest neighbor point cloud that conform to the distance less than a preset distance judgment value, and the distance judgment value is between 10-50% of the resolution distance, to be regarded as a The number of Inliers in the nearest point cloud; determine whether the root mean value of the nearest point cloud and the number of Inliers in the nearest point cloud reach a preset convergence condition, and the convergence condition is the nearest point The cloud mean root value is less than the preset value of a mean root accurate comparison judgment value and the number of the nearest neighbor point cloud inner clusters is greater than the preset value of a preset inner cluster number precision comparison judgment value; if the convergence condition is reached, then The nearest neighbor conversion relationship is regarded as the accurate comparison conversion relationship; and if the convergence condition is not reached, all the above operations are repeated until all points of the first conversion point cloud are completed, The curved surface error comparison method includes: selecting a point in the second converted point cloud as an estimated point; using the estimated point to find the closest 9 points in the standard object point cloud as a closest point group; using The nearest neighbor point group is used to calculate a set of surface parameters, including real numbers a0, a1, a2, a3, a4, a5, a6, a7, a8 to define a surface equation, which is expressed as z=a0+ a1x+a2x2+a3y+a4y2+a5xy+a6x2y+a7xy2+a8x2y2, and (x, y, z) represents a spatial right-angle coordinate of each point in the nearest neighbor point group on the x-axis, y-axis, and z-axis; use The estimated point is substituted into the surface equation defined by the set of surface parameters to calculate a z-axis error. The z-axis error system is expressed as △z=zk-a0+a1xk+a2xk2+a3yk+a4yk2+a5xkyk+a6xk2yk+a7xkyk2+ a8xk2yk2, and (xk, yk, zk) represents the spatial right-angle coordinates of the estimated point; judge whether the z-axis error is greater than a preset error threshold; if the z-axis error is greater than the error threshold, the estimated point is Treat it as the defect; and repeat all the above operations until each point of the second converted point cloud is processed. According to the three-dimensional image surface defect detection system described in item 1 of the scope of patent application, the three-dimensional camera includes a binocular stereo camera, a single-eye time-of-flight (ToF) camera or a single structured light (Structured Light) )camera. According to the three-dimensional image surface defect detection system described in item 1 of the scope of patent application, the initial posture difference includes translation and rotation. According to the three-dimensional image surface defect detection system described in item 1 of the scope of patent application, the defect detection result is displayed in a first color in the defect point cloud of the object to be tested, and is different from the first color A second color displays the rest of the point cloud of the object to be tested excluding the defect point cloud. According to the three-dimensional image surface defect detection system described in item 1 of the scope of patent application, the standard object is placed in a preset position and shot by the three-dimensional camera, and the at least one object to be tested is placed on a conveyor belt, and It moves forward in a conveying direction and is photographed by the three-dimensional camera one by one, and a fixed distance is maintained between the standard object, the object to be tested and the three-dimensional camera. According to the three-dimensional image surface defect detection system described in item 1 of the scope of patent application, the standard The quasi-object and the at least one object to be tested are placed on a conveyor belt and moved in a conveying direction to be photographed by the three-dimensional camera one by one, and a fixed object is maintained between the standard object, the object to be tested and the three-dimensional camera distance. The three-dimensional image surface defect detection system according to the first item of the patent application further includes a display device connected to the point cloud comparison device for receiving the defect detection result and displaying the content of the defect detection result. According to the three-dimensional image surface defect detection system described in item 7 of the scope of patent application, the display device, the point cloud comparison device and the three-dimensional camera are integrated into a single device. According to the three-dimensional image surface defect detection system described in item 1 of the scope of patent application, wherein the point cloud comparison device includes a shooting control unit, a point cloud receiving unit, a point cloud storage unit, a rough comparison unit, and an accurate comparison Unit, a flaw detection unit, and a flaw detection result storage unit, and the point cloud comparison device further includes an operation parameter storage unit, which is connected to the shooting control unit and used to store a The number of objects, the shooting control unit and the cloud receiving unit are connected to the 3D camera, the point cloud receiving unit, the rough comparison unit, the precise comparison unit and the defect detection unit are connected to the point cloud storage unit, The defect detection result storage unit is connected to the defect detection unit, and the shooting control unit reads the number of objects stored in the operation parameter storage unit, and controls the three-dimensional camera to shoot accordingly until the number of objects is reached , And then implement the operations of steps S10, S30 and S80. The shooting control unit controls the point cloud receiving unit to receive the three-dimensional point cloud photographed and transmitted by the three-dimensional camera, and the point cloud receiving unit further performs the three-dimensional point cloud The cloud is stored in the point cloud storage unit for reading, the point cloud receiving unit and the point cloud storage unit are used to implement the operations of steps S20 and S40, the rough comparison unit, the precise comparison unit, and the defect The detection unit is used to implement the operations of steps S50, S60, and S70 respectively, including the rough comparison processing, the rough comparison processing, and the flaw detection processing, and the flaw detection unit detects the flaw generated by the flaw detection processing The detection result is transmitted to the defect detection result storage unit and stored.

Images