KR102646286B1 - Part inspection system - Google Patents

Abstract

The present invention relates to a parts inspection system for identifying and counting parts, and more specifically, to a parts inspection system that can determine whether a part is defective by comparing and analyzing three-dimensional feature points of a part extracted from two cameras. will be.

Description

Part inspection system {Part inspection system}

The present invention relates to a parts inspection system for identifying and counting parts, and more specifically, to a parts inspection system that can determine whether a part is defective by comparing and analyzing three-dimensional feature points of a part extracted from two cameras. will be.

Consumers increasingly want differentiated products due to rapidly changing product trends. In addition, as all of a consumer's purchase history can be databased in real time, accurate inventory management of products needed on the same day is possible in modern times, eliminating the need to create large quantities of inventory.

Due to these product trends and inventory management, the need to produce small quantities of multiple varieties is increasing, but factories with automated facilities are unable to quickly respond to frequent changes in varieties. Additionally, in small factories that rely on manpower rather than automated facilities, product costs rise due to increased labor costs.

As a response to this increase in cost, we try to assemble one product through an automated process by minimizing the amount of parts in the product assembly process. However, since each part must be supplied through a separate transfer device, most assembly processes require parts to be assembled together. It is difficult or impossible to create a supply transfer device. A way to solve this problem is to sequentially supply various parts to one transfer device, and a device that can inspect the parts after they are sequentially supplied is essential.

In an environment where multiple parts are not supplied at accurate positions and intervals, an image inspection device that can distinguish parts through feature point extraction is essential in order to inspect each part. It is robust in inspecting parts that are not in the area, but when the part is not flat and has a three-dimensional structure, the two-dimensional feature points appear different for the same part as the location where the image is acquired and the angle between the image and the part change. There is.

To solve this problem, a method that can obtain and compare 3D feature point distributions must be used. To do this, two or more cameras must be used to obtain the 3D feature point distribution of the part and characterize the part through this.

Therefore, there is a need to develop an inspection system that can clearly distinguish the shapes of similar target objects, reduce the error rate, and improve identification accuracy.

1. Korean Patent No. 10-2185322 ‘Position detection system using infrared stereo camera’ (registration date 2020.11.25) 2. Korea Patent Publication No. 10-2019-0063967 ‘Position measurement method and device using a stereo camera and 3D barcode’ (Publication date 2019.06.10) 3. Korean Patent No. 10-0837776 ‘Image conversion device and method for converting a 2-dimensional image into a 3-dimensional image’ (registration date 2008.06.05)

The present invention was developed to solve the above problems. It identifies three-dimensional feature points of a product using images captured using a stereo camera, determines more clearly whether the product is defective, and counts various types of parts by type. The purpose is to provide a capable inspection system.

The parts inspection system of the present invention includes an image capture unit including at least two image capture devices to acquire a plurality of images by photographing a moving object from different angles; a feature point generator that generates at least one feature point from each image captured by the image capture device and obtains two-dimensional feature point coordinates of the object; a part extraction unit that receives two-dimensional feature point coordinate information generated through each of the image capture devices and obtains three-dimensional feature point coordinates through the intersection of each feature point; and a parts supply determination unit that compares the reference coordinates of the three-dimensional feature points of the object stored in the database with the three-dimensional feature point coordinates obtained from the parts extraction unit to determine whether they match.

In the present invention, a part identification unit that analyzes a point cloud of feature point coordinates obtained from the feature point generation unit or the part extraction unit and compares the similarity with the feature point coordinates of the object stored in the database to identify the object for each part; It is characterized in that it is further provided.

In the present invention, the parts identification unit further includes a parts counting unit that determines whether the identified objects are identified in a preset order.

In addition, the present invention first receives the two-dimensional feature point coordinates of the moving object, sets the coordinates of the reference object, and then compares them with the two-dimensional feature point coordinates of the moving object to identify whether the object's position has changed; a location identification unit; It is characterized in that it further includes.

In the present invention, since at least two or more cameras photograph parts from various angles and extract feature points, the position and posture information of the part can be obtained very accurately, improving the accuracy of classifying, counting, and determining defects of parts. There is an advantage.

1 is a schematic diagram of a component inspection system of the present invention.
Figure 2 is a configuration diagram of the component inspection system of the present invention.
Figure 3 is a drawing and a photograph showing an example of extracting feature points from a product image according to the present invention.
Figure 4 is a diagram showing a group of feature points of the present invention.
Figure 5 is a diagram showing component identification from the group of the present invention.
Figure 6 is a diagram showing an embodiment of comparing reference feature point extraction and feature points for location identification according to the present invention.
Figure 7 is a flowchart explaining the contents for inspecting an object of the present invention.

Hereinafter, an embodiment of the component inspection system of the present invention will be described in detail with reference to the drawings. In describing embodiments of the present invention, if it is determined that a detailed description of related known technologies may unnecessarily obscure the gist of the present invention, the detailed description will be omitted.

Referring to Figure 1, an object is seated on a conveyor belt and moves according to the drive of the conveyor belt. In addition, an image capture device is further provided that can capture moving objects at different angles, preferably at angles that are symmetrical while facing each other. It is desirable to have at least two video recording devices. Therefore, it is possible to obtain video images of an object photographed from different angles through an imaging device.

The video recording device can be a camera, and the PC can use a GPU-related device such as the Nvidia TX module to increase operation speed and configure a single interface module. The interface module includes functions such as USB 3.0, WIFI communication, and HDMI video output. The video recording device captures images of objects being transported along a conveyor belt. The parts inspection system mentioned above is a rough outline, and the location of the camera and means of transportation for transporting the object are all subject to design changes.

Referring to FIG. 2, the parts inspection system includes an image capture unit 100, a feature point generation unit 200, a parts extraction unit 300, a parts supply determination unit 400, a parts classification unit 500, and a location identification unit 600. ) and a database 700.

[Video shooting]

The image capture unit 100 includes a first image capture device 110 and a second image capture device 120. The first photographing device 110 and the second photographing device 120 are symmetrical to each other with respect to FIG. 1 and can continuously photograph an object by maintaining an inclination at a certain angle.

[Feature point extraction]

The feature point generation unit 200 is a device for identifying parts from images captured by the image capture unit 100, and may include a feature point extraction unit 210, a group identification unit 220, and a group extraction unit 230. there is. The feature point generator 210 generates at least one feature point from an image captured by the image capture unit 100. A feature point refers to a characteristic area such as a line, circle, or corner of a specific object in an image, and led corner edge detection, Image descriptor, SURF (Speeded Up Robust Feature), ORB (Oriented FAST and Rotated BRIEF) algorithms can be used. there is.

The present invention was created based on the FAST key point detector and BRIEF descriptor, and it is desirable to use the ORB (Oriented FAST and Rotated BRIEF) algorithm, which can achieve good performance with less calculation. Code change optimization for multi-GPU calculation can be performed to improve the recognition speed and real-time of the algorithm.

Additionally, the feature point extraction unit 210 may further include a noise filter, and remove feature points corresponding to noise from the extracted feature points. For example, the noise filter may determine whether feature points located within a certain area are noise based on whether they satisfy a preset number condition.

Referring to FIG. 3, it is a diagram showing the feature point extraction unit 210 extracting feature points from a captured image, and it can be seen that feature points have been created. The feature points shown in FIG. 3 are arbitrarily schematized to facilitate technical description of the present invention, and feature points generated in an actual image may be different from these.

The feature point extracted by the feature point extraction unit 210 extracts at least one or more two-dimensional feature point coordinates, preferably a plurality of them, from images captured by each of the first and second photographing devices 100 and 120.

The group identification unit 220 identifies adjacent feature points as one feature point group through preset conditions. As a preset condition, a method of identifying feature points located within an arbitrarily set distance in image pixel units as a group of feature points may be used.

Referring to FIG. 4, it can be seen that the first and second imaging devices 110 and 120 designated the feature points extracted from the object (P1) as one group (C1) and the other objects (P2) as another group (C2). there is.

Afterwards, the cluster extraction unit 230 extracts distribution information of the identified cluster of feature points. In other words, vector information is included by analyzing the size of the object designated as a group (C1, C2) and where the feature points are located.

[Acquisition of 3D coordinates]

The part extraction unit 300 receives the two-dimensional feature point coordinate information extracted from the first imaging device 110 and the two-dimensional feature point coordinate information extracted from the second photographing device 120, and uses the epipolar geometry principle ( It includes a coordinate generator 320 that generates three-dimensional feature point coordinates using an epipolar line. In addition, it further includes a noise removal unit 310 that removes feature points that are spaced apart from the two-dimensional feature points by more than a preset interval.

That is, using the information of the same feature points and the location of the feature points and the geometric relationship between each feature point from the epipolar geometry principle, a spatial model of the target object is created through curved application of interpolation when connecting the coordinates of each 3D feature point. can be implemented. In addition, by analyzing the 3D coordinates, characteristic values for the height and volume of the object can also be obtained, improving the accuracy of product classification.

[Judgment of match]

The parts supply determination unit 400 compares the reference coordinates of the 3D feature points of the object stored in the database 700 with the 3D feature point coordinates obtained from the part extraction unit 300 to determine whether they match. At this time, 3D information can be obtained including height and volume information of the product. Through the obtained information, the similarity between the parts information list stored in the database 700 and the distribution information of the feature point group is compared.

The parts information list is 3D representative coordinates registered by the administrator, and is data that stores distribution information for each part as well as data defining the part, such as part specifications, type, and volume. Afterwards, the vector group of feature points is compared to determine whether it matches the object. Therefore, the accuracy of the inspection system can be improved.

[Part Identification]

The part identification unit 500 analyzes a point cloud of feature point coordinates obtained from the feature point generation unit 200 or the part extraction unit 300 and compares the similarity with the feature point coordinates of the object stored in the database.

In the analysis of the point group of feature point coordinates, feature points located within a preset interval between feature points are designated into multiple groups. Afterwards, the shape of the designated group is compared with the shape of the point group of the parts list stored in the database 700, and the compared and analyzed data is stored in the database 700. Through this, the type of product of the object can be identified.

One group (A1) is set as a standard for the plurality of groups of feature points, and other groups (A2) adjacent to the set group (A1) are sequentially designated. Afterwards, the designation of all groups of feature points within the area designated as the cluster (C1) is completed, and data having a specific shape can be obtained.

[Part Count]

The parts identification unit 500 further includes a parts counting unit 510 that counts objects passing through the parts supply determination unit 400. In more detail, the parts counting unit 510 counts objects and determines whether the order of objects identified in the parts identification unit () is identified in a preset order.

In other words, while counting the objects, it is determined whether the objects are moving on the conveyor in order. For example, if the objects are PSU, PCB, and SPEAKER, they must move sequentially through the conveyor belt and be photographed in the video capture unit 100, but if the order in which they are listed on the conveyor belt does not match, an error judgment is made. The preset order of objects is determined by receiving and comparing information stored in the database 700. The arrangement of the preset order can be easily adjusted according to each task and process, and when an error in the order occurs, it is desirable to provide a separate signal means to stop the conveyor belt or provide a notification signal to the operator.

[Identification of location of object]

It further includes a location identification unit 600 that identifies whether the location of the object has changed. The location identification unit 600 first receives the two-dimensional feature point coordinates of the moved object, sets the coordinates of the reference object, and identifies whether the object's position has changed by comparing it with the two-dimensional feature point coordinates of the moving object.

In more detail, the location identification unit 600 randomly selects the feature point (B) of the reference object from the two-dimensional feature point generated by the feature point generator 200 based on the image first captured by the image capture unit 100. Set it. The feature points of the object that serve as the standard for setting are extracted from an area where feature points are dense, and at least two feature points (B1, B2...Bn) of the object that serve as the standard can be set. The data of the set feature point (B) is stored in the database 700. Meanwhile, as the number of reference feature points (B) increases, the accuracy of identifying the location of an object can be improved.

Thereafter, the two-dimensional feature point coordinates extracted from the image of another moving object are compared with the coordinate information of the reference feature point (B) of the object stored in the database 700 to identify whether the object's position has changed. If the feature point is located at the same location as the set feature point (B), the location of the object is considered to be the same. (see Figure 6)

If there is a change in position or an object that is difficult to identify is detected, the conveyor belt is stopped or a separate notification device is provided so that the worker can check the situation with the naked eye. If there is no change in the position of the object, a unique series of information on the identified object is provided. Numbers are assigned and only objects to which serial numbers are assigned are read to see if they match in the parts supply determination unit 400.

Therefore, accuracy can be improved by preventing errors in identifying products that are left and right symmetrical or have similar shapes as one part.

The sequence of the object parts inspection system of the present invention will be explained through FIG. 7.

An image of a specific product is captured through the image capture unit 100 (S1000).

The feature point extraction unit 210 generates feature points such as outlines and edges of a specific object from the image captured by the image capture unit 100 (S1010).

The group identification unit 220 identifies feature points located within a randomly set transaction on an image pixel basis as a group of feature points (S1020).

The cluster extraction unit 230 extracts distribution information of the cluster of feature points identified by the cluster identification unit 220 (S1030).

The part extraction unit 300 generates 3D feature point coordinates using the 2D feature point coordinates generated by the feature point generation unit 200 (S1040).

The location identification unit 500 identifies whether the object's location has changed and assigns a unique serial number. When the position of an object changes, a notification means is activated to notify the operator. (S1050)

The parts supply determination unit 400 compares the similarity between the three-dimensional information of the object given a serial number and the feature points of the parts information list stored in the database 700, and if the similarity is more than 90%, the information stored in the database 700 It is judged to be consistent. (S1060)

The parts inspection system of the present invention with this configuration uses two or more cameras to photograph parts from various angles and extract feature points, so the position and posture information of parts can be obtained very accurately, so that the parts can be classified, counted, and defected. It has the advantage of improving the accuracy of determining .

100: video recording unit 200: feature point generation unit
300: Parts extraction unit 400: Parts supply determination unit
500: Part identification unit 600: Location identification unit
700: database

Claims (4)

An image capture unit including at least two image capture devices to acquire a plurality of images by photographing a moving object from different angles;
a feature point generator that generates at least one feature point from each image captured by the image capture device and obtains two-dimensional feature point coordinates of the object;
a part extraction unit that receives two-dimensional feature point coordinate information generated through each of the image capture devices and obtains three-dimensional feature point coordinates through the intersection of each feature point;
It includes a parts supply determination unit that compares the reference coordinates of the three-dimensional feature points of the object stored in the database and the three-dimensional feature point coordinates obtained from the parts extraction unit to determine whether they match,
The feature point generation unit includes a feature point extraction unit that extracts a plurality of two-dimensional feature points from the image captured by the image capture unit;
It further includes a group identification unit that designates the two-dimensional feature points located within an arbitrarily set distance from the two-dimensional feature points extracted from the image as one feature point group,
A part identification unit that analyzes the point cloud of the feature point coordinates obtained from the feature point generator, compares the similarity with the feature point coordinates of the object stored in the database, and then identifies the object for each part; is further provided,
Analysis of the point cloud of the feature point coordinates specifies a number of feature point groups designated by the group identification unit into groups and compares the shape of the designated group with the shape of the parts list stored in the database to determine the type of the object. A parts inspection system characterized in that it identifies.
delete According to claim 1,
The parts identification unit further includes a parts counting unit that determines whether the identified objects are identified in a preset order.
According to claim 1,
A location identification unit that first receives the two-dimensional feature point coordinates of the moved object, sets the coordinates of the reference object, and then compares them with the two-dimensional feature point coordinates of the moving object to identify whether the object has changed in position; further comprising a. A parts inspection system characterized by:

Images