KR20220129770A - Apparatus for checking product - Google Patents

Abstract

According to an embodiment of the present invention, a product checking device comprises: a processing unit processing a capturing image about a product; and database storing reference information about a feature point of the capturing image. The processing unit generates the feature point of the product through the capturing image and derives whether there is a defect of the product through a distribution degree about the feature points of each of a plurality of unit sections of the capturing image and the reference information.

Description

Product Checking Device {APPARATUS FOR CHECKING PRODUCT}

The present invention relates to a product identification device.

Due to product trends and inventory management, the need for small-volume production of multiple varieties is increasing, but factories with automated facilities are not able to respond quickly to the frequently changing variety changes. In addition, in small factories that rely on manpower rather than automated facilities, the cost of products rises due to increased labor costs.

A number of parts such as screws and wires are assembled into one product, and even if one part is missing, it becomes a defective product. Defective products are inspected by manpower, but due to the nature of manpower, the error rate inevitably increases as the types of products increase.

Publication No. 10-2019-0061660 (published date: June 05, 2019)

The device for checking a product according to an embodiment of the present invention is for automatically checking a defective state of a product.

The task of the present application is not limited to the task mentioned above, and another task that is not mentioned will be clearly understood by those skilled in the art from the following description.

According to an aspect of the present invention, the processing unit for processing the captured image of the product; and a database for storing reference information on the feature points of the captured image, wherein the processing unit generates the feature points of the product through the captured image, and the degree of distribution of the feature points in each of the plurality of unit areas of the captured image and whether the product is defective is derived through the reference information.

The processing unit derives a unit area in which the density of the feature points among the plurality of unit areas is greater than or equal to the reference area density value, except for a unit area in which the density of the feature points is smaller than the reference area density value, among the plurality of unit areas. Whether the product is defective can be derived through the feature points of the unit area.

The processing unit excludes a sub-region in which the density of feature points among a plurality of sub-regions of the derived unit region is smaller than a reference sub-region density value, and includes a sub-region in which the density of feature points among the plurality of sub-regions is equal to or greater than a reference sub-region density value. Deriving a zone, reading the reference information corresponding to the derived feature point of the sub zone from the database, and comparing the derived feature point of the sub zone with the read reference information to determine whether the product is defective have.

When the degree of distribution of the feature points included in each of the derived sub-regions is greater than a reference value, the processing unit derives whether the product is defective according to the analysis of the density of the feature points among the sub-regions of the sub-region, and the When the distribution degree of the feature points included in each of the derived sub-regions is smaller than the reference value, it is possible to derive whether the product is defective through the feature points of the sub-regions.

The processing unit reads a feature point of the reference information corresponding to the derived unit zone from the database, calculates a similarity between the derived feature point of the unit zone and the feature point of the reference information, and derives a representative point of the derived unit zone can do.

The processing unit reads a feature point of the reference information corresponding to the derived sub-region from the database, calculates a similarity between the derived feature point of the sub-region and the feature point of the reference information, and derives a representative point of the derived sub-region can do.

The product checking apparatus according to an embodiment of the present invention may automatically check the defective state of the product through the analysis of the captured image of the product.

The effect of the present application is not limited to the above-mentioned effects, and another effect not mentioned will be clearly understood by those skilled in the art from the following description.

1 shows a product identification device according to an embodiment of the present invention.
2 shows an example of a product identification system including a product identification device according to an embodiment of the present invention.
3 shows an example of a captured image.
4 is for explaining the density of the unit area and the feature point.
5 is for explaining the density of sub-regions and feature points.
6 and 7 are for explaining a sub-region and a lower sub-region.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, the accompanying drawings are only described in order to more easily disclose the contents of the present invention, and the scope of the present invention is not limited to the scope of the accompanying drawings. you will know

In addition, the terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly dictates otherwise.

In the present application, terms such as “comprise” or “have” are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, but one or more other features It is to be understood that this does not preclude the possibility of the presence or addition of numbers, steps, operations, components, parts, or combinations thereof.

1 shows a product identification apparatus 100 according to an embodiment of the present invention. The product identification device 100 according to an embodiment of the present invention may include a bus 102 or other communication mechanism for communicating information.

This bus 102 or other communication mechanism interconnects one or more of the processing unit 104, memory unit RM, communication unit 112, display unit 114, input unit 118, and/or subsystems. do.

Transmission media including wires including bus 102 include coaxial cables, copper wire, and optical fibers.

The memory unit RM may include a volatile memory such as RAM and a non-volatile memory. Non-volatile memory includes, but is not limited to, ROM and one or more disk drives 110 (eg, HDDs, SSDs, optical disks, flash memory drives, etc.). The optical disc of the disc drive may include, but is not limited to, a CD, DVD, Blu-ray disc.

The product confirmation apparatus 100 according to the embodiment of the present invention is connected to an external disk drive through the communication unit 112 to receive information or logic necessary for the operation of the product confirmation apparatus 100 according to the embodiment of the present invention. can

The memory unit RM may store an operating system, a driver, an application program, data, and a database required for the operation of the product identification apparatus 100 .

The display unit 114 may display an operation situation and a user interface of the 3D model restoration apparatus 100 under the control of the processing unit 104 . Such a display unit 114 may be a CRT, LCD, LED, or OLED, but is not limited thereto.

The input unit 118 may be, but is not limited to, a keyboard, a keypad, a virtual keyboard, a mouse, a trackball, a stylus, a touch sensing means, etc. according to the control of the processing unit 104 .

The interfacing unit 120 enables connection and information transmission between the product identification device 100 of the present invention and a peripheral device and may include at least one of a USB interface, an HDMI interface, and a Thunderbolt interface, but is limited thereto not.

The one or more processing units 104 may be, but are not limited to, a CPU, an application processor (AP), a microcontroller, a digital signal processor (DSP), or the like. Also, the one or more processing units 104 may include a graphics processing unit (GPU).

The processing unit 104 controls the operation of the product identification apparatus 100 to be described later by accessing the memory unit RM and executing one or more sequences of commands or logic stored in the memory unit RM. Also, the GPU of the processing unit 104 may operate exclusively for graphic operation.

The processing unit 104 processes the captured image of the product, which will be described in detail later with reference to the drawings.

The communication unit 112 may perform at least one of wired communication and wireless communication through a communication link. For example, the communication unit 112 may support a LAN, a wireless LAN, a mobile communication network, Bluetooth, and NFC, but is not limited thereto.

Such a product confirmation apparatus 100 according to an embodiment of the present invention may be implemented through an Nvidia TX board suitable for graphic operation, but is not limited thereto.

2 shows an example of a product identification system including a product identification device 100 according to an embodiment of the present invention. As shown in FIG. 2 , the product confirmation system may include an image capturing unit 210 including a camera. The Nvidia TX board described above can provide an interface module as well as a high-speed graphics operation. The interface module can perform functions such as USB 3.0, WIFI communication, and HDMI video output.

The product to be photographed is positioned on the transport belt 220 , and the image capturing unit 210 takes a captured image of the product being transported. The belt driving unit 230 may include a motor for driving the belt 220 for transport and a power supply unit for supplying power to the motor.

On the other hand, the configuration of the product confirmation system of FIG. 2 is an example, and the position of the image capturing unit 210 , the conveying belt 220 or the belt driving unit 230 are all design changes.

The image capturing unit 210 may include at least one camera capable of capturing an image. The image capturing unit 210 may generate a captured image of the product. For example, FIG. 3 is a photographed image of a product generated by the image capturing unit 210, and the product of FIG. 3 may include a plurality of screws and air outlets.

The processing unit 104 generates feature points of the product through the captured image. A characteristic point may be set in a region characterized by a line, a circle, a corner, etc. of a specific object in the captured image, and a characteristic point group consisting of one or more characteristic points may be set in the characteristic region of the product.

For example, like the dotted line region of FIG. 3 , a feature point and a feature point group may be set in a region characterized by a product, such as a screw or an air outlet.

The processing unit 104 may use a led corner edge detection, an image descriptor, a Speeded Up Robust Feature (SURF), an Oriented FAST and Rotated BRIEF (ORB) algorithm, and the like. ORB algorithm, based on FAST key point detector and BRIEF descriptor, can achieve good performance with few calculations.

The processing unit 104 may generate feature points, such as contour lines, corners, etc. of a product or a part provided in the product.

The processing unit 104 derives whether or not the product is defective through the distribution degree and reference information for the feature points of each of the plurality of unit regions of the captured image. For example, as shown in FIG. 3 , the processing unit 104 may divide the captured image into 8×5 unit areas, but the present invention is not limited to the unit area of FIG. The number may be changed.

The processing unit 104 may calculate the degree of distribution of the feature points in each of the unit regions. For example, the degree of distribution of key points may relate to the number of key points per unit area or to which part of the unit area where the key points are concentrated, but is not limited thereto.

The product identification apparatus 100 according to the embodiment of the present invention includes a database for storing reference information on the feature points of the captured image. The reference information may be generated in advance and stored in a database. In order to determine whether the product is defective, the reference information may include information on characteristic points for a normal product.

When the product to be compared is transferred through the conveying belt 220 of FIG. 2 , the image capturing unit 210 generates a captured image, and the product identification apparatus 100 and 100 of the present invention are characteristic points of the captured image. can be deduced whether the product to be compared is defective or not by analyzing it through the reference information stored in the database.

For example, it can be determined whether the screw is provided in a normal position by comparing the feature points and reference information for the screw of the product to be compared shown in FIG. 3 .

Such a database may be stored in the memory unit RM, and the processing unit 104 may access the database located remotely through the communication unit 112 .

The processing unit 104 may exclude a unit region in which a density of feature points is smaller than a reference region density value from among the plurality of unit regions. Also, the processing unit 104 may derive a unit region in which the density of feature points is equal to or greater than the reference region density value among the plurality of unit regions. Through this, the processing unit 104 may derive whether the product is defective through the derived feature points of the unit area.

For example, as shown in FIGS. 3 and 4 , a feature point does not exist in the hatched unit area, and accordingly, the density of each feature point in the hatched unit area is smaller than the reference area density value.

Accordingly, the processing unit 104 excludes the hatched unit area from the entire unit area to be analyzed from the unit area to be analyzed, and for the remaining unit areas, that is, the unit area in which the density of the feature points is equal to or greater than the reference area density value. can be analyzed.

By the operation of the present invention as described above, feature point analysis of the captured image can be performed quickly. In the case of a general technique, since feature points located in each pixel region constituting the captured image are analyzed, the amount of calculation and the calculation time may increase.

The reference area density value may be stored in the memory unit RM. The reference area density value may be changed by the designer or operator. The reference area density value may be initially determined at the time of product or part registration. For example, when the number of feature points is a small part, the reference area density value may be adjusted downward.

The product analysis apparatus of the present invention may derive whether the product is defective according to the distribution of feature points in the unit area, or may derive whether the product is defective according to the distribution of the feature points in the sub-regions constituting the unit area.

The sub-region will be described in detail later with reference to the drawings.

Meanwhile, the processing unit 104 may exclude a sub-region having a density of feature points smaller than a reference sub-region density value among a plurality of sub-regions of the derived unit region. As described above, the derived unit area is a unit area in which the density of feature points is equal to or greater than the reference area density value. Also, the processing unit 104 may derive a sub-region having a density of feature points equal to or greater than a reference sub-region density value among the plurality of sub-regions.

For example, as shown in FIGS. 4 and 5 , the unit region A is composed of four sub-regions, and a feature point and a feature point group corresponding to a screw exist in the sub-region A-1.

Accordingly, the density of the feature points in the sub-region A-1 is greater than the reference sub-region density value, and the density of the feature points in the sub-regions A-2, A-3, and A-4 is smaller than the reference sub-region density value.

The processing unit 104 may analyze the sub-region A-1 by excluding the sub-regions A-2, A-3, and A-4 from the analysis target among all the sub-regions.

In this case, the reference sub-region density value may be stored in the memory unit RM, and the number of sub-regions in the unit area is not limited to FIG. 5 .

The processing unit 104 reads reference information corresponding to a feature point of a sub-region (eg, sub-region A-1 in FIG. 5 ) derived from the database, and reads the reference information and the feature point of the derived sub-region. By comparison, it is possible to derive whether the product is defective or not.

On the other hand, when the distribution degree of the feature points included in each of the derived sub-regions is greater than the reference value, the processing unit 104 may derive whether the product is defective according to the analysis of the density of the feature points among the sub-regions of the sub-region. can

When the distribution degree of the feature points included in each of the derived sub-regions is smaller than the reference value, the processing unit 104 may derive whether the product is defective through the feature points of the sub-regions.

For example, as shown in FIGS. 6 and 7 , the processing unit 104 may calculate the degree of distribution of key points between sub-regions through a difference in the number of key points between sub-regions.

As shown in FIG. 6 , the processing unit 104 may calculate the difference in feature points between the 2×2 sub-regions constituting the sub-region A-1 of the unit region A. The number of feature points in the sub-regions a-1 to a-4 is 6, 4, 2, and 4, respectively, and the difference in the number of feature points in the sub-sub-zones a-1 and a-3 is the largest as 4 .

When the reference value is 6, since the difference in the number of feature points in the lower sub-regions a-1 to a-4 is less than 6, the processing unit 104 analyzes the feature points in the sub-region A-1 without analyzing the lower sub-region. Whether or not the product is defective can be derived.

On the other hand, as shown in FIG. 7 , the number of feature points in the lower sub-zones a-1 to a-4 is 9, 2, 0, and 4, respectively, and the number of feature points in the lower sub-zones a-1 and a-3 The difference is the largest as 9.

When the reference value is 6, the maximum value 9 of the difference in the number of feature points in the lower sub-regions a-1 to a-4 is less than 6, so the processing unit 104 performs feature point analysis in the lower sub-region of the sub-region A-1. Through this, it is possible to derive whether the product is defective or not.

Meanwhile, as described above with reference to FIGS. 3 and 4 , the processing unit 104 selects a unit area in which the density of the feature points is equal to or greater than the reference area density value, except for a unit area in which the density of the feature points is smaller than the reference area density value. It is possible to derive whether the product is defective or not through the derived feature points of the unit area.

The processing unit 104 reads the feature points of the reference information corresponding to the derived unit zone from the database, calculates the similarity between the derived feature points of the unit zone and the feature points of the reference information to derive representative points of the derived unit zone. can

Meanwhile, as described above with reference to FIGS. 4 and 5 , the processing unit 104 excludes a sub-region in which the density of feature points is smaller than the reference sub-region density value, and the sub-region in which the density of the feature points is greater than or equal to the reference sub-region density value. area can be derived.

The processing unit 104 reads the feature point of the reference information corresponding to the derived sub-region from the database, calculates the similarity between the feature point of the derived sub-region and the feature point of the reference information, and derives the representative point of the derived sub-region. have.

Such representative points of the present invention will be described.

The reference information on the feature point includes vector information about where the feature point is located, what size it is, in which direction it is, and the like, and the gradation value of each feature point, and such vector information may be stored in a database.

The processing unit 104 may calculate the similarity based on the vector information and grayscale values of the feature points in the unit region, sub-region, or lower sub-region of the captured image and the vector information and gray scale values of the feature points stored in the database.

The processing unit 104 may translate and rotate the feature point of the captured image through the vector information to match the position of the feature point stored in the database. In addition, the processing unit 104 may derive the similarity through an error between the feature points of the captured image matched with each other and the grayscale values of the feature points stored in the database.

The similarity calculation method described above is only an example and is not limited thereto.

When the number of feature points having the degree of similarity greater than the reference degree of similarity is greater than the reference number, the processing unit 104 may derive a representative point from the feature points having the degree of similarity greater than the reference degree of similarity.

For example, the processing unit 104 calculates the similarity through the feature points in one of the unit region, the sub-region, and the lower sub-region of the captured image. points can be derived.

A parts information list is stored in the database, and the parts information list may include not only data defining parts such as specifications and types of parts registered by the administrator, but also information on the characteristic points of each part and the representative points of these characteristic points. .

The information on the representative points may include representative points of at least some of all the feature points corresponding to each part. For example, as exemplified above, when the number of feature points greater than the reference similarity among feature points in one of the unit region, sub-region, and sub-region of the captured image is 70, the database contains representative points for the 70 feature points. can be pre-saved.

The processing unit 104 may read from the database representative points of the feature points satisfying the similarity condition from the database. The processing unit 104 may identify a specific part through the representative point.

If n representative points for each part are stored in the database, and m representative points through analysis of captured images (n>m, m and n are natural numbers), unidentified representative points may exist.

The processing unit 104 may determine that the component corresponding to the unidentified representative point is not provided in the product or that the wrong component is provided, thereby deriving the product as a defective state.

In this case, the processing unit 104 may perform similarity analysis once more on the feature points around the representative points corresponding to the unidentified parts. That is, when the number of feature points having a similarity greater than the reference similarity among the feature points around the representative point corresponding to the unidentified part is greater than the reference number, the processing unit 104 may determine that the part exists. The feature point around the representative point may be a feature point within a reference distance from the representative point.

As described above, the embodiments according to the present invention have been reviewed, and the fact that the present invention can be embodied in other specific forms without departing from the spirit or scope of the present invention in addition to the above-described embodiments is recognized by those with ordinary skill in the art. It is self-evident to Therefore, the above-described embodiments are to be regarded as illustrative rather than restrictive, and accordingly, the present invention is not limited to the above description, but may be modified within the scope of the appended claims and their equivalents.

Product Verification Device (100)
processing unit 104
Memory unit (RM)
communication unit (112)

Claims (6)

a processing unit for processing a captured image of a product; and
and a database for storing reference information on the feature points of the captured image,
the processing unit
generating a feature point of the product through the captured image,
Product identification apparatus, characterized in that the product is deriving whether the defect or not through the reference information and the degree of distribution of each of the feature points of the plurality of unit areas of the captured image. According to claim 1,
the processing unit
Except for a unit area in which the density of feature points among the plurality of unit areas is smaller than the reference area density value,
A product confirmation apparatus, characterized in that by deriving a unit area in which the density of the feature points is greater than or equal to a reference area density value among the plurality of unit areas, and deriving whether the product is defective through the derived feature points of the unit area. 3. The method of claim 2,
the processing unit
Except for a sub-region in which the density of feature points among a plurality of sub-regions of the derived unit region is smaller than the reference sub-region density value,
Deriving a sub-region in which the density of feature points among the plurality of sub-regions is equal to or greater than the reference sub-region density value,
reading the reference information corresponding to the feature point of the derived sub-region from the database,
Product identification device, characterized in that the deriving whether the product is defective by comparing the derived feature points of the sub-region with the read reference information. 4. The method of claim 3,
the processing unit
When the degree of distribution of the feature points included in each of the derived sub-regions is greater than the reference value, whether the product is defective is derived according to the analysis of the density of the feature points among the sub-regions of the sub-region,
When the degree of distribution of the feature points included in each of the derived sub-regions is smaller than a reference value, the product confirmation apparatus, characterized in that the product is defective through the feature points of the sub-regions. 3. The method of claim 2,
the processing unit
Reading the feature points of the reference information corresponding to the derived unit area from the database,
Product identification apparatus, characterized in that the derived representative point of the unit area by calculating the similarity between the derived feature point of the unit area and the reference information feature point. 4. The method of claim 3,
the processing unit
Reading the feature points of the reference information corresponding to the derived sub-region from the database,
Product identification apparatus, characterized in that the derived representative point of the sub-region is derived by calculating a similarity between the derived feature point of the sub-region and the feature point of the reference information.

Images