JP2021051024A - Inspection system - Google Patents

Abstract

To provide an inspection system with which it is possible to appropriately determine the presence of a defect in a test object having a structurally difficult to see area that cannot be viewed from the front.SOLUTION: According to one embodiment of the present invention, provided is an inspection system for inspecting a test object having a structurally difficult to see area that cannot be viewed from the front, said inspection system comprising a camera and an information processing device. The camera is constituted so as to image a plurality of inspection images that include the test object located in an inspection area, the plurality of inspection images being those that are imaged with respectively different viewpoints and visual lines so that the difficult to see area is visualized. The information processing device includes a determination unit, the determination unit being constituted so as to determine the presence of a defect in the test object, on the basis of the plurality of inspection images having been imaged by the camera.SELECTED DRAWING: Figure 1

Description

The present invention relates to an inspection system.

At the production site, an information processing device may be used instead of visual inspection by a person to inspect the appearance of the object to be inspected. For example, an image of an object to be inspected is acquired by a camera such as a camera, and the acquired image is image-processed by an information processing device to automatically extract a defective portion. Patent Document 1 discloses an inspection system in which an object to be inspected, which is a container, is continuously performed on a production line.

Japanese Unexamined Patent Publication No. 2016-85137

On the other hand, there are some tasks that the inspection system is not good at, and in the production line that includes such tasks, inspections are performed by humans. For example, in an inspection system as disclosed in Patent Document 1, there is a problem that a defective portion cannot be extracted in the case of an inspected object having a difficult-to-see region structurally invisible to the front.

The present invention has been made in view of such circumstances, and an object of the present invention is to provide an inspection system capable of appropriately determining the presence or absence of defects in an inspected object having a difficult-to-see region that cannot be viewed from the front. ..

According to one aspect of the present invention, it is an inspection system for inspecting an object to be inspected having a difficult-to-see region structurally invisible to the front, and includes a camera and an information processing device, and the camera is inspected. It is configured to capture a plurality of inspection images including the object to be inspected located in the region, and here, the plurality of inspection images are imaged from different viewpoints and lines of sight so as to image the difficult-to-see region. The information processing apparatus is provided with a determination unit, and the determination unit is configured to determine the presence or absence of defects in the object to be inspected based on the plurality of inspection images captured by the camera. What is provided is provided.

Since the inspection system according to the present invention includes the above-mentioned camera and an information processing device, a plurality of inspection images including an object to be inspected located in the inspection area are imaged from different viewpoints and lines of sight, and the inspection images are captured. It is configured to perform image processing to determine defects. Therefore, it is possible to automatically and appropriately determine the presence or absence of defects in the inspected object having a difficult-to-see area that cannot be viewed from the front. Even an object to be inspected having a complicated shape can be inspected at high speed. In other words, in the inspection process of the production line, defects can be automatically identified without human intervention, so that it is possible to prevent the occurrence of defective products due to oversight of defects.

Overall view of the inspection system according to the embodiment. The hardware block diagram of the inspection system which concerns on embodiment. Hardware configuration diagram of information processing device in inspection system. A functional block diagram showing the functions of the control unit of the information processing device. The activity diagram of the inspection system according to the embodiment. Inspection image diagram of the object to be inspected. Image drawing after image processing of the object to be inspected. The figure which was judged as a defect in the image figure after image processing of the object to be inspected.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. The various features shown in the embodiments shown below can be combined with each other. In particular, in the present specification, the "part" may include, for example, a combination of hardware resources implemented by circuits in a broad sense and information processing of software that can be concretely realized by these hardware resources. .. Further, in this embodiment, various information is handled, and these information are represented by high and low signal values as a bit set of binary numbers composed of 0 or 1, and communication / calculation is executed on a circuit in a broad sense. Can be done.

A circuit in a broad sense is a circuit realized by at least appropriately combining a circuit, a circuit, a processor, a memory, and the like. That is, an integrated circuit for a specific application (Application Special Integrated Circuit: ASIC), a programmable logic device (for example, a simple programmable logic device (Simple Programmable Logical Device: SPLD), a composite programmable logic device (Complex Program)) It includes a programmable gate array (Field Programmable Gate Array: FPGA) and the like.

1. 1. Overall configuration 1-1. Inspection system 1
Section 1 describes the overall configuration of the inspection system 1 according to the present embodiment. FIG. 1 is a diagram showing a hardware configuration of the inspection system 1 according to the present embodiment. The inspection system 1 for inspecting the object to be inspected B1 having a difficult-to-see region SA that is structurally invisible from the front is an image pickup device 2 (first camera 21A, second camera 21B, first illumination 22A, first lighting 22A, first 2), an information processing device 3, a transport device 4, and a display device 5 are provided. Each component will be described in more detail below.

1-2. Imaging device 2
The image pickup apparatus 2 includes a first camera 21A, a second camera 21B, a first illumination 22A, a second illumination 22B, a first robot arm 23A, and a second robot arm 23B. Details will be described below.

<First camera 21A>
The first camera 21A is configured to capture a plurality of inspection images IM1 including an object to be inspected B1 located in the inspection region R, and here, the plurality of inspection images IM1 are images of the difficult-to-see region SA without omission. The images were taken from different viewpoints and lines of sight. The first camera 21A is provided at the tip of the driveable first robot arm 23A, and has a variable viewpoint and line of sight. The first robot arm 23A moves to a predetermined position and images as an inspection image IM1 of the inspected object B1 from a predetermined line of sight, so that the inspected object B1 includes the difficult-to-see region SA that is structurally invisible to the front. The inspection image IM1 can be imaged.

<Second camera 21B>
The second camera 21B is configured to capture a plurality of inspection images IM1 including the object to be inspected B1 located in the inspection region R, where the plurality of inspection images IM1 are images of the difficult-to-see region SA without omission. The images were taken from different viewpoints and lines of sight. The second camera 21B is provided at the tip of the driveable second robot arm 23B, and has a variable viewpoint and line of sight. The second camera 21B is configured to be capable of capturing the inspection image IM1 of the object to be inspected B1 that cannot be captured by the first camera 21A outside the movable range of the first robot arm 23A. By providing the second camera 21B in addition to the first camera 21A, it is possible to capture the entire inspection image IM1 including the difficult-to-see region SA that is structurally invisible to the front of the object B1 to be inspected. As described above, the first camera 21A and the second camera 21B are a plurality of cameras arranged so as to have different viewpoints and lines of sight.

<First lighting 22A>
The first lighting 22A is attached to the first robot arm 23A. The first robot arm 23A is configured to move to a predetermined position and illuminate the object B1 to be inspected when the first camera 21A captures the inspection image IM1 of the object B1 to be inspected. By moving the first illumination 22A to a predetermined position in conjunction with the first robot arm 23A, the object B1 to be inspected can always be irradiated with the object B1 to be inspected from substantially the same line of sight at the time of imaging. it can. As a result, the inspection image IM1 which is less affected by the shadow can be imaged.

<Second lighting 22B>
The second illumination 22B is attached to the second robot arm 23B. The second robot arm 23B is configured to move to a predetermined position and illuminate the object B1 to be inspected when the second camera 21B captures the inspection image IM1 of the object B1 to be inspected. Like the first illumination 22A, the second illumination 22B can always irradiate the object to be inspected B1 from substantially the same line of sight at the time of imaging, and the inspection image IM1 which is less affected by shadows can be imaged.

The first illumination 22A and the second illumination 22B are variably configured at least one of their positions, orientations, and illumination intensities according to the viewing point and line of sight of the first camera 21A and the second camera 21B. .. With such a configuration, even if the illuminance of the first camera 21A and the second camera 21B decreases due to aging, when the inspection image IM1 of the object to be inspected B1 is imaged, the first robot arm It is possible to reduce the shadow of the inspection image IM1 caused by the shapes of the 23A, the second robot arm 23B, the object to be inspected B1, and the like.

<First robot arm 23A>
The first camera 21A and the second camera 21B are configured to capture a plurality of inspection images IM1 including the object to be inspected B1 located in the inspection area R, so that the first robot arm 23A and the second camera arm 23A and the second camera 21B It is attached to the robot arm 23B. As a result, the inspection image IM1 can be imaged from a plurality of predetermined positions from different viewpoints and lines of sight, so that the difficult-to-see region SA, which is structurally impossible to see in front, can be imaged without omission. The first robot arm 23A is configured to be movable in order to sequentially move to a predetermined position based on a command from the communication unit 31 of the information processing device 3 and to capture the inspection image IM1 of the object to be inspected B1.

<Second robot arm 23B>
Like the first robot arm 23A, the second robot arm 23B also sequentially moves to a predetermined position outside the movable range of the first robot arm 23A based on a command from the communication unit 31 of the information processing device 3, and is covered. It is configured to be movable to capture the inspection image IM1 of the inspection object B1.

1-3. Information processing device 3
FIG. 2 is a block diagram showing a hardware configuration of the information processing device 3. FIG. 3 is a functional block diagram showing a function carried out by the control unit 33 in the information processing device 3. The information processing device 3 has a communication unit 31, a storage unit 32, and a control unit 33 (image processing unit 332, determination unit 333), and these components provide a communication bus 30 inside the information processing device 3. It is electrically connected via. Hereinafter, each component will be further described.

<Communication unit 31>
Although wired communication means such as USB, IEEE1394, Thunderbolt, and wired LAN network communication are preferable, the communication unit 31 performs wireless LAN network communication, mobile communication such as LTE / 3G, Bluetooth (registered trademark) communication, and the like as necessary. May be included. That is, it is more preferable to carry out as a set of these plurality of communication means. In particular, it is preferable that the first camera 21A and the second camera 21B are configured to be communicable according to a predetermined communication standard.

The communication unit 31 is configured to receive the plurality of inspection images IM1 when the first camera 21A and the second camera 21B capture the plurality of inspection images IM1. Further, the communication unit 31 is configured to be able to transmit information for controlling each joint angle of the first robot arm 23A and the second robot arm 23B described above to a desired angle based on inverse kinematics. Further, it is more preferable that the current joint angle can be acquired as information (encoder). With such a configuration, the first camera 21A and the first illumination 22A (second camera 21B and second illumination 22B) provided at the tip of the first robot arm 23A (second robot arm 23B). Can be controlled to be displaced in a desired position and direction in space coordinates.

<Memory unit 32>
The storage unit 32 stores various information defined by the above description. This is, for example, as a storage device such as a solid state drive (SSD), or a random access memory (Random Access Memory:) that stores temporarily necessary information (arguments, arrays, etc.) related to program operations. It can be implemented as a memory such as RAM). Moreover, these combinations may be used. In particular, the storage unit 32 stores the inspection image IM1 imaged by the first camera 21A and the second camera 21B and received by the communication unit 31. The inspection image IM1 is, for example, array information including pixel information of 8 bits for each of RGB. Further, the storage unit 32 stores the acquisition program, the image processing program, the determination program, the result storage program, and the display control program. In addition to this, the storage unit 32 stores various programs related to the information processing device 3 executed by the control unit 33.

<Control unit 33>
The control unit 33 processes and controls the entire operation related to the information processing device 3. The control unit 33 is, for example, a central processing unit (CPU) (not shown). The control unit 33 realizes various functions related to the information processing device 3 by reading a predetermined program stored in the storage unit 32. Specifically, the acquisition function, the image processing function, the judgment function, the result saving function, and the display control function are applicable. That is, the information processing by the software (stored in the storage unit 32) is specifically realized by the hardware (control unit 33), so that the acquisition unit 331, the image processing unit 332, the determination unit 333, and the result storage unit It can be executed as 334 and display control unit 335. Although it is described as a single control unit 33 in FIG. 3, it is not actually limited to this, and it may be implemented so as to have a plurality of control units 33 for each function. Moreover, it may be a combination thereof. Hereinafter, the acquisition unit 331, the image processing unit 332, the determination unit 333, the result storage unit 334, and the display control unit 335 will be described in more detail.

[Acquisition unit 331]
In the acquisition unit 331, information processing by software (stored in the storage unit 32) is specifically realized by hardware (control unit 33). The acquisition unit 331 is configured to acquire the inspection image IM1 captured by the first camera 21A and the second camera 21B via the communication unit 31. FIG. 6 shows an example of the inspection image IM1 in which the carbide (carbide D1 in the inspection image) adhering to the difficult-to-see region SA structurally invisible to the front is imaged on the object B1 to be inspected.

[Image processing unit 332]
In the image processing unit 332, information processing by software (stored in the storage unit 32) is specifically realized by hardware (control unit 33). The image processing unit 332 determines a plurality of inspection image IM1 including the inspection object B1 located in the inspection area R transmitted from the first camera 21A and the second camera 21B and received by the communication unit 31. Is configured to perform image processing. More specifically, the image processing unit 332 executes predetermined image processing on the inspection image IM1 transmitted from the first camera 21A and the second camera 21B and received by the communication unit 31, and processes the inspection image IM1. Post-image IM2 is generated. More specifically, a predetermined threshold value is set for the brightness of the inspection image IM1 and converted into a binary image (an example of the processed image IM2). FIG. 7 shows an example of the processed image IM2 in which the inspection image IM1 of FIG. 6 is converted. The carbide is converted and extracted (carbide D2 in the binary image).

[Judgment unit 333]
In the determination unit 333, information processing by software (stored in the storage unit 32) is specifically realized by hardware (control unit 33). The determination unit 333 extracts defects in the appearance of the object to be inspected B1 having a difficult-to-see region SA that is structurally invisible to the front. As a result, the information processing apparatus 3 determines the presence or absence of defects in the object to be inspected B1 based on the plurality of inspection images IM1 captured by the first camera 21A and the second camera 21B. Specifically, the determination unit 333 is configured to determine the presence or absence of a defective carbide based on the binary image which is the processed image IM2. The determination unit 333 determines whether or not the defect corresponds to the defect based on at least one of the position and area of the defect in the appearance of the object to be inspected B1 calculated by the image processing unit 332. It should be noted that the noise of the first camera 21A and the second camera 21B, the calculation error in the image processing, or the minute defect that has no quality problem may be determined not to be a defect. Further, the determination unit 333 is based on the plurality of inspection images IM1 received by the communication unit 31, and the presence or absence of defects in the inspected object B1 being inspected by the time the conveying device 4 conveys the next inspected object B2. Is configured to determine. FIG. 8 shows an example of an image in which the above-mentioned carbide is determined to be defective (carbide D3 determined to be defective).

[Result storage unit 334]
In the result storage unit 334, information processing by software (stored in the storage unit 32) is specifically realized by hardware (control unit 33). After the inspection of the inspected object B1 is completed, the result storage unit 334 identifies the ID to identify the inspected object B1, the inspection result, the inspection time, and the type if there is a defect, for the purpose of quality analysis and analysis. Data such as the content (position and area of defects), the number of defects generated, the processed image IM2, the inspection process, and the like are stored in the storage unit 32.

[Display control unit 335]
In the display control unit 335, information processing by software (stored in the storage unit 32) is specifically realized by hardware (control unit 33). After the inspection is completed, the display control unit 335 determines the ID for identifying the above-mentioned object B1 to be inspected, the inspection result, the inspection time, the type if there is a defect, the content (position and area of the defect, etc.), the number of defects generated, and after processing. The image IM2 or the data of the inspection process or the like is displayed on the display device 5. Since these data display the quality status at the production site in real time, they are useful for quickly grasping and identifying problems and for quick countermeasures.

1-4. Transport device 4
The transport device 4 is configured to sequentially transport a plurality of objects B1 to be inspected to the inspection area R. The transfer device 4 may be a belt conveyor, a roller conveyor, a chain conveyor, or any other device that conveys the object B1 to be inspected, and the transfer type is not limited.

The transport device 4 is a device that transports the object B1 to be inspected to the inspection system 1, and a sensor (not shown) provided in the transport device 4 recognizes that the object B1 to be inspected has been transported to the inspection system 1. .. The sensor (not shown) may be any sensor that can detect the presence or absence of the object to be inspected B1. For example, a photoelectric sensor that can detect the presence or absence of the object to be inspected B1 in a non-contact manner is applicable.

The first robot arm 23A and the second robot arm 23B move to predetermined positions, and the first camera 21A and the second camera 21B structurally have a difficult-to-see region SA that cannot be seen from the front. Image the appearance of the robot. The communication unit 31 is configured to receive the plurality of inspection images IM1 received by the first camera 21A and the second camera 21B, and the determination unit 333 receives the plurality of inspection images IM1 received by the communication unit 31. Based on the above, the transport device 4 is configured to determine the presence or absence of defects in the inspected object B1 being inspected by the time the next inspected object B1 is conveyed. As described above, the inspection system 1 is provided with the transport device 4, so that the appearance inspection of the object to be inspected B1 can be continuously performed on-site. Since the presence or absence of defects can be continuously determined in a short time, the manufacturing cost can be reduced because defective products are not sent to the next process.

The inspection system 1 is effective when the object to be inspected B1 has a difficult-to-see region SA that is structurally invisible to the front. Therefore, the object to be inspected B1 is not limited to being a specific product as long as it has such a structure. The object to be inspected B1 may be a molded product or may have a difficult-to-see region SA that is structurally invisible to the front other than the molded product.

In particular, in the inspection system 1, the object to be inspected B1 is a container having a handle portion. The difficult-to-see area SA is an area inside the handle portion.

2. Inspection Method Section 2 describes an inspection method using the inspection system 1 described in Section 1. FIG. 5 is an activity diagram of the inspection system 1 according to the embodiment. Hereinafter, description will be given with reference to this figure.

[from here]
(Activity A11)
When the sensor (not shown) recognizes that the object to be inspected B1 has reached the inspection area R, the transfer device 4 stops the operation of the transfer device 4.

(Activity A12)
Subsequently, the first robot arm 23A and the second robot arm 23B move to predetermined positions in order to appropriately image the object B1 to be inspected.

(Activity A13)
Subsequently, the first camera 21A and the second camera 21B image the object B1 to be inspected. If the specified number of images has not been reached, the process returns to activity A12 so that the image can be taken from another viewpoint and line of sight.

(Activity A14)
The acquisition unit 331 in the information processing device 3 acquires the inspection image IM1 from the first camera 21A and the second camera 21B via the communication unit 31.

(Activity A15)
Subsequently, the image processing unit 332 in the information processing device 3 performs appropriate image processing on the inspection image IM1 acquired by the acquisition unit 331 to generate the processed image IM2.

(Activity A16)
Subsequently, the determination unit 333 in the information processing apparatus 3 determines the presence or absence of defects in the inspected object B1 based on the processed image IM2.

(Activity A17)
Subsequently, the result storage unit 334 in the information processing device 3 stores the result determined by the determination unit 333 in the storage unit 32.

(Activity A18)
Subsequently, the display control unit 335 of the information processing device 3 causes the display device 5 to display the result stored by the result storage unit 334. If the specified number of images has not been reached, the process returns to activity A14 so as to acquire an image captured from another viewpoint and line of sight again.

(Activity A19)
Finally, the transport device 4 is operated again to inspect the next object to be inspected B1.
[to this point]

3. 3. Modification Example Section 3 describes a modification of the inspection system 1 according to the present embodiment. That is, the inspection system 1 may be provided in the following aspects.

(1) Instead of the first robot arm 23A and the second robot arm 23B, the position coordinates of the first camera 21A, the second camera 21B, and the like may be changed by using the XYZ stage.
(2) When the difficult-to-see region SA, which is structurally invisible from the front, is captured without omission by the two inspection images IM1 captured by the first camera 21A and the second camera 21B, the first camera 21A and the second camera 21B may be fixed and carried out.
(3) In addition to the first camera 21A and the second camera 21B, additional cameras may be added.

4. Conclusion As described above, according to the present embodiment, it is possible to realize an inspection system capable of appropriately determining the presence or absence of defects in the subject to be inspected having a difficult-to-see region that cannot be viewed from the front.

It may be provided in each of the following aspects.
In the inspection system, the camera is provided at the tip of a driveable robot arm, and the viewpoint and the line of sight are variably configured.
In the inspection system, the cameras are a plurality of cameras arranged so that the viewpoint and the line of sight are different.
The inspection system further includes a transport device, which is configured to sequentially transport a plurality of the objects to be inspected to the inspection area.
In the inspection system, the information processing device further includes a communication unit, the communication unit is configured to receive the plurality of inspection images when the camera captures the inspection images, and the determination unit is the communication unit. Based on the plurality of inspection images received by the inspector, the transfer device is configured to determine the presence or absence of defects in the inspected object before the next inspected object is conveyed.
In the inspection system, the information processing apparatus further includes an image processing unit, and the image processing unit sets a predetermined threshold value for the brightness of the inspection image and converts it into a binary image. It is configured to determine the presence or absence of the carbon dioxide which is the defect based on the binary image.
The inspection system further includes lighting, which is configured to illuminate the object to be inspected when the camera captures the inspection image.
In the inspection system, the illumination is configured to have at least one of its position, orientation, and illumination intensity variably configured according to the viewpoint and the line of sight of the camera.
In the inspection system, the object to be inspected is a molded product.
In the inspection system, the object to be inspected is a container having a handle portion, and the difficult-to-see area is an area inside the handle portion.
Of course, this is not the case.

Finally, various embodiments of the present invention have been described, but these are presented as examples and are not intended to limit the scope of the invention. The novel embodiment can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the gist of the invention. The embodiment and its modifications are included in the scope and gist of the invention, and are included in the scope of the invention described in the claims and the equivalent scope thereof.

1: Inspection system 2: Image pickup device 21A: First camera 21B: Second camera 22A: First lighting 22B: Second lighting 23A: First robot arm 23B: Second robot arm 3: Information processing Device 30: Communication bus 31: Communication unit 32: Storage unit 33: Control unit 331: Acquisition unit 332: Image processing unit 333: Judgment unit 334: Result storage unit 335: Display control unit 4: Conveyor device 5: Display device B1: Object B2 to be inspected: Object to be inspected D1: Carbide D2: Carbide D3: Carbide IM1: Inspection image IM2: Post-processed image R: Inspection area SA: Hard-to-see area

Claims (10)

It is an inspection system for inspecting an object to be inspected that has a difficult-to-see area that cannot be seen from the front.
Equipped with a camera and an information processing device
The camera is configured to capture a plurality of inspection images including the object to be inspected located in the inspection area, and is different from the plurality of inspection images so as to image the difficult-to-see area. It was taken from the viewpoint and line of sight.
The information processing device includes a determination unit.
The determination unit is configured to determine the presence or absence of defects in the object to be inspected based on the plurality of inspection images captured by the camera. In the inspection system according to claim 1,
The camera is provided at the tip of a driveable robot arm, and the viewpoint and the line of sight are variably configured. In the inspection system according to claim 1,
The camera is a plurality of cameras arranged so that the viewpoint and the line of sight are different. In the inspection system according to claim 1,
Equipped with a transfer device
The transport device is configured to sequentially transport a plurality of the objects to be inspected to the inspection area. In the inspection system according to claim 4,
The information processing device further includes a communication unit.
The communication unit is configured to receive the plurality of inspection images when the camera captures them.
Based on the plurality of inspection images received by the communication unit, the determination unit determines whether or not the inspection object is defective before the transfer device conveys the next inspection object. What is configured to do. In the inspection system according to any one of claims 1 to 5.
The information processing device further includes an image processing unit.
The image processing unit sets a predetermined threshold value for the brightness of the inspection image and converts it into a binary image.
The determination unit is configured to determine the presence or absence of carbides, which are the defects, based on the binary image. In the inspection system according to any one of claims 1 to 6.
With more lighting
The illumination is configured to illuminate the object to be inspected when the inspection image is captured by the camera. In the inspection system according to claim 7,
The illumination is configured to have at least one of its position, orientation, and illumination intensity variably configured according to the viewpoint and the line of sight of the camera. In the inspection system according to any one of claims 1 to 8.
The object to be inspected is a molded product. In the inspection system according to any one of claims 1 to 9.
The object to be inspected is a container having a handle portion.
The difficult-to-see area is an area inside the handle portion.

Images