JP7188821B1 - inspection system - Google Patents

Abstract

A new inspection system capable of detecting minute defects on a target surface is provided even when the target surface is moving. An inspection system (10) includes a processing device (20), a display device (28), a projection device (50), and an imaging device (60) movable relative to an object surface. The processing device 20 includes a device main body 22 . The image capturing device 60 captures the captured image two or more times during the relative movement of the image capturing device 60 with respect to the target surface. The projection device 50 is moving relative to the target surface while the image capturing device 60 captures the captured image. The device body 22 receives each of the captured images and detects defects while reducing the resolution of the received captured images. The display device 28 displays the defects detected by the device body 22 . [Selection drawing] Fig. 1

Description

The present invention relates to an inspection system for detecting defects such as scratches and dents formed on the surface of an object.

For example, Patent Literature 1 discloses an inspection apparatus that can be used as this type of inspection system.

Patent Literature 1 discloses a surface inspection device for inspecting the surface of an object such as a car. A surface inspection device includes an imaging unit and a robot arm. The imaging unit includes an illumination device and a line sensor camera (imaging device). The imaging unit is supported by a robot arm and moves along a preset route. A predetermined area illuminated by the illumination device is photographed while the photographing unit is moving. A surface inspection apparatus detects surface defects based on a photographed image of a predetermined area. According to Patent Document 1, in order to shorten the scanning time of the robot arm, a predetermined area is photographed while the photographing unit is moving.

Japanese Unexamined Patent Application Publication No. 2008-046103

The inspection apparatus of Patent Document 1 is considered suitable for detecting relatively large defects such as scratches and foreign matter. On the other hand, defects on the surface of the object (target surface) include minute defects called bumps, repellency, dents, and the like. The inspection apparatus of Patent Document 1 is not suitable for detecting such minute defects. Further, when the object surface is moved while the inspection apparatus of Patent Document 1 is fixed so as not to move, vibration in the moving direction of the object surface cannot be completely prevented. Therefore, in such a case, the line sensor camera cannot continuously photograph the target surface. That is, according to the invention of Patent Document 1, it is necessary to fix the target surface so that it does not move.

SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to provide a new inspection system capable of detecting minute defects on a target surface even when the target surface is moving.

The present invention, as a first inspection system,
An inspection system for detecting defects in a target surface, comprising:
The inspection system includes a processing device, a display device, a projection device, and an imaging device,
The projection device is capable of projecting a planar projection image onto the target surface,
The projected image includes one or more bright portions and one or more dark portions having a luminance lower than that of the bright portions,
The imaging device is arranged so as to be able to capture the projection image projected onto the target surface,
The processing device includes a device body,
The device main body is capable of communicating with the imaging device,
The imaging device captures a captured image including the projected image while moving relative to the target surface;
The projection device moves relative to the target surface while the imaging device captures the captured image,
The device main body receives the captured image from the imaging device, detects the defect while reducing the resolution of the received captured image,
The display device provides an inspection system that displays the defects detected by the device body.

Further, the present invention provides a first inspection system as a second inspection system,
The device main body performs inspection processing on the received captured image to detect defects,
The inspection processing includes defect detection processing using a neural network,
The defect detection processing includes encoding processing, post-processing, and defect identification processing,
In the encoding process, the captured image is convolved to reduce the resolution while extracting features to generate a first output layer;
In the post-processing, while maintaining the low resolution of the first output layer, an intermediate layer obtained during the encoding process is superimposed to generate a second output layer;
The defect identification process provides an inspection system that identifies regions of the defect based on the second output layer.

Further, the present invention provides a third inspection system, which is the first or second inspection system,
The imaging device is fixed so as not to move,
Providing an inspection system in which the target surface is moving relative to the imaging device as the imaging device captures the projection image.

Further, the present invention provides a fourth inspection system, which is the first or second inspection system,
The inspection system comprises a support device,
The support device has a fixed portion and a support portion,
The fixed part is fixed so as not to move,
The support portion is relatively movable with respect to the fixed portion,
The imaging device is supported on the support, thereby providing an inspection system that is movable along the object surface.

Further, the present invention provides a fourth inspection system as a fifth inspection system,
The inspection system comprises an optical head,
The optical head includes the projection device, the imaging device, and a supported portion,
The projection device and the imaging device are fixed to each other,
The supported portion is supported by the support portion, whereby the optical head maintains the distance between the photographing device and the target surface within a range in which the photographing device can photograph the projected image. is movable along the target surface while
Each of the projection device and the imaging device is directly or indirectly fixed to the supported portion,
The projection device projects the projection image along a projection direction,
The imaging device captures the projected image along an optical axis,
When the photographing device photographs the projection image, the projection direction of the projection device and the optical axis of the photographing device intersect at a predetermined angle across a normal line at an intermediate point located in the middle of the projection image. To provide an inspection system arranged to:

Further, the present invention provides a fourth or fifth inspection system as a sixth inspection system,
The support device provides an inspection system that is a robotic arm.

In addition, the present invention is any one of the fourth to eighth inspection systems as a seventh inspection system,
The inspection system provides that the object surface is moving relative to the fixed part of the support device when the imaging device captures the projection image.

Further, the present invention provides a seventh inspection system as an eighth inspection system,
The photographing device moves along a predetermined movement route,
The movement path provides an inspection system including a path along a direction oblique to the direction of movement of the object surface.

Further, the present invention provides an eighth inspection system as a ninth inspection system,
The moving path of the imaging device provides an inspection system including a figure-eight path in which the imaging device starts moving from an initial position and returns to the initial position.

Further, the present invention provides, as a tenth inspection system, any one of the first to ninth inspection systems,
The projection device comprises a main section and a cover,
the main portion includes one or more light-emitting portions corresponding to the bright portions of the projected image;
each of the light emitting units has a light emitting surface and emits projection light from the light emitting surface;
The cover has one or more transmissive portions and one or more shielding portions corresponding to the light emitting portions, respectively;
The cover is attached to the main portion,
The transmission part transmits the projection light, and the shielding part shields the projection light to provide an inspection system.

Further, the present invention provides an eleventh inspection system, which is any one of the first to ninth inspection systems,
The projection device comprises a main section and a cover,
The main part has a light emitting surface, and emits projection light from the entire light emitting surface,
The cover has one or more transmission parts and one or more shielding parts,
The cover is attached to the main portion,
The transmission part transmits the projection light, and the shielding part shields the projection light to provide an inspection system.

Further, the present invention provides a 10th or 11th inspection system as a 12th inspection system,
each of the transmitting portion and the shielding portion of the cover extends along an oblique direction that intersects a side of the cover;
The transmissive portion and the shielding portion of the cover are arranged alternately with each other in an arrangement direction orthogonal to the oblique direction to provide an inspection system.

Further, the present invention provides, as a thirteenth inspection system, a tenth or eleventh inspection system,
The cover has a peripheral transmitting portion in addition to the transmitting portion and the shielding portion,
The peripheral transparent portion surrounds all the transparent portions and the shielding portions,
each of the transmitting portion and the shielding portion extends between the peripheral transmitting portions along an oblique direction that crosses the sides of the cover;
The transmissive portion and the shielding portion of the cover are arranged alternately with each other in an arrangement direction orthogonal to the oblique direction to provide an inspection system.

Further, the present invention provides a 12th or 13th inspection system as a 14th inspection system,
The inspection system, wherein the cover has only one transmissive portion and only two shielding portions.

Further, according to the present invention, as a first program,
A program for causing a computer to function as an apparatus main body in any one of the first to fourteenth inspection systems is provided.

Further, according to the present invention, as a first storage medium,
A storage medium storing files of a first program is provided.

The photographing device of the present invention photographs a planar projection image. Therefore, even if the target surface vibrates in the moving direction, the entire target surface can be photographed.

In general, microscopic defects on the target surface include shading defects, scattering defects, and refractive defects. A light-shielding defect blocks light and looks dark. Scattering defects scatter light from the surroundings and appear to emit light themselves. Refractive defects refract light from the surroundings and distort the reflected image. On the other hand, the projected image of the present invention contains bright and dark areas. When the projection device is moved relative to the target surface, the bright and dark areas move relative to the target surface. By photographing the projected image two or more times during this relative movement, minute defects on the target surface can be accurately detected.

A high-resolution captured image is required to capture fine defects on the target surface. On the other hand, when a high-resolution captured image is used as it is to detect a defect, the processing time for defect detection may become longer than the interval between capturing images. On the other hand, the apparatus main body of the present invention detects defects while reducing the resolution of the captured image, so processing time can be reduced.

As described above, according to the present invention, it is possible to provide a new inspection system capable of detecting minute defects on a target surface even when the target surface is moving.

1 schematically illustrates an inspection system according to an embodiment of the invention; FIG. 2 is a block configuration diagram showing the inspection system of FIG. 1; FIG. 2 is a side view of the optical head of the inspection system of FIG. 1; FIG. A portion of the outline of the support device is drawn in dashed lines. FIG. 4 is a plan view showing a projection device of the optical head of FIG. 3; FIG. 5 is a plan view schematically showing the main part of the projection device of FIG. 4; 2 is a diagram showing a movement path of the optical head of FIG. 1 with respect to a target surface; FIG. The outlines of the optical head, support and support base are drawn with dashed lines. FIG. 5 is a plan view schematically showing a modification of the main part of the projection device of FIG. 4; 2 is a flow chart showing an inspection procedure by the inspection system of FIG. 1; 9 is a flowchart showing inspection processing in the inspection procedure of FIG. 8; FIG. 10 is a diagram showing an example of a display image created by the inspection process of FIG. 9; FIG. FIG. 10 is a diagram showing defect detection processing in the inspection processing of FIG. 9; FIG. 2 schematically shows a first modification of the inspection system of FIG. 1; 14 is a block configuration diagram showing the inspection system of FIG. 13; FIG. Figure 5 shows a modification of the cover of the projection device of Figure 4; Figure 15 shows another modification of the cover of Figure 14; FIG. 15 shows yet another modification of the cover of FIG. 14; FIG. 13 is a diagram illustrating an example of a travel path of the optical head of the inspection system of FIG. 12 relative to the target surface; FIG. 18 is a diagram showing a modification of the moving route of FIG. 17; FIG. 4 is a diagram showing an example of a movement path of an optical head with respect to a three-dimensional target surface; FIG. 13 illustrates a target surface moving relative to the inspection system of FIG. 12; Part of the outline of the robot arm is drawn with a dashed line. FIG. 21 is a diagram showing an example of a movement path of an optical head when inspecting the target surface of FIG. 20; 21 is a diagram showing a timing chart of the inspection system of FIG. 20; FIG. 13 schematically shows an inspection system with three optical heads of the inspection system of FIG. 12; FIG. FIG. 2 schematically shows a second modification of the inspection system of FIG. 1; FIG. 3 is a diagram schematically showing a third modification of the inspection system of FIG. 1; FIG. 4 is a diagram schematically showing a fourth modification of the inspection system of FIG. 1; The positions of the projection device and the imaging device provided inside the examination table are drawn with dashed lines.

Referring to FIG. 1 in conjunction with FIG. 6, inspection system 10 according to embodiments of the present invention is an inspection system for detecting defects 84 in object surface 82 of object 80 . Inspection system 10 projects projection image 50P onto target surface 82 and captures projection image 50P projected onto target surface 82 . Specifically, the inspection system 10 captures the region of the target surface 82 onto which the projection image 50P is projected as a captured image 60P together with the projection image 50P. The inspection system 10 detects defects 84 such as scratches and dents formed on the target surface 82 based on the photographed image 60P.

The inspection system 10 and the object 80 of the present embodiment are placed in an inspection space 90 such as an inspection field of a factory and used. An installation surface 92 such as a floor or a ceiling is provided in the inspection space 90 . The installation surface 92 is a stationary surface that does not move relative to the inspection space 90 . Each of the inspection system 10 and the object 80 is placed directly on the installation surface 92 or indirectly on the installation surface 92 via a member such as a desk or a stand.

The object 80 of the present embodiment is located in front of the inspection system 10 in the front-rear direction. The front-rear direction in this embodiment is the X direction. Forward is the +X direction and backward is the -X direction. However, the directions in the present embodiment are for indicating relative positions for explaining the positions on the drawings, and do not specify the positions of members or parts with respect to the ground.

Object surface 82 preferably has a high light reflectance that allows specular reflection of projected projection image 50P. More specifically, the target surface 82 is preferably a glossy, glossy and smooth surface. Preferred target surfaces 82 are, for example, mirror surfaces, painted surfaces of articles, plated surfaces of articles, glass surfaces, and film surfaces. However, the present invention is not limited to this, and the glossiness, luster, and smoothness of the target surface 82 may correspond to the detection accuracy required for detecting the defect 84 .

The material, size and shape of the object 80 are not particularly limited. For example, the object 80 may be an opaque article or transparent glass through which the inside of the article can be seen. Object 80 may be a relatively large item, such as a car body, or a relatively small item, such as a car door mirror cover. If the object 80 is large, multiple inspection systems 10 may be provided to inspect multiple target surfaces 82 of the object 80, respectively.

Referring to FIGS. 1 and 2, an inspection system 10 of this embodiment includes a processing device 20, a projection control device 30, and an optical head 40. As shown in FIG. The optical head 40 has a projection device 50 and an imaging device 60 . Specifically, the inspection system 10 includes a projection device 50 and an imaging device 60 . Each of the projection device 50 and the imaging device 60 of this embodiment is part of the optical head 40 . However, the present invention is not limited to this. For example, the imaging device 60 may be part of the optical head 40 while the projection device 50 may be a separate device from the optical head 40 . The devices described above are communicatively connected to each other by wired communication means such as communication cables. However, the present invention is not limited to this. For example, the devices described above may be communicatively connected to each other by wireless communication means.

The projection device 50 is an electronic device for projecting a projection image 50P (see FIG. 6). The photographing device 60 is an electronic device for photographing the projected projection image 50P. The projection control device 30 is an illumination power source that supplies power to the projection device 50 . The processing device 20 of this embodiment directly controls the imaging device 60 via the communication cable and indirectly controls the projection device 50 via the communication cable and the projection control device 30 . However, the present invention is not limited to this. For example, the processing device 20 may directly control the projection device 50 . In this case, the inspection system 10 need only include the processor 20 and the optical head 40 . On the other hand, the inspection system 10 may include other devices in addition to the devices described above.

The projection device 50 of this embodiment will be described below.

Referring to FIG. 3, the projection device 50 of the present embodiment is planar illumination, and projects a planar projection image 50P (see FIG. 6) spread two-dimensionally. The projection device 50 has a main section 51 and a cover 56 . A cover 56 is attached to the main portion 51 . Each of the main portion 51 and the cover 56 of the present embodiment has a rectangular shape on a predetermined plane (YZ plane) perpendicular to the X direction. However, the present invention is not limited to this, and the shapes of the main portion 51 and the cover 56 are not particularly limited as long as the projection device 50 can project the required projection image 50P.

Referring to FIG. 5, the main portion 51 of this embodiment includes a large number of light emitting elements 522 and a translucent diffuser plate 53 . Each of the light emitting elements 522 of this embodiment is an LED (light emitting diode). Each of the light emitting elements 522 emits light with an intensity corresponding to the amount of power supplied. The light emitting elements 522 are densely arranged over the entire main portion 51 on the YZ plane. The diffusion plate 53 is positioned in front of the light emitting elements 522 and diffuses the light emitted by the light emitting elements 522 . The light emitting element 522 of this embodiment emits white light. However, the present invention is not limited to this. For example, light emitting elements 522 may be capable of emitting different colors of light. Also, the light emitting element 522 is not limited to an LED.

Referring to FIG. 3 together with FIG. 5, the main portion 51 of this embodiment has a light emitting surface 54 . The light-emitting surface 54 of this embodiment is the front surface of the diffusion plate 53 . The light diffused by the diffuser plate 53 becomes uniform projection light 54L on the light emitting surface 54 . The projection light 54L is emitted forward from the light emitting surface 54 . That is, the main portion 51 of the present embodiment emits projection light 54L from the entire light emitting surface 54. As shown in FIG. In other words, according to the present embodiment, the entire main portion 51 functions as the light emitting portion 52 that emits the projection light 54L.

Referring to FIG. 4 together with FIG. 3 , the cover 56 of the present embodiment is attached to the light emitting surface 54 of the main portion 51 and covers the entire light emitting surface 54 . The cover 56 has one or more transmitting portions 57 and one or more shielding portions 58 . Each of the transmission portions 57 transmits the projection light 54L forward, and the shielding portion 58 shields the projection light 54L. The cover 56 partially shields the projection light 54L with the shielding portion 58 to create the projection image 50P.

Referring to FIG. 6 in conjunction with FIG. 4, projection image 50P is projected onto target surface 82 from projection device 50 . That is, the projection device 50 can project the planar projection image 50P onto the target surface 82 . The projected image 50P includes one or more bright portions 52P corresponding to the transmissive portions 57 and one or more dark portions 54P corresponding to the shielding portions 58, respectively. Each of the bright portions 52P is created by the projection light 54L (see FIG. 3) transmitted through the corresponding transmissive portion 57. As shown in FIG. Each of the dark portions 54P is a portion where the projection light 54L is hardly projected. Therefore, the brightness of the dark portion 54P is lower than the brightness of the bright portion 52P.

Referring to FIG. 4 , the cover 56 of this embodiment has a plurality of transmission portions 57 and a plurality of shielding portions 58 . The transmissive portions 57 and the shielding portions 58 are alternately arranged in the horizontal direction perpendicular to the X direction. The horizontal direction in this embodiment is the Y direction. Each of the transmissive portion 57 and the shielding portion 58 extends linearly along the vertical direction perpendicular to both the X and Y directions while maintaining a constant size in the Y direction. The vertical direction in this embodiment is the Z direction. All transmissive portions 57 have the same size in the Y direction. All shields 58 have the same size in the Y direction. Also, the size of the transmissive portion 57 in the Y direction is the same as the size of the shielding portion 58 in the Y direction.

The cover 56 of this embodiment has the structure described above. However, the present invention is not limited to this. For example, the arrangement of the transmissive portion 57 and the shielding portion 58 can be variously modified as described later. In the present embodiment and modified examples to be described later, the number of transmitting portions 57 may be one, and the number of shielding portions 58 may be one. The size of the transmissive portion 57 in the Y direction may be different from the size of the shielding portion 58 in the Y direction.

Referring to FIG. 6 in conjunction with FIG. 4, the transmissive portion 57 of this embodiment is transparent and the shielding portion 58 of this embodiment is black. Therefore, the bright portion 52P is white, and the dark portion 54P is gray close to black. However, as long as the brightness of the bright portion 52P is higher than the brightness of the dark portion 54P, the color and brightness of the bright portion 52P and the dark portion 54P are not particularly limited. For example, the color of the bright portion 52P may be bright pink, and the color of the dark portion 54P may be dark red. Further, the structure of the projection device 50 is not limited to the present embodiment, and various modifications are possible as long as such a planar projection image 50P can be projected.

For example, referring to FIG. 7 together with FIGS. 4 and 5 , a projection device 50X according to the modification includes a main portion 51X different from the main portion 51 and a cover 56 that is the same as that of the projection device 50 . Referring to FIG. 7 together with FIG. 6, the main portion 51X includes one or more light emitting portions 52X respectively corresponding to the bright portions 52P of the projected image 50P. Each of the light-emitting portions 52X is part of the main portion 51X and includes a large number of densely arranged light-emitting elements 522 and part of the diffusion plate 53 . On the other hand, the light-emitting element 522 is not provided in a portion of the main portion 51X other than the light-emitting portion 52X. Referring to FIG. 7 together with FIG. 3, each of the light-emitting portions 52X thus formed has a light-emitting surface 54 that is part of the front surface of the diffusion plate 53. As shown in FIG. Each of the light-emitting portions 52X emits projection light 54L from the light-emitting surface 54. As shown in FIG.

Referring to FIG. 4 together with FIG. 7, the cover 56 of the projection device 50X according to the modification has a structure similar to that of the present embodiment. More specifically, the cover 56 has one or more transmissive portions 57 and one or more shielding portions 58 respectively corresponding to the light emitting portions 52X. The cover 56 is attached to the main portion 51X. Referring to FIG. 4 in conjunction with FIG. 3, the transmitting portion 57 transmits the projection light 54L, and the shielding portion 58 blocks the projection light 54L. Referring to FIG. 6 together with FIG. 4 , also according to this modification, by providing the cover 56 having the shielding portion 58, the projection image 50P having the clear bright portion 52P and the dark portion 54P can be projected onto the target surface 82.

Referring to FIG. 7 together with FIG. 6, a projection device 50X according to the present modification can project a projection image 50P similar to that of the present embodiment onto the target surface 82. FIG. For example, the light-emitting portion 52X of the main portion 51X can be formed by using a required number of bar lights, which are elongated lighting devices. According to this modification, the manufacturing cost can be reduced by reducing the number of light emitting elements 522 of the projection device 50X. In addition, it is possible to reduce power consumption when the light emitting element 522 emits light. Referring to FIG. 5 in conjunction with FIG. 4, the projection device 50 of this embodiment can be further modified. For example, the projection device 50 may control so that only the light emitting elements 522 corresponding to the transmissive portions 57 of the cover 56 emit light.

The imaging device 60 (see FIG. 3) of this embodiment will be described below.

Referring to FIG. 3 together with FIG. 6, the imaging device 60 of the present embodiment is a CCD (charge-coupled device) camera, and includes a lens (not shown) and a large number of light receiving elements arranged two-dimensionally. (not shown). A photographed image 60P photographed by the photographing device 60 is a planar digital image composed of a large number of pixels arranged two-dimensionally. In other words, the imaging device 60 is an area camera. The structure of the photographing device 60 is not particularly limited as long as the photographing device 60 can photograph a two-dimensionally spread planar photographed image 60P.

The pixels of the captured image 60P according to the present embodiment take gray scale values from 0 (black) to 255 (white). In other words, the image capturing device 60 captures a 256-gradation grayscale image. However, the present invention is not limited to this, and the photographing device 60 may photograph a full-color photographed image 60P. However, the inspection system 10 detects the defect 84 by referring to the brightness of the captured image 60P, as will be described later. Therefore, from the viewpoint of detecting the defect 84 in a short time, a grayscale image whose brightness can be easily referred to is preferable.

The optical head 40 of this embodiment will be described below.

Referring to FIG. 1, the optical head 40 of this embodiment is fixed to a support device 72 separate from the optical head 40 . The support device 72 is supported by a support base 78 such as a rail, and is movable only in the Y direction. That is, the optical head 40 of this embodiment is movable only in the Y direction. However, the present invention is not limited to this. For example, the support device 72 may be a robot arm that allows the optical head 40 to move three-dimensionally, as will be described later.

Referring to FIG. 3, the optical head 40 of this embodiment includes a base member 42, a supported portion 44, and a column 46 in addition to the projection device 50 and the imaging device 60. As shown in FIG. The base member 42 of this embodiment is a single metal plate having a bend. The base member 42 has rigidity and is difficult to bend. The projection device 50 and the imaging device 60 are fixed to the base member 42 . The supported portion 44 is part of the rear surface of the base member 42 . The strut 46 is a cylindrical member. The strut 46 is fixed to the supported portion 44 and extends rearward from the supported portion 44 . A rear end of the strut 46 is fixed to a support portion 76 of the support device 72 .

By providing the post 46 as described above, the base member 42 to which the projection device 50 and the imaging device 60 are fixed is moved away from the support device 72, thereby preventing the projection device 50 and the imaging device 60 from hitting the support device 72. can be prevented. In particular, if the support device 72 is a robotic arm, the struts 46 prevent damage to the projection device 50 and imaging device 60 fixed to the base member 42 without restricting the movement of the robotic arm. However, the present invention is not limited to this. For example, the supported portion 44 may be fixed directly to the support portion 76 without the strut 46 interposed therebetween. That is, the struts 46 may be provided as required.

A projection device 50 fixed to the base member 42 projects a projection image 50P along the projection direction. The illustrated projection device 50 projects a projection image 50P onto the front object surface 82 along the X direction. The imaging device 60 captures the projected image 50P projected onto the target surface 82 along the optical axis 62 of the lens. The projection device 50 and the imaging device 60 are arranged so as to satisfy imaging conditions such that the imaging device 60 can clearly capture the projection image 50P projected onto the target surface 82, as described below.

As one of the photographing conditions, the projection device 50 and the photographing device 60 are arranged so as to assume a predetermined posture with respect to an intermediate point 86 located in the intermediate portion of the projected image 50P. More specifically, when the imaging device 60 captures the projection image 50P, the projection direction of the projection device 50 and the optical axis 62 of the imaging device 60 intersect at a predetermined angle across the normal 88 at the midpoint 86. placed in The predetermined angle in this embodiment is 40 degrees. Specifically, the intersection angle θ between the projection direction of the projection device 50 and the normal 88 is 20 degrees, and the intersection angle θ between the optical axis 62 and the normal 88 is 20 degrees. However, as long as the imaging device 60 can capture the projection image 50P, the intersection angle θ is not particularly limited. That is, the imaging device 60 may be arranged so as to be able to capture the projected image 50P projected onto the target surface 82 .

Another photographing condition is that the front surface of the projection device 50 is positioned closer to the midpoint 86 than the lens of the photographing device 60 . According to this arrangement, the reflected image of the projection image 50P incident on the lens can be enlarged, thereby widening the shooting range of one shot. The distance DL between the illustrated midpoint 86 and the front surface of the projection device 50 is 30 cm. The distance DC between the illustrated midpoint 86 and the lens of the imager 60 is 36 cm. By setting this distance, for example, a range of 150×200 mm can be photographed. However, the distance DL and the distance DC may be appropriately set according to the required shooting range. For example, distance DL>distance DC may be satisfied. That is, the front surface of the projection device 50 may be the same distance or more away from the midpoint 86 than the lens of the imaging device 60 .

As described above, the projection device 50 and the imaging device 60 of this embodiment are fixed so as not to move with respect to the base member 42 . Therefore, even if the optical head 40 moves, the projection direction of the projection device 50 relative to the base member 42 does not change, and the orientation of the optical axis 62 of the lens of the photographing device 60 relative to the base member 42 does not change. In addition, according to this embodiment, the focal length of the lens of the photographing device 60 is also kept constant. In other words, the optical head 40 of this embodiment is not provided with a movable portion that is susceptible to vibration. Therefore, even if the optical head 40 vibrates as it moves, the photographing conditions described above are hardly affected. However, the present invention is not limited to this. For example, the focal length of the lens may be adjustable during movement of the optical head 40 under control of the processor 20 (see FIG. 1).

As will be described later, according to the present embodiment, the defect 84 (see FIG. 6) can be detected even if the captured image 60P is slightly out of focus. More specifically, the depth of field of the lens of the photographing device 60 is about ±4 cm from the predetermined distance, and a certain amount of error is allowed. Therefore, even if the distance DC between the intermediate point 86 and the lens slightly changes with the movement of the optical head 40, the photographing device 60 can photograph the photographed image 60P in which the defect 84 can be detected.

Each of the projection device 50 and the imaging device 60 of this embodiment is directly fixed to the single base member 42 . In other words, each of the projection device 50 and the imaging device 60 is directly fixed to the supported portion 44 of the base member 42 . However, the present invention is not limited to this, and each of the projection device 50 and the imaging device 60 may be directly or indirectly fixed to the supported portion 44 . For example, the projection device 50 may be directly fixed to the supported portion 44 , and the imaging device 60 may be fixed to the projection device 50 . That is, the projection device 50 and the imaging device 60 only need to be fixed to each other.

The processing apparatus 20 (see FIG. 1) of this embodiment will be described below.

Referring to FIGS. 1 and 2, the processing device 20 of this embodiment is a general-purpose PC (Personal Computer), and includes a device body 22, a storage device 24, an input device 26, and a display device 28. I have. Specifically, the inspection system 10 includes an apparatus body 22 , a storage device 24 , an input device 26 and a display device 28 . Each of the storage device 24, the input device 26, and the display device 28 of this embodiment is part of one processing device 20, or is connected to a common device body 22 so as to be able to communicate with each other. However, the present invention is not limited to this. For example, display device 28 may be communicatively connected only to a monitoring device (not shown) separate from processing device 20 . In this case, the monitoring device may be communicably connected to the device main body 22 by wired communication means or wireless communication means. Also, the monitoring device including the display device 28 may be arranged at a location different from the device main body 22 .

The device main body 22 is a PC main body, and includes a CPU (Central Processing Unit: not shown) and a main storage device (not shown). The storage device 24 is, for example, a magnetic disk device, and can store various files (not shown) including program executable files. The storage device 24 acquires and stores files according to instructions from the device body 22 . The CPU of the device body 22 acquires the executable file stored in the storage device 24, loads it into the main storage device, and executes the instructions in the executable file to implement various functions. The input device 26 is, for example, a keyboard and a mouse, and transmits characters input from the keyboard and the position and range indicated by the mouse to the device body 22 . The display device 28 is, for example, a liquid crystal display, and displays characters, images, and the like transmitted from the device body 22 .

The apparatus main body 22 of this embodiment includes a projection control program 201 which is a driver for controlling the projection device 50 , an imaging control program 202 which is a driver for controlling the imaging device 60 , and an inspection processing program 203 . The apparatus body 22 can execute a projection control program 201, an imaging control program 202, and an inspection processing program 203, respectively.

For example, the apparatus main body 22 loads the inspection processing program 203 into the main storage device (not shown) according to the instruction input from the input device 26, and executes the inspection processing by the CPU (not shown). That is, the inspection system 10 has a program (inspection processing program 203 ) for causing the computer to function as the device main body 22 in the inspection system 10 . These programs are installed in the storage device 24 from a storage medium such as a CD-ROM (compact disk read only memory) storing program files.

As described above, it is the CPU (not shown) of the apparatus main body 22 that actually executes these programs. However, in the following description, when it is described that these programs are the subject of execution, or the processing of these programs by the CPU (for example, the inspection processing by the CPU of the inspection processing program 203) is the subject of execution. may be written as

The device main body 22 is communicably connected to each of the projection device 50 and the imaging device 60 . Specifically, the device body 22 is cable-connected to the projection device 50 via the projection control device 30 and is also directly cable-connected to the imaging device 60 . However, the present invention is not limited to this. For example, the projection control device 30 may be a control board built into the device body 22 . In this case, the device body 22 may be directly cable-connected to each of the projection device 50 and the imaging device 60, or may be wirelessly connected.

A method of photographing the target surface 82 (see FIG. 6) according to this embodiment will be described below. The projection device 50 and the imaging device 60 of the present embodiment are integrated as an optical head 40 and move as the optical head 40 moves. A projection device 50 and an imaging device 60 of a first modified example of the inspection system 10 to be described later are also integrated as an optical head 40 and move as the optical head 40 moves. Therefore, in the description of the present embodiment and the description of the first modified example of the inspection system 10, the descriptions of "movement of the optical head 40", "movement of the projection device 50", and "movement of the imaging device 60" are mutually exclusive. can be read.

Referring to FIGS. 1 and 6, the imaging device 60 of the optical head 40 moves relative to the target surface 82 to capture a captured image 60P including the projected image 50P. The photographing device 60 photographs the entire target surface 82 by photographing the photographed images 60P at a plurality of photographing points 484 . The inspection process of the apparatus main body 22 inspects the entire target surface 82 by detecting defects 84 from each of the captured images 60P.

The photographing device 60 of the present embodiment moves along the movement path 48 at a constant speed in the Y direction with respect to the object 80 that is fixed so as not to move with respect to the installation surface 92 . A projected projection image 50P is captured. The photographing device 60 photographs the photographed image 60P without stopping at the photographing point 484 .

According to this embodiment, it is possible to reduce the time required to photograph the entire target surface 82 by photographing without stopping while maintaining the movement of the optical head 40 at a constant speed. That is, the entire target surface 82 can be inspected in a shorter time than when the optical head 40 stops at the imaging point 484 . However, the present invention is not limited to this embodiment. For example, the target surface 82 may move relative to the optical head 40 when the imaging device 60 captures the projection image 50P projected onto the target surface 82 . More specifically, the optical head 40 may be fixed so as not to move with respect to the installation surface 92 . On the other hand, the object 80 may be moving at a constant speed with respect to the installation surface 92 . Also, the relative speed of movement between object surface 82 and optical head 40 need not be constant.

According to this embodiment, when the optical head 40 is fixed so as not to move with respect to the installation surface 92, each of the projection device 50 and the photographing device 60 is fixed so as not to move with respect to the installation surface 92. It is However, the present invention is not limited to this. For example, as described above, if the imaging device 60 is a part of the optical head 40 and the projection device 50 is a separate device from the optical head 40 , the imaging device 60 is prevented from moving with respect to the installation surface 92 . It may be fixed and the projection device 50 may be movably supported with respect to the installation surface 92 .

The object 80 shown in FIG. 6 has an object surface 82 that is substantially free of irregularities. Also, the size in the Z direction of the target surface 82 shown in FIG. 6 is smaller than the size in the Z direction of the photographed image 60P. Therefore, by photographing the target surface 82 while moving the optical head 40 straight along the Y direction, the entire target surface 82 can be photographed. However, the present invention is not limited to this. For example, the target surface 82 may be uneven. Also, the size of the target surface 82 in the Z direction may be larger than the size of the captured image 60P in the Z direction. In this case, for example, by changing the position of the optical head 40 in the Z direction and moving the optical head 40 along the Y direction two or more times, the entire target surface 82 can be photographed.

The area of the captured image 60P shown in FIG. 6 completely matches the area of the projected image 50P, which is most preferable from the viewpoint of reducing the number of images taken for inspecting the target surface 82. FIG. However, it is practically impossible to arrange the imaging device 60 in this manner at all imaging points 484 . In reality, it is preferable that the area of the captured image 60P is completely included in the area of the projected image 50P. In other words, it is preferable that the projection image 50P appears in all the pixels of the captured image 60P. However, the present invention is not limited to this, and it is sufficient that the area of the captured image 60P partially overlaps the area of the projected image 50P.

The procedure for inspecting the target surface 82 according to this embodiment will be described below.

Referring to FIGS. 6 and 8, first, the optical head 40 and the object 80 are arranged so that the inspection start position of the object surface 82 can be photographed (S810 in FIG. 8). The inspection start position of the target surface 82 shown in FIG. 6 is the left end of the target surface 82 in the Y direction.

Referring to FIG. 2, when the optical head 40 is arranged as described above, the apparatus main body 22 performs projection setting of the projection control apparatus 30 according to the projection control program 201 ("1-1" in FIG. 2). For example, the device body 22 sets the emission time and brightness when the projection device 50 projects the projection image 50P. Further, the device main body 22 performs shooting settings for the shooting device 60 by the shooting control program 202 (“1-2” in FIG. 2). For example, the device body 22 sets the brightness, exposure time, and shooting cycle (frame rate) when the shooting device 60 shoots. The frame rate of this embodiment is, for example, 40 frames/second.

6 and 8, movement of at least one of the optical head 40 and the target surface 82 is then started (S820 in FIG. 8). According to this embodiment, the optical head 40 is moved from the position where the left end (inspection start position) of the target surface 82 in the Y direction shown in FIG. move up to Referring to FIG. 2, when the optical head 40 starts to move, the device body 22 instructs the imaging device 60 to start imaging by means of the imaging control program 202 (“2” in FIG. 2).

Referring to FIGS. 6 and 8, next, the photographing device 60 photographs a photographed image 60P with the set brightness and exposure time according to the set photographing cycle (S830 in FIG. 8).

Specifically, referring to FIGS. 2 and 6, the photographing device 60 instructs the projection control device 30 to perform projection at regular time intervals based on the photographing cycle (“3-1” in FIG. 2). The projection control device 30 that has received the projection instruction supplies power corresponding to the set brightness to the projection device 50 only during the set light emission time (“3-2” in FIG. 2). The projection device 50 causes the light emitting element 522 (see FIG. 5) to emit light with the supplied power. As a result, the projection image 50P is projected onto the target surface 82 (“3-3” in FIG. 2). The photographing device 60 photographs the photographed image 60P in accordance with the projection timing of the projected image 50P (“3-4” in FIG. 2). The photographing device 60 transmits the photographed image 60P to the device body 22 (“3-5” in FIG. 2).

The imaging device 60 of the present embodiment performs non-stop imaging during movement. The exposure time in the present embodiment is set as short as 100 μs, for example, so that the captured image 60P that is difficult to be out of focus can be captured while moving. On the other hand, bright illumination is required to obtain sufficient brightness with a short exposure time. Therefore, the projection device 50 of the present embodiment projects the projection image 50P with high brightness by the supplied large power only when photographing. That is, the photographing device 60 of the present embodiment performs strobe photography while moving without stopping. However, the present invention is not limited to this. For example, the projection device 50 may always project the projection image 50P. In this case, the projection device 50 may make the projected image 50P when photographing is brighter than the projected image 50P when not photographing.

Referring to FIGS. 2 and 8, next, the device main body 22 performs inspection processing by the device main body 22 (S840 in FIG. 8). 2 and 9, in the inspection process, the device body 22 receives the captured image 60P from the imaging device 60 (S910 in FIG. 9). Next, the device main body 22 performs defect detection processing 204 (see FIG. 11), which will be described later, and detects defects 84 (see FIG. 6) while reducing the resolution of the received captured image 60P (S920 in FIG. 9). Next, the apparatus main body 22 creates a display image 219 (see FIG. 10) by superimposing the detected defect 84 on the captured image 60P (S930 in FIG. 9). Next, the device body 22 displays the display image 219 on the display device 28 (S940 in FIG. 9). That is, the display device 28 displays the defect 84 detected by the device body 22 .

Referring to FIGS. 2 and 8, next, the apparatus body 22 determines whether or not the inspection end position of the target surface 82 has been photographed (S850 in FIG. 8). The inspection end position shown in FIG. 6 is the right end of the target surface 82 in the Y direction. If the inspection end position of the target surface 82 has been photographed (YES in S850 of FIG. 8), the inspection ends. On the other hand, if the inspection end position of the target surface 82 has not been photographed (NO in S850 of FIG. 8), photographing is performed again.

Referring to FIG. 2 in conjunction with FIG. 6, for example, optical head 40 may be provided with a sensor (not shown) such as an infrared sensor. The sensor may detect whether the target surface 82 is positioned forward and notify the device body 22 of the detection result. In this case, when it is first notified that the target surface 82 is located in front, the apparatus main body 22 determines that the imaging apparatus 60 is arranged so that the inspection start position can be photographed, and the projection setting (Fig. 2 "1-1") and shooting settings ("1-2" in FIG. 2), and instructs the shooting device 60 to start shooting ("2" in FIG. 2). Further, when notified that the target surface 82 is not positioned forward, the device body 22 determines that the inspection end position of the target surface 82 has been photographed, and instructs the photographing device 60 to finish photographing. (“4” in FIG. 2). The photographing device 60 ends the photographing when the photographing end is instructed.

To summarize the above description with reference to FIGS. 2 and 6, when the target surface 82 is moved, vibrations in the movement direction of the target surface 82 cannot be completely prevented. Therefore, in such a case, the line sensor camera as in Patent Document 1 cannot continuously photograph the target surface 82 . On the other hand, the photographing device 60 of the present embodiment photographs a planar projection image 50P. Therefore, in the case where the imaging device 60 is fixed and the target surface 82 is moved in the moving direction (+Y direction in FIG. 6) for imaging, the entire target surface 82 is imaged even if the target surface 82 vibrates in the moving direction. can.

Defects 84 on the target surface 82 include minute defects 84 called bumps, repellency, dents, and the like. The diameter of the fine defect 84 is, for example, 0.1 mm. These fine defects 84 can be classified into shading defects 84 , scattering defects 84 and refractive defects 84 . A light-shielding defect 84 blocks light and looks dark. Scattering defects 84 scatter light from the surroundings and appear to emit light themselves. Refractive defects 84 refract light from the surroundings and distort the reflected image. It is extremely difficult to detect a minute defect 84 by a line sensor camera only by illuminating it.

On the other hand, projected image 50P of the present embodiment includes bright portion 52P and dark portion 54P. Also, the projection device 50 moves relatively to the target surface 82 while the imaging device 60 captures the captured image 60P. Therefore, the relative position of the projected image 50P on the target surface 82 differs each time the imaging device 60 captures the captured image 60P. That is, when the projection device 50 is moved relative to the target surface 82 , the bright portion 52</b>P and the dark portion 54</b>P move relative to the target surface 82 . By setting the imaging cycle of the imaging device 60 to a sufficiently short time interval according to the relative moving speed of the projection device 50 with respect to the target surface 82, all parts of the target surface 82 (that is, all defects 84), both a captured image 60P when the bright portion 52P is projected and a captured image 60P when the dark portion 54P is projected can be captured.

A light blocking defect 84 can be detected by the bright portion 52P. Scattering defects 84 can be detected by dark areas 54P. The refractive defect 84 can be detected by photographing the bright portion 52P and the dark portion 54P in a curved manner. That is, according to the present embodiment, when the optical head 40 is moved relative to each other, the projected image 50P including the bright portion 52P and the dark portion 54P is photographed two or more times, so that a minute defect on the target surface 82 can be detected. 84 can be detected accurately.

As can be understood from the above description, each of the bright portion 52P and the dark portion 54P can be projected one or more times onto all portions of the target surface 82 as the optical head 40 moves relative to the target surface 82. As long as the transmission part 57 of the projection device 50 and the arrangement of the transmission part 57 are not particularly limited. However, from the viewpoint of accurately detecting the fine defect 84, it is preferable that the transmissive portion 57 and the transmissive portion 57 have a slit pattern as in the present embodiment.

In order to photograph the fine defect 84 of the target surface 82, a high-resolution photographed image 60P is required. The pixel size of the captured image 60P of this embodiment is 2048×1500 pixels. This large pixel size allows an area of 150×200 mm to be imaged with a resolution of 100 μm/pixel. Generally, the minimum diameter at which minute defects 84 such as bumps and dents can be recognized as defects is 0.3 to 0.5 mm. According to this embodiment, a sufficient resolution is obtained to detect generally recognizable minute defects 84 .

However, when the defect 84 is detected by using the high-resolution captured image 60P as it is, there is a possibility that the processing time for defect detection becomes longer than the imaging interval (imaging cycle). On the other hand, the device body 22 of the present embodiment detects the defect 84 while reducing the resolution of the captured image 60P. More specifically, the device body 22 detects the defect 84 in the process of lowering the resolution of the captured image 60P and creating the output image 218 (see FIG. 10). This processing method can reduce the processing time.

As described above, according to the present embodiment, it is possible to provide a new inspection system 10 capable of detecting minute defects 84 on the target surface 82 even when the target surface 82 is moving.

According to this embodiment, the imaging by the imaging device 60 and the inspection processing by the device main body 22 are executed in parallel with each other. The photographed image 60P is queued in a buffer area of the main storage device (not shown) of the device body 22 . In the inspection process of the device main body 22, the photographed images 60P are sequentially extracted from the buffer and processed. According to this processing method, even if the inspection processing takes a long time, the defect 84 can be displayed on the display device 28 apparently in real time by sufficiently increasing the size of the buffer.

According to this embodiment, the display image 219 (see FIG. 10) including the defect 84 is displayed on the display device 28 of the processing device 20 provided with the device main body 22 . However, the present invention is not limited to this. For example, the display image 219 may be transmitted to and displayed on the display device 28 of a monitoring device (not shown) arranged at a location different from the device body 22 .

As described above, the apparatus main body 22 of the present embodiment detects the defect 84 by performing inspection processing on the received photographed image 60P using the inspection processing program 203 . Referring to FIG. 11, the inspection processing of this embodiment includes defect detection processing 204 using a neural network. In other words, the inspection process program 203 uses the defect detection process 204 as part of the inspection process. The defect detection processing 204 of this embodiment will be described below.

Defect detection processing 204 of this embodiment includes encoding processing 206 , post-processing 207 , and defect identification processing 208 . The encoding process 206 is similar to the encoding process of U-Net, which is a general model for semantic segmentation. On the other hand, defect detection processing 204 includes post-processing 207 and defect identification processing 208 instead of U-Net decoding processing.

The captured image 60P of the present embodiment consists of a one-channel grayscale image. The encoding process 206 processes this captured image 60P as an input image 210. FIG. More specifically, the encoding process 206 performs two convolutions on the input image 210 using a 3×3 kernel to create a 16-channel intermediate layer 212 (convolution layer) while maintaining the resolution of the input image 210. ). The encoding process 206 then uses a 2x2 kernel for the 16-channel hidden layer 212 to create a max pooling layer of 1024x750 pixels while maintaining the number of channels. The activation function of the convolution layer mentioned above is the ReLU layer (ramp function). Also, a batch normalization layer (not shown) is added between the convolution layer and the ReLU layer (not shown).

The encoding process 206 performs the same process three times in total. As a result, the pixel size (that is, resolution) of the captured image 60P gradually decreases, while the number of channels whose features are extracted gradually increases. The encoding process 206 results in an intermediate layer 212 of 2048×1500 pixels and 16 channels (first intermediate layer 212), an intermediate layer 212 of 1024×750 pixels and 32 channels (second intermediate layer 212), and 512×375 pixels. And an intermediate layer 212 (third intermediate layer 212) of 64 channels and a first output layer 214 of 256×187 pixels and 64 channels are obtained.

Post-processing 207 uses the first output layer 214 of encoding process 206 as an input layer. Post-processing 207 performs two convolutions using a 3×3 kernel on the first output layer 214 to create an intermediate layer 215 (convolution layer) of 128 channels while maintaining the resolution of the first output layer 214. create. The activation function of the convolution layer mentioned above is the ReLU layer. Also, a batch normalization layer (not shown) is added between the convolution layer and the ReLU layer (not shown).

Post-processing 207 then deconvolves the 128-channel intermediate layer 215 with a 3×3 kernel to create a resolution-preserving 64-channel transposed convolution layer. Post-processing 207 then copies&crops the third hidden layer 212 into the transposed convolution layer. Specifically, post-processing 207 uses a 2×2 kernel for the third hidden layer 212 to create a max pooling layer of 256×187 pixels while maintaining the number of channels and overlays the transposed convolution layer. The activation function of the transposed convolution layer mentioned above is the ReLU layer. Also, a batch normalization layer (not shown) is added between the transposed convolution layer and the ReLU layer (not shown).

Post-processing 207 performs the same processing three times in total. As a result, the pixel size (ie, resolution) of the first output layer 214 is maintained while the number of feature-extracted channels converges over time. Post-processing 207 results in a second output layer 216 of 256×187 pixels and 16×2 channels.

The defect identification process 208 uses the second output layer 216 of post-processing 207 as an input layer. The defect identification process 208 performs two convolutions on the second output layer 216 using a 3×3 kernel to create a 16-channel convolution layer while maintaining the resolution of the second output layer 216 . The defect identification process 208 then convolves the 16-channel convolution layer using a 1×1 kernel to produce a 2-channel output image 218 while maintaining pixel size. The activation function of the convolution layer mentioned above is the ReLU layer. Also, a batch normalization layer (not shown) is added between the convolution layer and the ReLU layer (not shown).

One of the two channels of the output image 218 of this embodiment is a channel for detecting minute defects 84 (see FIG. 6) such as bumps and dents. Another of the two channels of the output image 218 of this embodiment is a channel for detecting elongated defects 84 such as line scratches. However, the invention is not so limited and the output image 218 may have only one of the two channels.

The defect detection process 204 of this embodiment is a learned model after machine learning using teacher data. The training data includes, for example, an image (original image) of the actual defect 84 (see FIG. 10) and a correct image created from the original image by the developer. The original image is, for example, the input image 210 shown in FIG. The correct image is, for example, the output image 218 shown in FIG. According to the present embodiment, the machine learning of the defect detection process 204 is performed about 800 times using about 3000 sets of original images and result images, thereby optimizing parameters such as kernel weights and biases. A trained defect detection process 204 with a defect detection accuracy of 90% or greater is obtained.

Referring to FIG. 6, imager 60 and target surface 82 are moving relative to each other during inspection. Therefore, the same portion of the target surface 82 can be imaged only once. In addition, defect 84 on captured image 60P is often an area without a clear boundary. In such a case, by using AI (artificial intelligence), the defect detection processing 204 (see FIG. 11) with high defect detection accuracy can be developed relatively easily. Also, when the inspection system 10 is used in the actual inspection space 90, the defect detection accuracy can be further improved. Furthermore, even if the photographed image 60P is somewhat out of focus, the defect 84 can be detected. However, the present invention is not limited to this. For example, a non-learning program that performs the same processing as the defect detection processing 204 may be created based on optimized parameters obtained by machine learning of the defect detection processing 204 . That is, the defect detection processing 204 does not have to be AI.

Referring to FIG. 11, the defect detection process 204 of this embodiment has an encoding process 206 similar to that of U-Net. However, the present invention is not limited to this. Even when using AI as the defect detection process 204, the defect detection process 204 may have encoding processes 206 similar to other semantic segmentation models. For example, the defect detection process 204 may have an encoding process 206 similar to that of SegNet or DeeplabV3+.

As described above, the post-processing 207 of the present embodiment copies and crops the intermediate layer 212 obtained during the encoding process 206 to generate the second output layer 216 in the same manner as the U-Net decoding process. U-Net copy&crop is performed to compensate for boundary information lost due to resolution reduction in the decoding process for increasing resolution. On the other hand, the post-processing 207 of the present embodiment does not increase the resolution, and the copy&crop of the post-processing 207 cannot supplement boundary information. According to the copy & crop of this embodiment, during machine learning, by creating a loophole from the back layer to the front layer of the neural network, a large error can be propagated to the front layer, thereby improving the learning ability. can prevent a decrease in Also, by making the formal structure of the post-processing 207 similar to the formal structure of the decoding process of U-Net, it is possible to prevent deterioration in analysis performance compared to U-Net.

The target image size in a general segmentation model is at most about 256×256 pixels. The pixel size of the input image 210 in this embodiment is 2048×1500 pixels, which is extremely large. On the other hand, the encoding process 206 of the present embodiment convolves the captured image 60P to reduce the resolution while extracting features to generate the first output layer 214 . Post-processing 207 produces a second output layer 216 with a converged number of channels while maintaining the low resolution of the first output layer 214 . That is, according to the present embodiment, the pixel size of the input image 210 is reduced by extracting the features while reducing the pixel size of the input image 210, and narrowing down the features while maintaining the pixel size of the layer from which the features are extracted. It is possible to prevent a decrease in processing capacity due to a large .

The number of types of defects 84 to be detected in this embodiment is at most two. Therefore, the number of channels required to detect the defect 84 in the convolution layer such as the intermediate layer 212 is less than in general segmentation. According to the present embodiment, the number of channels in the convolution layer can be reduced to about 1/13 of that in U-Net, and a decrease in processing capability due to an increase in pixel size can be prevented. Further, according to the present embodiment, the defect 84 can be detected from one input image 210 without performing processing such as extracting differences between two or more images. As a result, defects 84 can be displayed in real time.

A trial to detect the defect 84 using U-Net from the input image 210 of 2048×1500 pixels takes about 380 milliseconds. This time is extremely long compared to the imaging cycle (25 ms) of this embodiment, and cannot be used for the defect detection processing 204 of this embodiment. On the other hand, if the defect detection processing 204 of this embodiment is used, the defect 84 can be detected in about 30 milliseconds. According to this embodiment, real-time inspection becomes possible.

Referring to FIG. 11 together with FIG. 10, the defect identification processing 208 of this embodiment identifies the area of the defect 84 in the captured image 60P based on the second output layer 216. FIG. More specifically, defect identification process 208 creates output image 218 based on second output layer 216 . An area corresponding to the defect 84 is identified in the output image 218 . The inspection processing program 203 creates a display image 219 by superimposing the output image 218 on the captured image 60P. The display image 219 may be a grayscale image, or may be a color image obtained by converting a grayscale image.

As the resolution of the captured image 60P is lowered, the resolution of the area corresponding to the defect 84 is also lowered. More specifically, a resolution of 100 μm/pixel in input image 210 is reduced to 800 μm/pixel in output image 218 . However, this degree of resolution is sufficient for an inspector to locate defects 84 on the target surface 82 . Also, as a result of the reduced resolution, the size of the area corresponding to the defect 84 may be slightly larger than the actual size of the defect 84 in the output image 218 and the display image 219 . In this case, the defect 84 can be displayed more easily while maintaining high defect detection accuracy.

The inspection processing described above can be modified in various ways. For example, the layer structure of the defect detection process 204, the pixel size of each layer, and the number of channels are not limited to those of this embodiment, and may be appropriately set according to the defect 84 to be detected. The inspection process may also display on the display device 28 the position of the region of the defect 84 displayed in the display image 219 relative to the entire target surface 82 .

Referring to FIG. 1, the inspection system 10 of the present embodiment can be modified in various ways in addition to the various modifications already described.

Comparing FIG. 12 with FIG. 1, the inspection system 10A according to the first modification of the inspection system 10 has the same processing device 20, projection control device 30 and optical head 40 as the inspection system 10, plus a support device control device 70A and , and a support device 72A. A support device 72A of this modified example is a robot arm, and supports the optical head 40 so as to be movable. The support device controller 70A is a robot arm controller. The movement path 48 (see FIGS. 17 to 19) of the optical head 40 is preset in the support device control device 70A by teaching. The support device controller 70A controls the support device 72A so that the optical head 40 moves along the movement path 48. FIG.

72 A of support apparatuses of this modification are six-axis robot arms. However, the present invention is not limited to this. The support device 72A can be any robotic arm or non-robotic arm as long as it can move the optical head 40 along the required movement path 48 (see FIGS. 17-19). good too. The support device control device 70A may be provided as required.

Referring to FIG. 12, the support device 72A has a fixed portion 74A and a support portion 76A. 74 A of fixed parts of this modification are being fixed on the stand put on the installation surface 92. As shown in FIG. That is, the fixed portion 74A is fixed so as not to move with respect to the installation surface 92 . The support portion 76A is relatively movable with respect to the fixed portion 74A. Referring to FIG. 3, the supported portion 44 of the optical head 40 is fixed to one end of the support portion 76A via a support 46. As shown in FIG. That is, the imaging device 60 of the optical head 40 is supported by the support portion 76A and is thereby movable along the target surface 82. FIG. Specifically, the supported portion 44 is supported by the support portion 76A, so that the optical head 40 can capture the projected image 50P by the imaging device 60 at the distance DC between the imaging device 60 and the target surface 82. It can move along the target surface 82 while maintaining a sufficient range.

More specifically, the optical head 40 of this modified example is movable along each of the three directions of the X direction, the Y direction and the Z direction, and can move along a direction obtained by combining two or more of the three directions. can be moved. In addition, the optical head 40 is rotatable about axes extending parallel to each of the three directions. As described above, the optical head 40 of this modification can move in various ways. However, the present invention is not limited to this, as long as the optical head 40 can move and rotate in the directions necessary to photograph the entire target surface 82 .

Referring to FIG. 17, the optical head 40 of this modified example may move along both the Y direction and the Z direction. In such a case, it is preferable that each of the bright portion 52P and the dark portion 54P projected onto the target surface 82 is oblique to each of the Y direction and Z direction. In other words, each of the bright portion 52P and the dark portion 54P preferably extends obliquely with respect to the moving direction of the optical head 40 .

Referring to FIG. 14 together with FIG. 17, the cover 56F according to the first modified example of the optical head 40 is formed so as to be able to project the obliquely extending bright portion 52P and dark portion 54P. More specifically, each of the transmitting portion 57 and the shielding portion 58 of the cover 56F extends along the diagonal direction that crosses the side 562 of the cover 56F. The transmitting portions 57 and the shielding portions 58 of the cover 56F are alternately arranged in the arrangement direction orthogonal to the oblique direction. The oblique direction of the first modification is preferably oblique with respect to the side 562 in the range of 20 to 70 degrees or in the range of -20 to -70 degrees, and is preferably about ±45 degrees. More preferred.

The side 562 of the first modification is one of the four sides of the rectangle. Each of the transmissive portion 57 and the shielding portion 58 extends over the entire cover 56F. The sizes (widths) in the arrangement direction of the transmitting portion 57 and the shielding portion 58 are the same. However, the present invention is not limited to this. For example, side 562 may be one of the sides of a polygon. Each end of the transmitting portion 57 and the shielding portion 58 may be separated from the side of the cover 56F. The ends remote from the sides of the cover 56F may have a rounded shape or a polygonal shape. The width of each of the transmissive portion 57 and the shielding portion 58 can be changed as required.

Referring to FIG. 15 together with FIG. 17, the cover 56S according to the second modification of the optical head 40 is a further modification of the cover 56F (see FIG. 14) according to the first modification described above. The cover 56S has a peripheral transmitting portion 59 in addition to the transmitting portion 57 and the shielding portion 58 similar to those of the cover 56F. The peripheral transmitting portion 59 extends along the four sides of the cover 56S in the YZ plane and surrounds all the transmitting portions 57 and shielding portions 58 . Each of the transmissive portion 57 and the shielding portion 58 extends between the peripheral transmissive portions 59 along an oblique direction that intersects the side 562 of the cover 56S. The transmitting portions 57 and the shielding portions 58 of the cover 56S are alternately arranged in the arrangement direction orthogonal to the oblique direction. The oblique direction of the second modification is preferably oblique with respect to the side 562 in the range of 20 to 70 degrees or in the range of -20 to -70 degrees, and is preferably about ±45 degrees. More preferred.

When the target surface 82 is photographed by the photographing device 60, in a normal case, the surrounding area where the projected image 50P is not projected is also photographed in the photographed image 60P. The defect detection processing 204 (see FIG. 11) may detect a defect 84 that does not actually exist in the peripheral area of the captured image 60P. According to the cover 56S of the second modified example, by masking the output image 218 (see FIG. 11) of the defect detection processing 204 with the peripheral transparent portion 59, the erroneously detected defect 84 can be removed. The defect detection accuracy of the detection process 204 can be improved.

Referring to FIG. 16 together with FIG. 17, the cover 56T according to the third modification of the optical head 40 is a further modification of the cover 56F (see FIG. 14) according to the first modification described above. The cover 56T has only one transmitting portion 57 and only two shielding portions 58 . Each of the transmissive portion 57 and the shielding portion 58 extends along an oblique direction that intersects the side 562 of the cover 56T. The transmitting portions 57 and the shielding portions 58 of the cover 56T are alternately arranged in the arrangement direction orthogonal to the oblique direction. The oblique direction of the third modification is preferably oblique to the side 562 by about 45 degrees or -45 degrees. According to the cover 56T of the third modified example, the contrast between the bright portion 52P and the dark portion 54P becomes higher, and the defect 84 becomes easier to detect.

The cover 56T of the third modified example can be further modified. For example, each of the transmissive portion 57 and the shielding portion 58 may extend parallel to the side 562 or may be orthogonal to the side 562 . The cover 56T may have a peripheral transparent portion 59 (see FIG. 15).

The size of the transparent portion 57 of the cover 56T of the third modification is small. Referring to FIG. 16 together with FIG. 5, most of the light emitting element 522 is covered with the shielding portion 58 of the cover 56T. Referring to FIG. 16 together with FIG. 7, when providing the cover 56T of the third modified example, the light emitting element 522 may be provided only in the portion corresponding to the transmissive portion 57 in the main portion 51 . More specifically, one bar illumination corresponding to the transmissive portion 57 may be provided in the main portion 51 . By forming the main portion 51 in this way, the number of light emitting elements 522 can be greatly reduced, and the manufacturing cost can be reduced. In addition, it is possible to greatly reduce the electric power required for causing the light emitting element 522 to emit light.

12 and 17 to 19, the inspection system 10A inspects the target surface 82 fixed so as not to move with respect to the installation surface 92 in the same procedure as the inspection system 10 (see FIG. 1). can. Specifically, comparing FIG. 13 with FIG. 2, the inspection procedure by inspection system 10A is similar to the inspection procedure by inspection system 10, except that control by support device controller 70A is added. Only parts that differ from the inspection procedure by the inspection system 10 will be described below.

Referring to FIGS. 13 and 17 to 19, before the apparatus main body 22 instructs the photographing apparatus 60 to start photographing ("2-2" in FIG. 13), the apparatus main body 22 instructs the supporting apparatus controller 70A to to start moving (“2-1” in FIG. 13). The support device 72A instructed to start moving moves the optical head 40 along the set movement path 48 from the initial position 488 to the end position 489 of the movement path 48 . The photographing device 60 instructed to start photographing continues photographing according to the set photographing cycle.

When the optical head 40 reaches the end position 489 of the movement path 48, the device main body 22 instructs the support device control device 70A to end the movement of the support device 72A ("4-1" in FIG. 13). As a result, the movement of the optical head 40 is completed. Next, the device main body 22 instructs the photographing device 60 to finish photographing (“4-2” in FIG. 13). As a result, the photographing by the photographing device 60 ends.

Referring to FIG. 17, the imaging device 60 of the optical head 40 may move along the movement path 48 shown. The photographing device 60 temporarily stops at each of the stop points 482 and photographs the photographed images 60P at each of the photographing points 484 . The moving path 48 in FIG. 17 can be easily set by teaching. However, according to the movement path 48 of FIG. 17, the speed of the optical head 40 decreases around the stop point 482 . In addition, since the optical head 40 vibrates at each stop point 482 due to deceleration, a waiting time is required until the vibration subsides. That is, time loss occurs at each stop point 482 .

Referring to FIG. 18, the imager 60 of the optical head 40 (see FIG. 12) may move along the illustrated movement path 48 . The photographing device 60 continuously photographs the photographed images 60P at each of the photographing points 484 without stopping even once from the initial position 488 to the end position 489 of the movement path 48 . It is difficult to set the moving path 48 in FIG. 18 by teaching. However, according to the moving path 48 of FIG. 18, the time loss associated with stopping the optical head 40 can be eliminated, thereby shortening the overall inspection time. Comparing FIG. 18 with FIG. 17, from the viewpoint of shortening the overall inspection time, the more stopping points 482, the more preferable the moving path 48 of FIG.

Referring to FIG. 19, the target surface 82 may have a three-dimensional shape. In this case, the optical head 40 may move along the illustrated movement path 48, and the photographing device 60 moves from the initial position 488 to the end position 489 of the movement path 48 without stopping even once. can be shot without stopping.

Referring to FIG. 20, the target surface 82 may have a complex shape with unevenness in the X direction, and may have a size larger than the movable range of the optical head 40 in the YZ plane. good. In this case, the object 80 may be moving with respect to the installation surface 92 . For example, the illustrated object 80 is fixed to the support 77A. The support portion 77A is supported by a support base 78A such as a rail, and is movable only in the Y direction. The support portion 77A continues to move along the -Y direction at a constant speed VC during the inspection except for the effects of vibration and the like. That is, when the imaging device 60 captures the projection image 50P (see FIG. 17), the target surface 82 is moving relative to the fixed portion 74A of the support device 72A.

Referring to FIG. 21 together with FIG. 20, the optical head 40 of this modification moves along the movement path 48 shown in FIG. As described above, the projection device 50 and the imaging device 60 of this modified example are integrated as the optical head 40 . Therefore, the movement path 48 shown in FIG. 21 is also the movement path 48 of each of the projection device 50 and the imaging device 60 . In other words, each of the projection device 50 and the imaging device 60 moves along the movement path 48 shown in FIG.

The support device 72A of this modified example moves the optical head 40 in a figure eight shape while the target surface 82 continues to move at a constant speed VC. That is, the movement path 48 of the imaging device 60 of this modified example includes a figure-eight path in which the imaging device 60 starts moving from the initial position 488 and returns to the initial position 488 . More specifically, the optical head 40 moves the passing point 486 located at the center of the figure 8 at a speed such that the -Y component matches the constant speed VC, or the -Y component is kept constant considering the influence of vibration and the like. It moves linearly at a speed slightly higher than the speed VC. The optical head 40 moves at a constant speed in a curved line while maintaining a moving speed through four passing points 486 positioned at the four corners of the figure eight. This figure-of-eight movement enables non-stop photography by the photographing device 60 .

Referring to FIG. 21 in conjunction with FIG. 18, the figure-of-eight motion path 48 of the optical head 40 maps onto the object surface 82 resulting in a motion path 48 similar to that of FIG. As can be understood from FIGS. 18 and 21, by repeating the figure-of-eight movement of this modified example, the entire object surface 82 that continues to move can be photographed without stopping. However, the present invention is not limited to this. For example, the optical head 40 may move along a direction that is oblique to the direction of movement of the target surface 82, then pause briefly, and then move in the opposite direction. In this case, when the movement path 48 due to the reciprocating movement of the optical head 40 is mapped onto the target surface 82, the movement path 48 is the same as that shown in FIG. As can be understood from the above description, the movement path 48 of the photographing device 60 should include a path along a direction oblique to the movement direction of the target surface 82 . Furthermore, the optical head 40 may repeat vertical movement at an appropriate speed.

Referring to FIG. 22 in conjunction with FIG. 12 , the optical head 40 of this modification may include a sensor (not shown) capable of detecting the target surface 82 . The sensor may be connected to the device main body 22 so as to be communicable. As will be explained below, by providing such a sensor, the position of the defect 84 on the target surface 82 on the YZ plane can be easily calculated.

Referring to FIG. 22 in conjunction with FIG. 13, a sensor (not shown) continues to detect whether the target surface 82 is within the imageable range. After performing the projection setting ("1-1" in FIG. 13) and the shooting setting ("1-2" in FIG. 13), the device main body 22 acquires the detection result of the sensor at regular intervals. When the device main body 22 detects that the target surface 82 has moved into the photographable range, it instructs the support device 72A to start moving (“2-1” in FIG. 13). The device body 22 then instructs the photographing device 60 to start photographing (“2-2” in FIG. 13). When the device main body 22 detects that the object surface 82 has moved out of the photographable range, it instructs the support device 72A to end the movement (“4-1” in FIG. 13). After that, the device body 22 instructs the photographing device 60 to finish photographing ("4-2" in FIG. 13).

When the photographing device 60 transmits the photographed image 60P (see FIG. 17) to the device main body 22 (“3-5” in FIG. 13), the photographing device 60 also transmits the elapsed time TS from when the photographing start instruction is given until the photographing is performed. do. When the defect 84 is detected by the inspection, the apparatus body 22 calculates the position of the defect 84 on the target surface 82 on the YZ plane based on the elapsed time TS of the photographed image 60P in which the defect 84 is detected.

More specifically, the apparatus main body 22 adds the time (moving time correction value T2) from when the support device 72A starts moving until when the photographing device 60 starts photographing to the elapsed time TS. The position of 60P on the YZ plane (the position on the figure-of-eight path) is calculated. In addition, the device main body 22 adds the time from when the target surface 82 is detected to when the imaging device 60 starts imaging (transportation time correction value T1) to the elapsed time TS, so that the captured image 60P in which the defect 84 is detected is corrected. A relative position in the Y direction with respect to the target surface 82 is calculated. Based on the above calculation results, the position of the defect 84 on the target surface 82 on the YZ plane can be calculated.

The apparatus main body 22 of this modified example calculates the position of the defect 84 on the YZ plane on the target surface 82 as described above. Referring to FIG. 10, the calculated position may be displayed on display 28 along with display image 219 . However, the present invention is not limited to this. For example, the method of calculating the position of the defect 84 on the YZ plane on the target surface 82 is not particularly limited.

Referring to FIG. 23, when inspecting a target surface 82 such as the body of a car, the inspection system 10A may include three optical heads 40. The optical head 40 may be a single optical head. Two of the optical heads 40 may be placed on the installation surface 92, which is the floor. Another one of the optical heads 40 may be placed on the installation surface 92 which is the ceiling.

The present invention is not limited to the embodiments and modifications already described, and can be applied in various ways. For example, referring to FIG. 12, support device 72A may be controlled directly using a controller that is separate from device body 22 . Also, the optical head 40 may be made small and light enough to be portable. The portable optical head 40 may be carried by an inspector for inspection instead of being supported by the support device 72A.

Furthermore, a device such as the optical head 40 combining the two devices of the projection device 50 and the imaging device 60 is not necessarily required. For example, the projection device 50 may be supported by a robot arm separate from the support device 72A that supports the imaging device 60 . That is, the projection device 50 and the imaging device 60 may be movable independently of each other. The arrangement of the projection device 50 and the imaging device 60 is not limited to this example, and various modifications are possible. Three modified examples of the inspection system 10 (second to fourth modified examples of the inspection system 10) will be described below. Each of the second to fourth modifications of the inspection system 10 is not provided with a device such as the optical head 40 that integrates the projection device 50 and the imaging device 60 .

Comparing FIG. 24 with FIG. 20, inspection system 10B according to the second modification of inspection system 10 (see FIG. 1) does not include optical head 40 of inspection system 10A. More specifically, the inspection system 10B includes a plurality of projection devices 50 and one imaging device 60 instead of the optical head 40 . Each projection device 50 is fixed to the wall surface of the examination space 90 . On the other hand, the imaging device 60 is movably supported by the same supporting device 72A as the inspection system 10A. Except for the differences noted above, inspection system 10B is configured and functions similarly to inspection system 10A, as described below.

Referring to FIG. 24 in conjunction with FIGS. 12 and 17, inspection system 10B is an inspection system for detecting defects 84 in target surface 82. FIG. In addition to the projection device 50 and the imaging device 60, the inspection system 10B includes a processing device 20, a display device 28 incorporated in the processing device 20, and a support device 72A. Each projection device 50 is capable of projecting a planar projection image 50P onto a target surface 82 . Each of the projection images 50P includes one or more bright portions 52P and one or more dark portions 54P having a luminance lower than that of the bright portions 52P. The photographing device 60 is arranged so as to be able to photograph the projection image 50P projected onto the target surface 82 . The processing device 20 includes a device main body 22 . The device body 22 can communicate with the imaging device 60 .

Referring to FIG. 24 together with FIG. 12, the support device 72A of this modified example is a robot arm similar to the support device 72A of the inspection system 10A, and has a fixed portion 74A and a support portion 76A. . The fixed portion 74A is fixed so as not to move. The support portion 76A is relatively movable with respect to the fixed portion 74A. Imaging device 60 is supported by support 76 A and is thereby movable along object surface 82 .

Referring to FIG. 24 in conjunction with FIG. 17, when the imaging device 60 captures the projection image 50P, the target surface 82 is moving relative to the fixed portion 74A of the support device 72A. More specifically, the object 80 of this modified example is fixed to a support portion 77A movably supported by a support base 78A, like the object 80 of the inspection system 10A. The support portion 77A continues to move along the -Y direction at a constant speed VC during inspection. That is, the target surface 82 is moving relative to the fixed portion 74A of the support device 72A.

In addition to the movement of object surface 82 described above, imager 60 is moving relative to mounting surface 92 during inspection. For example, the imaging device 60 moves along a predetermined movement path 48 (see FIG. 21). The movement path 48 of the imaging device 60 includes a figure 8-shaped path along which the imaging device 60 starts moving from an initial position 488 and returns to the initial position 488 . That is, the movement path 48 includes a path along a direction oblique to the movement direction of the target surface 82 .

The imaging device 60 captures a captured image 60P including the projected image 50P while moving relative to the target surface 82 . The target surface 82 is moving relative to the wall-fixed projection device 50 while the imaging device 60 captures the captured image 60P. In other words, the projection device 50 is moving relative to the target surface 82 while the imaging device 60 captures the captured image 60P.

Referring to FIG. 24 together with FIG. 13, the device body 22 receives the captured image 60P from the imaging device 60, and detects the defect 84 (see FIG. 17) while reducing the resolution of the received captured image 60P. The display device 28 displays the defects 84 detected by the device body 22 . As can be understood from the above description, this modification can also provide a new inspection system 10B capable of detecting minute defects 84 on the target surface 82 even when the target surface 82 is moving. .

Referring to FIG. 24, the photographing device 60 of this modified example is directly fixed and supported by a support device 72A. Further, according to this modification, four projection devices 50 are fixed to the wall surface. However, the present invention is not limited to this. For example, the imaging device 60 may be indirectly secured and supported by the support device 72A via a device similar to the optical head 40 (see FIG. 3). The support device 72A is not limited to a robot arm. For example, the support device 72A may be a support portion 76 (see FIG. 1) that is movable only in the Y direction. On the other hand, the photographing device 60 may be fixed so as not to move with respect to the installation surface 92 . Also, the number and arrangement of the projection devices 50 are not particularly limited. For example, one large projection device 50 may be fixed to the wall. Furthermore, the entire wall surface may function as the projection device 50 .

Comparing FIG. 25 with FIG. 23, an inspection system 10C according to the third modification of the inspection system 10 (see FIG. 1) has a wall surface functioning as a projection device 50 and three imaging devices 60 instead of the optical head 40. It has A slit pattern illumination such as a fluorescent lamp or a bar illumination is provided on the wall surface, and surrounds the entire object 80 on the XZ plane. The slit pattern illumination provided on the wall functions as a light emitting section of the projection device 50 . Each imaging device 60 is movably supported by the same support device 72A as the inspection system 10A. Except for the differences described above, the inspection system 10C is configured similarly to the inspection system 10A shown in FIG. 23 and functions similarly to the inspection system 10A.

Referring to FIG. 25, the wall surface (projection device 50) of this modification has a size large enough to project a projection image 50P (see FIG. 17) over the entire body of the vehicle. However, the present invention is not limited to this. For example, the wall surface may have a size small enough to project the projection image 50P over the entire door mirror cover of the car. The support device 72A is not limited to a robot arm. For example, the support device 72A may be a support base 78 (see FIG. 1) that is movable only in the Y direction. good.

Referring to FIG. 26 together with FIG. 19, an inspection system 10D according to the fourth modification of the inspection system 10 (see FIG. 1) includes an inspection table 72D instead of the optical head 40. FIG. The inspection table 72D is fixed to the installation surface 92 of the inspection space 90, and includes a portion (hereinafter referred to as "inspection section") in which a tunnel-shaped hole is formed, and a conveyor. The conveyor continues to move inside the inspection section along the +Y direction at a constant speed VC. A projection device 50 and an imaging device 60 are provided on the wall surface inside the inspection section. Inspection system 10C is configured and functions similarly to inspection system 10, except for the differences described above.

A small object 80 is placed on one end of the conveyor of the inspection table 72D, passes through the inspection section together with the conveyor, and moves to the other end of the conveyor. When the object 80 passes through the inspection section, the projecting device 50 projects a projected image 50P onto the object surface 82, and the imaging device 60 takes a captured image 60P including the projected image 50P.

Each of the projection device 50 and the imaging device 60 of the inspection system 10</b>D is fixed so as not to move with respect to the installation surface 92 . On the other hand, when the imaging device 60 captures the projection image 50P, the target surface 82 is moving relative to each of the projection device 50 and the imaging device 60 . In other words, the imaging device 60 captures the captured image 60P including the projected image 50P while moving relative to the target surface 82 . Also, the projection device 50 moves relatively to the target surface 82 while the imaging device 60 captures the captured image 60P. This modification can also provide a new inspection system 10D capable of detecting minute defects 84 on the target surface 82 even when the target surface 82 is moving.

The above-described second to fourth modified examples can be variously combined with the already described embodiment and first modified example, and can be variously modified in the same manner as the already described embodiment and first modified example. .

10, 10A, 10B, 10C, 10D inspection system 20 processor 201 projection control program 202 imaging control program 203 inspection processing program 204 defect detection processing 206 encoding processing 207 post-processing 208 defect identification processing 210 input image 212 intermediate layer 214 first output Layer 215 Intermediate layer 216 Second output layer 218 Output image 219 Display image 22 Apparatus body 24 Storage device 26 Input device 28 Display device 30 Projection control device 40 Optical head 42 Base member 44 Supported part 46 Post 48 Movement path 482 Stop point 484 Shooting point 486 Passing point 488 Initial position 489 End position 50, 50C, 50X Projection device 51, 51X Main part 52, 52X Light emitting part 522 Light emitting element 53 Diffusion plate 54 Light emitting surface 54L Projected light 56, 56F, 56S, 56T Cover 562 Side 57 transmission part 58 shielding part 59 peripheral transmission part 50P projected image 52P bright part 54P dark part 60 imaging device 62 optical axis 60P captured image 70A support device control device 72, 72A support device 72D examination table 74A fixed part 76, 76A, 77A support Parts 78, 78A Support table 80 Target object 82 Target surface 84 Defect 86 Intermediate point 88 Normal line 90 Inspection space 92 Installation surface

Claims (15)

An inspection system for detecting defects in a target surface, comprising:
The inspection system includes a processing device, a display device, a projection device, and an imaging device,
The projection device is capable of projecting a planar projection image onto the target surface,
The projected image includes one or more bright portions and one or more dark portions having a luminance lower than that of the bright portions,
The imaging device is arranged so as to be able to capture the projection image projected onto the target surface,
The processing device includes a device body,
The device main body is capable of communicating with the imaging device,
The imaging device captures a captured image including the projected image while moving relative to the target surface;
The projection device moves relative to the target surface while the imaging device captures the captured image,
The device main body receives the captured image from the imaging device,
The device main body performs inspection processing on the received captured image to detect defects,
The inspection processing includes defect detection processing using a neural network,
The defect detection processing includes encoding processing, post-processing, and defect identification processing,
In the encoding process, the captured image is convolved two or more times to extract features while reducing the resolution to generate a first output layer;
In the post-processing, while maintaining the low resolution of the first output layer, an intermediate layer obtained during the encoding process is superimposed to generate a second output layer;
The defect specifying process creates an output image in which the defect area is specified based on the second output layer and has a lower resolution than the captured image,
The apparatus main body detects the defect while gradually decreasing the resolution of the received captured image two or more times by the inspection process ,
The display device displays an image in which the output image is superimposed on the captured image, thereby displaying the defect detected by the device body superimposed on the captured image,
The inspection system, wherein the resolution of the area corresponding to the defect displayed on the display device is lower than the resolution of the captured image. An inspection system according to claim 1 ,
The imaging device is fixed so as not to move,
An inspection system wherein the target surface is moving relative to the imaging device when the imaging device captures the projection image. An inspection system according to claim 1 ,
The inspection system comprises a support device,
The support device has a fixed portion and a support portion,
The fixed part is fixed so as not to move,
The support portion is relatively movable with respect to the fixed portion,
The inspection system, wherein the imager is supported by the support and is thereby movable along the object surface. An inspection system according to claim 3 ,
The inspection system comprises an optical head,
The optical head includes the projection device, the imaging device, and a supported portion,
The projection device and the imaging device are fixed to each other,
The supported portion is supported by the support portion, whereby the optical head maintains the distance between the photographing device and the target surface within a range in which the photographing device can photograph the projected image. is movable along the target surface while
Each of the projection device and the imaging device is directly or indirectly fixed to the supported portion,
The projection device projects the projection image along a projection direction,
The imaging device captures the projected image along an optical axis,
When the photographing device photographs the projection image, the projection direction of the projection device and the optical axis of the photographing device intersect at a predetermined angle across a normal line at an intermediate point located in the middle of the projection image. An inspection system arranged to The inspection system according to claim 3 or claim 4 ,
The inspection system, wherein the support device is a robot arm. The inspection system according to any one of claims 3 to 5 ,
The inspection system, wherein the target surface moves relative to the fixed portion of the support device when the imaging device captures the projection image. An inspection system according to claim 6 ,
The photographing device moves along a predetermined movement route,
The inspection system, wherein the moving path includes a path along a direction oblique to the moving direction of the target surface. An inspection system according to claim 7 ,
The inspection system, wherein the movement path of the imaging device includes a figure-eight path in which the imaging device starts moving from an initial position and returns to the initial position. The inspection system according to any one of claims 1 to 8 ,
The projection device comprises a main section and a cover,
the main portion includes one or more light-emitting portions corresponding to the bright portions of the projected image;
each of the light emitting units has a light emitting surface and emits projection light from the light emitting surface;
The cover has one or more transmissive portions and one or more shielding portions corresponding to the light emitting portions, respectively;
The cover is attached to the main portion,
The inspection system, wherein the transmitting section transmits the projection light, and the shielding section shields the projection light. The inspection system according to any one of claims 1 to 8 ,
The projection device comprises a main section and a cover,
The main part has a light emitting surface, and emits projection light from the entire light emitting surface,
The cover has one or more transmission parts and one or more shielding parts,
The cover is attached to the main portion,
The inspection system, wherein the transmitting section transmits the projection light, and the shielding section shields the projection light. The inspection system according to claim 9 or 10 ,
each of the transmitting portion and the shielding portion of the cover extends along an oblique direction that intersects a side of the cover;
The inspection system, wherein the transmitting portion and the shielding portion of the cover are alternately arranged in an arrangement direction perpendicular to the oblique direction. The inspection system according to claim 9 or 10 ,
The cover has a peripheral transmitting portion in addition to the transmitting portion and the shielding portion,
The peripheral transparent portion surrounds all the transparent portions and the shielding portions,
each of the transmitting portion and the shielding portion extends between the peripheral transmitting portions along an oblique direction that crosses the sides of the cover;
The inspection system, wherein the transmitting portion and the shielding portion of the cover are alternately arranged in an arrangement direction perpendicular to the oblique direction. The inspection system according to claim 11 or claim 12 ,
The inspection system, wherein the cover has only one transmissive portion and only two shielding portions. A program for causing a computer to function as an apparatus body in the inspection system according to any one of claims 1 to 13 . 15. A storage medium storing files of the program of claim 14 .

Images