JP2021184180A - Image inspection system - Google Patents

Abstract

To improve convenience by making it possible to easily grasp an object area to be an object of image inspection.SOLUTION: An inspection setting part generates a three-dimensional object 200 that three-dimensionally represents an inspection object to display the three-dimensional object on a monitor, and receives designation of an object area to be an object of image inspection on the three-dimensional object 200 obtained by adjusting a posture. The inspection setting part sets the designated object area in height data.SELECTED DRAWING: Figure 14

Description

The present invention relates to, for example, an image inspection system for inspecting an imaged work or the like.

Conventionally, as disclosed in Patent Document 1, for example, an image processing device that acquires height information of an inspection object such as a work by using a pattern projection method has been known. In the image processing apparatus disclosed in Patent Document 1, a plurality of luminance images are acquired by a camera while sequentially irradiating a plurality of pattern projection lights on an inspection object, and a plurality of luminance images acquired by the camera are used as a controller. Transfer to generate height data in the controller. The controller executes a process of converting the height data within a predetermined height range into a height image having a smaller number of gradations than the height data, that is, a height extraction process. After the height extraction processing, the controller performs image processing using the height image to determine the quality of the inspection target object, and outputs the determination signal to an external device such as a PLC.

Japanese Unexamined Patent Publication No. 2019-168285

By the way, it is also possible to connect a setting device composed of a general-purpose personal computer or the like to the controller and make various settings for the height data output from the controller on the setting device. In this case, first, the setting device converts the height data output from the controller into an image having a smaller number of gradations than the height data. At the time of image inspection, the user sets a target area (line area or surface area) to be image-inspected on the height image, and the height image at the time of this setting fixes the viewpoint at the position of the camera. Since it is an image when the inspection object is viewed from directly above, it is difficult for the user to grasp the three-dimensional effect and the height difference of the inspection object, which is a problem in terms of usability.

Furthermore, in the height image displayed on the monitor, it is common to express the height difference of the inspection object by adding an artificially generated surface color different from the actual surface color of the inspection object. It is done. When such an expression form is adopted, if there is a large change in height, it can be recognized by the change in color, but there is almost no difference in surface color in a portion such as a small step, and it is difficult to grasp the difference in height.

In addition, the two-dimensional image of the inspection object captured at the same time as the height image and the artificially colored height image may be combined and displayed. This makes it easier to grasp the relationship between the portion of the inspection target and the height, but it becomes difficult to grasp the height difference because the color indicating the height difference and the actual surface color of the inspection target are mixed.

As described above, when it becomes difficult to grasp the height difference on the height image, what kind of target area is set for the inspection target when the user sets the target area for image inspection. It was difficult to grasp on the height image.

The present invention has been made in view of this point, and an object of the present invention is to easily grasp a target area to be image-inspected and to improve usability.

In order to achieve the above object, the first disclosure is an image inspection imaging device provided with an acquisition unit for acquiring height data in which height information of an inspection object is arranged on two-dimensional coordinates, and the image inspection. It is possible to assume an image inspection system including an inspection setting unit that performs various inspection settings based on user operations with a setting device connected to the image pickup device via a network.

The inspection setting unit has an object generation means that generates a three-dimensional object that three-dimensionally represents an inspection object based on the height data acquired by the acquisition unit, and a three-dimensional object generation means that is generated by the object generation means. On the display means for displaying the object on the monitor of the setting device, the posture adjusting means for adjusting the posture of the three-dimensional object displayed by the display means, and the three-dimensional object whose posture is adjusted by the posture adjusting means. An area designation receiving means that accepts the designation of the target area to be the target of the image inspection, and an area setting means that sets the target area designated by the area designation receiving means with respect to the height data acquired by the acquisition unit. May have.

According to this configuration, when the acquisition unit acquires the height data in which the height information of the inspection object is arranged on the two-dimensional coordinates, the object generation means generates a three-dimensional object based on the height data. .. The three-dimensional object can be displayed on the monitor of the setting device in the form of, for example, a three-dimensional polygon display or a three-dimensional point cloud display. Since the posture of the three-dimensional object can be adjusted while it is displayed on the monitor, the three-dimensional object viewed from a desired direction of the inspection object can be displayed on the monitor. The user can specify the target area on the three-dimensional object after adjusting the posture.

That is, since the user can see the three-dimensional object that three-dimensionally represents the inspection object from a desired direction, the height difference can be easily and surely grasped even in a portion such as a small step. Since the target area can be specified while grasping the height difference of the inspection object, the user can easily grasp where and what size or shape the target area is specified on the inspection object. The target area includes, for example, a linear area, an area surrounded by a figure, and the like, and the shape and size of the area can be freely specified as long as the area has a certain range.

When the target area is specified, the target area is set for the height data acquired by the acquisition unit. That is, since the information of the target area specified on the three-dimensional object can be reflected in the height data acquired at the time of operation, the image inspection can be performed on the desired area.

In the second disclosure, the acquisition unit may include an image pickup device that captures an image of an inspection object. The inspection setting unit may include an element generation means for generating a viewpoint specifying element for specifying a viewpoint at a position of the image pickup element when the designation of the target area is received by the area designation receiving means. .. The display means can display the viewpoint specifying element generated by the element generation means on the three-dimensional object and display it on the monitor.

According to this configuration, when the user specifies the target area to be the target of the image inspection, the viewpoint specifying element is displayed superimposed on the three-dimensional object. As a result, the user can specify the position of the image pickup device that images the inspection object, that is, the position of the viewpoint.

In the third disclosure, the viewpoint specifying element includes a plurality of line segments extending by a predetermined length from each of the plurality of points designated by the region designation receiving means along the image pickup axis of the image pickup device, and the plurality of line segments. It may include a graphic element composed of a plane or a curved surface formed based on a line segment.

According to this configuration, when the user specifies a plurality of points when specifying the target area, a surface formed by a plurality of line segments extending along the imaging axis is displayed as being overlapped on a three-dimensional object. .. This makes it easy to identify the position of the viewpoint.

In the fourth disclosure, the graphic element may be translucent. According to this configuration, the three-dimensional object on the other side of the graphic element can also be visually recognized, so that the target area can be easily specified.

In the fifth disclosure, the predetermined lengths of the plurality of line segments forming the graphic element can be determined according to the maximum height and the minimum height in the height image.

That is, if the dimension in the height direction of the graphic element or line segment becomes unnecessarily long, it may be difficult to see the height image, and if the dimension in the height direction of the graphic element or line segment is too short, the graphic element or line may be difficult to see. It is possible that the minute cannot be confirmed. In the present disclosure, the dimension in the height direction of the graphic element or the line segment is not unnecessarily long or too short, so that the position of the viewpoint can be easily specified.

In the sixth disclosure, the viewpoint-specific element may be position-adjustable on the three-dimensional object after being generated by the element generation means.

According to this configuration, if it is desired to change the position after the viewpoint specifying element is generated, it can be arranged at a desired position by moving the viewpoint specifying element. As the viewpoint specific element moves, the target area also moves to another area.

In the seventh disclosure, the viewpoint specifying element can be changed in shape on the three-dimensional object after being generated by the element generation means.

According to this configuration, after the viewpoint specifying element is generated, the shape can be changed to an arbitrary shape. The shape of the target area is also changed as the shape of the viewpoint specific element is changed.

In the eighth disclosure, the viewpoint specifying element may be prohibited from moving in the height direction of the three-dimensional object. In the XYZ coordinate system, for example, when the Z direction is the height direction, the Z coordinate of the viewpoint specifying element can be fixed and the viewpoint specifying element can be moved only in the XY direction.

That is, since the viewpoint specifying element is an element displayed when the target area to be inspected is set to the height data arranged on the two-dimensional coordinates, the X direction or Y of the three-dimensional object is displayed. It suffices to move in the direction, and it is not necessary to move in the direction in which the viewpoint is located (the height direction of the 3D object). If the viewpoint specific element can be moved in the height direction of the 3D object, the target area It is possible that the setting operation becomes complicated. In the present disclosure, since the viewpoint specifying element can be moved only in the X direction or the Y direction of the three-dimensional object, the operation of setting the target area becomes easy.

In the ninth disclosure, when the area designation receiving means receives the designation of the start point and the end point on the three-dimensional object, at the position of the line segment extending from the start point to the end point in the two-dimensional coordinates of the height data. The target area can be set to the corresponding coordinates.

According to this configuration, the target area including the start point and the end point can be automatically set in the height data only by the user performing an operation of designating the start point and the end point on the three-dimensional object.

In the tenth disclosure, the acquisition unit can generate a height image based on the acquired height data. The display means is configured so that the height image generated by the acquisition unit and the three-dimensional object generated by the object generation means can be switched and displayed on the monitor. The area designation receiving means can receive the designation of the target area to be the target of the image inspection on the height image generated by the acquisition unit.

For example, it may be easier to specify the target area for image inspection in a height image than in a three-dimensional object. In this case, the height image is displayed on the monitor and the image is inspected on the height image. It is possible to accept the designation of the target area that is the target of.

In the eleventh disclosure, the setting device includes an image processing unit that performs image processing on a target area set for height data by the area setting means, and an image processed by the image processing unit. It is equipped with an inspection unit that conducts visual inspections based on the above. This makes it possible to perform various visual inspections such as step measurement and scratches.

In the twelfth disclosure, the setting device externally creates a file in which a setting item consisting of a target area received by the area designation receiving means and register information indicating a place where a setting value of the setting item is stored are described. The setting value of the setting item acquired from the above and set by the user and the register information corresponding to the setting item included in the file can be transmitted to the image inspection image pickup apparatus. The image inspection imaging device can store the setting value of the setting item received from the setting device in a place indicated by the register information corresponding to the setting item.

According to this configuration, when both the image pickup device and the setting device conform to a common standard, the setting item and the register information indicating the location where the setting value of the setting item is stored are described. The setting device acquires the file. This file is, for example, a Device XML file when the standardization standard is the GenICam standard. This file may be acquired by the setting device from the image pickup device, or may be downloaded from, for example, a website or the like, and a file to be referred to to GenApi may be specified as a separate file.

When the user sets the target area to be image-inspected, the coordinates and size of the target area to be image-inspected become the set values, and the setting item called the target area and the coordinates and size are imaged together with the register information. Sent to the device. As a result, the settings in the image pickup device are made, so that the above settings can be freely changed in the setting device as long as the image pickup device conforms to the standardization standard, and the degree of freedom in selecting the model of the image pickup device is improved.

As described above, according to the present disclosure, the target area to be inspected is specified while adjusting the posture of the three-dimensional object representing the inspected object in three dimensions and grasping the height difference of the inspected object. Since it can be done, the user can easily grasp the target area, and by extension, the usability can be improved.

It is a figure which shows typically the schematic structure and operation state of the image inspection system which concerns on Embodiment 1 of this invention. It is a figure corresponding to FIG. 1 which concerns on Embodiment 2. FIG. FIG. 1 is a view corresponding to FIG. 1 according to the third embodiment. FIG. 1 is a view corresponding to FIG. 1 according to the fourth embodiment. It is a flowchart which shows the procedure which generates the height image based on the principle of a deflation metric. FIG. 1 is a diagram corresponding to FIG. 1 according to the fifth embodiment in which a height image is generated by using a photometric stereo method. It is a figure explaining the connection interface between an image inspection application and a camera. It is a flowchart which conceptually shows an example of the internal processing of an image pickup apparatus. It is a figure which shows the parameter set in the case of generating the inspection image using the principle of deflationmetry. It is a figure which shows the parameter set when the inspection image is generated by multi-spectral imaging. It is a block diagram of an inspection setting part. It is a flowchart which shows the example of the procedure of creating a new target area. It is a conceptual diagram of 3D hit determination. It is a figure explaining the case where the new creation of the linear target area is performed on the three-dimensional display or the two-dimensional display. It is a figure explaining the case where the new creation of the rectangular object area is performed on the three-dimensional display or the two-dimensional display. It is a flowchart which shows the 1st stage of the procedure of changing a target area. It is a flowchart which shows the 2nd stage of the procedure of changing a target area. It is a conceptual diagram of 3D hit determination. It is a conceptual diagram of 2D hit determination. It is a figure which shows the specific example 1 which performs the position adjustment of a linear target area. It is a figure which shows the specific example 2 which performs the position adjustment of a linear target area. It is a figure which shows the specific example which performs the position adjustment of a rectangular object area. It is a flowchart which shows the example of the procedure of measuring a step in a linear target area at the time of operation. It is a flowchart which shows the example of the procedure of inspecting a scratch in a rectangular target area at the time of operation. It is a flowchart which shows the example of the procedure of inspecting the height in a rectangular target area at the time of operation. It is a flowchart which shows the example of the procedure of inspecting the expiration date in the target area of an arc shape at the time of operation. It is a flowchart which shows the example of the procedure of engraving discrimination in the target area of an arc shape at the time of operation.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. It should be noted that the following description of the preferred embodiment is essentially merely an example and is not intended to limit the present invention, its application or its use.

(Embodiment 1)
FIG. 1 schematically shows an operating state of the image inspection system 1 according to the embodiment of the present invention. The image inspection system 1 is configured to inspect the presence or absence of defects in the work W by using an image of the work W as an inspection object, and images the lighting device 2 that illuminates the work W and the work W. The image pickup unit 3 is provided with a setting device 4 composed of a personal computer (PC) or the like for setting the image pickup unit 3. Since the work W is an object to be imaged by the image pickup unit 3, it can also be called an image pickup object. The lighting device 2 and the image pickup unit 3 constitute an acquisition unit A for acquiring height data. The image inspection image pickup device 300 constituting a part of the image inspection system 1 is an apparatus including the acquisition unit A. Further, the image pickup device 300 and the setting device 4 may be separated from each other, and the image pickup device 300 alone may be an embodiment of the present invention.

The setting device 4 includes a display unit 5, a keyboard 6, and a mouse 7. The display unit 5 includes a monitor including, for example, an organic EL display or a liquid crystal display, and includes an image captured by the image pickup device 300, an image after various processing of the image captured by the image pickup device 300, and various users. It is a part where an interface image (GUI) or the like can be displayed. Various user interface images and the like are generated in the main body of the setting device 4. The horizontal direction of the display unit 5 may be the X direction of the display unit 5, and the vertical direction of the display unit 5 may be the Y direction of the display unit 5.

Further, the keyboard 6 and the mouse 7 are conventionally known devices for operating a computer. By operating the keyboard 6 or the mouse 7, various information can be input to the setting device 4, various settings can be made, and an image or the like displayed on the display unit 5 can be selected. Instead of the keyboard 6 and the mouse 7, or in addition to the keyboard 6 and the mouse 7, a computer operation device such as a voice input device and a pressure-sensitive touch operation panel can also be used.

FIG. 1 shows a case where a plurality of work W are transported in the direction indicated by the white arrow in FIG. 1 in a state of being placed on the upper surface of the transport belt conveyor B. An external control device 8 including a programmable logic controller (PLC) or the like is connected to the setting device 4 as a device for sequence control of the conveyor belt B for transportation and the image inspection system 1. The external control device 8 may form a part of the image inspection system 1. Further, the external control device 8 does not have to be a component of the image inspection system 1.

In the description of this embodiment, the transport direction of the work W by the transport belt conveyor B (moving direction of the work W) is the Y direction, and the direction orthogonal to the Y direction in the plan view of the transport belt conveyor B is the X direction. The direction orthogonal to the X direction and the Y direction (the direction orthogonal to the upper surface of the conveyor belt B for transportation) is defined as the Z direction, but this is defined only for convenience of explanation.

The image inspection system 1 can be used for visual inspection of the work W, that is, for inspecting the presence or absence of defects such as scratches, stains, and dents on the surface of the work W, and the work W is based on the inspection results. It is also possible to make a pass / fail judgment. During its operation, the image inspection system 1 receives an inspection start trigger signal that defines the start timing of defect inspection (pass / fail determination inspection) from the external control device 8 via a signal line. The image inspection system 1 takes an image of the work W, illuminates, and the like based on the inspection start trigger signal, and obtains an inspection image after a predetermined process. After that, a visual inspection is performed based on the inspection image, and the inspection result is transmitted to the external control device 8 via the signal line. As described above, during the operation of the image inspection system 1, the inspection start trigger signal is repeatedly input and the inspection result is output between the image inspection system 1 and the external control device 8 via the signal line. As described above, the input of the inspection start trigger signal and the output of the inspection result may be performed via the signal line between the image inspection system 1 and the external control device 8, or other signals (not shown). It may be done via a line. For example, a sensor for detecting the arrival of the work W and the image inspection system 1 may be directly connected, and an inspection start trigger signal may be input from the sensor to the image inspection system 1. Further, the image inspection system 1 may be configured to automatically generate a trigger signal internally for inspection.

In addition to being configured with dedicated hardware, the image inspection system 1 may be configured by installing software on a general-purpose device, for example, installing an image inspection program on a general-purpose or dedicated computer. For example, the image inspection program may be installed on a dedicated computer in which hardware such as a graphic board is specialized for image inspection processing.

(Structure of acquisition unit A)
The acquisition unit A is configured to acquire a luminance image according to the amount of received light reflected from the work W, and to acquire height data in which the height information of the work W is arranged on two-dimensional coordinates. Is. The luminance image can be acquired by taking an image of the work W illuminated by the lighting device 2 by the image pickup unit 3. When the work W does not need to be illuminated, the luminance image can be acquired by capturing the work W with the image pickup unit 3 without illuminating the work W with the lighting device 2.

The method for acquiring the height data in which the height information of the work W is arranged on the two-dimensional coordinates is not particularly limited, and any method may be used, and the hardware and software corresponding to the method may be used. It can be selected as appropriate. The acquisition unit A can generate a three-dimensional image by acquiring height data.

That is, there are roughly two methods for acquiring height data, and one is a passive method for acquiring height data using an image captured under lighting conditions for obtaining a normal brightness image. (Passive measurement method), the other is an active method (active measurement method) in which height data is acquired by actively irradiating light for measuring in the height direction. A typical method of the passive method is a stereo measurement method. This involves preparing two image pickup units 3, arranging these two image pickup units 3 in a predetermined positional relationship to execute imaging, and performing a well-known calculation based on the two generated images. You can get the height data with.

Typical methods of the active method are the optical cutting method and the pattern projection method. In the optical cutting method, in the stereo measurement method described above, one of the imaging units 3 is replaced with a laser floodlight, a line-shaped laser beam is projected onto the work W, and the line light according to the shape of the surface of the work W. The three-dimensional shape of the work W is restored based on the degree of distortion of the image of. The optical cutting method does not require determination of the corresponding point, so stable measurement is possible.

The pattern projection method captures a plurality of images by shifting the shape, phase, etc. of a predetermined projection pattern projected on the work W, and analyzes the captured images to obtain a three-dimensional shape of the work W. How to restore. There are several types of pattern projection methods. The phase of the sine wave striped pattern is staggered to capture multiple images (at least 3 or more), and the phase of the sine wave is obtained for each pixel from the multiple images. The three-dimensional shape is restored by using the phase shift method to obtain the three-dimensional coordinates on the work surface using the obtained phase and the kind of spatial frequency swell phenomenon that occurs when two regular patterns are combined. More repotography method, the pattern itself projected on the work W is different for each shooting, for example, the stripe width becomes half the screen, 1/4, 1/8, etc. at a black and white duty ratio of 50%. A spatial coding method that sequentially projects striped patterns, captures a pattern projection image for each pattern, and obtains the absolute phase of the height of the work W, and projects multiple fine line-shaped pattern illuminations (multi-slits) on the work. A typical example is a multi-slit method in which the pattern is moved at a pitch narrower than the slit cycle and multiple shots are taken.

Further, the height data can be acquired by combining the above-mentioned phase shift method and the spatial coding method, but the height data is not limited to this, and the height data may be acquired by another method. In addition, any method other than the above-mentioned method, such as optical radar method (time of flight), in-focus method, co-focus method, white light interferometry, etc., can be adopted to acquire height data. I do not care.

The lighting device 2 and the imaging unit 3 are configured to realize a pattern projection method. Hereinafter, a specific configuration of the lighting device 2 and the image pickup unit 3 will be described.

(Structure of lighting device 2)
First, the lighting device 2 will be described. The lighting device 2 is a light emitting unit that projects structured lighting having a predetermined projection pattern on the work W, and includes a light emitting unit 2a and a lighting control unit 2b that controls the light emitting unit 2a. The light emitting unit 2a and the lighting control unit 2b may be separate or integrated. Further, the lighting control unit 2b may be incorporated in the setting device 4. The light emitting unit 2a can be composed of, for example, a light emitting diode, a projector using a liquid crystal panel, an organic EL panel, a digital micromirror device (DMD), or the like, and can also be called an illumination unit. Although the light emitting diode, the liquid crystal panel, the organic EL panel, and the DMD are not shown, those having a conventionally known structure can be used. The lighting device 2 is connected to the setting device 4 via a signal line 100a, and can be installed away from the image pickup unit 3 and the setting device 4.

The lighting device 2 of the first embodiment is configured to be capable of performing uniform surface light emission. Further, the illuminating device 2 is configured to be capable of performing deflationary metric processing as an example of structured illuminating having a predetermined projection pattern, and therefore has a periodic illuminance distribution. It has a light emitting unit 2a that irradiates the work W with pattern light. That is, the lighting device 2 can be a pattern light illuminating unit that executes pattern light illumination that sequentially irradiates the work W with a plurality of different pattern lights. Hereinafter, the illuminating device 2 used when acquiring an inspection image including height data by performing a deflection metering process will be described.

When a plurality of light emitting diodes are used, the plurality of light emitting diodes can be arranged in a dot matrix to generate patterned light having a periodic illuminance distribution by controlling the current value. For example, in the case of Y-direction pattern light whose brightness changes in the Y direction, it can be expressed as pattern light in which a striped pattern is repeated in the Y direction, and when this Y-direction pattern light is generated, the phase of the illuminance distribution is generated. By shifting the light in the Y direction, it is possible to generate a plurality of Y-direction pattern lights having different phases of the illuminance distribution. The illuminance distribution of the Y-direction pattern light can also be represented by a waveform similar to the sin waveform. In this case, for example, the phase is changed by 90 °, and the Y-direction pattern light in the case of 0 ° and the Y-direction in the case of 90 °. It is possible to generate a pattern light, a Y-direction pattern light in the case of 180 °, and a Y-direction pattern light in the case of 270 °.

Further, in the case of the X-direction pattern light in which the brightness changes in the X-direction, it can be expressed as a pattern light in which the striped pattern is repeated in the X-direction, and when the X-direction pattern light is generated, the phase of the illuminance distribution is generated. By shifting the light in the X direction, it is possible to generate a plurality of X-direction pattern lights having different phases of the illuminance distribution. The illuminance distribution of the X-direction pattern light can also be represented by a waveform similar to the sin waveform. In this case, for example, the phase is changed by 90 °, and the X-direction pattern light in the case of 0 ° and the X-direction in the case of 90 °. It is possible to generate a pattern light, an X-direction pattern light in the case of 180 °, and an X-direction pattern light in the case of 270 °. That is, the lighting device 2 can illuminate the work W in different lighting modes. When the deflect metering process is performed, the pattern light irradiating the work W can be not only a sine waveform but also a pattern light such as a triangular wave.

By passing a current having the same current value through all the light emitting diodes, it is possible to irradiate light having a uniform illuminance distribution in the plane. By changing the current value flowing through all the light emitting diodes to be the same, the light emitting state can be changed from the dark surface emitting state to the bright surface emitting state.

Further, in the case of a liquid crystal panel and an organic EL panel, by controlling each panel, the light emitted from each panel can be made into a pattern light having a periodic illuminance distribution. In the case of a digital micromirror device, it is possible to generate and irradiate a pattern light having a periodic illuminance distribution by controlling the built-in micromirror surface. The configuration of the lighting device 2 is not limited to that described above, and any device or device capable of generating patterned light having a periodic illuminance distribution can be used.

Further, as will be described later, as the illuminating device 2 used when generating an inspection image by using the photometric stereo method, an illuminating device capable of individually irradiating illumination from at least three or more different directions can be used. Further, the lighting device 2 may be a lighting device configured to enable multispectral lighting, and the configuration of the lighting device 2 is not particularly limited.

(Structure of Imaging Unit 3)
Next, the image pickup unit 3 will be described. The image pickup unit 3 includes a camera 31, a condensing system optical system 32, and an image pickup control unit 33. The image pickup unit 3 is connected to the setting device 4 via a signal line 100b, and can be installed away from the lighting device 2 and the setting device 4.

The camera 31 has an image sensor including an image sensor 31d such as a CCD or CMOS that converts the intensity of light obtained through the condensing optical system 32 into an electric signal. The camera 31 acquires a luminance image according to the amount of light received from the work W, and the structured lighting projected by the lighting device 2 receives the reflected light reflected by the work W to receive a plurality of patterns. It is an image generation unit that generates a projected image. Further, the image pickup control unit 33 has a storage device and a signal processing device, and is a part that controls the exposure start and end of the camera 31, controls the exposure time, adjusts the analog gain, and the like. The drive of the image sensor and the transfer of image data can be controlled by the internal logic circuit. Further, the image pickup control unit 33 can perform various image processing, filter processing, image generation, and the like, and the image pickup unit 3 is a device having a filter processing function.

The condensing system optical system 32 is an optical system for condensing light incident from the outside, and typically has one or more optical lenses. Further, the condensing optical system 32 is configured to be capable of performing autofocus. The positional relationship between the image pickup unit 3 and the lighting device 2 is set so that the light emitted from the lighting device 2 toward the surface of the work W is reflected by the surface of the work W and incident on the condensing optical system 32. can do. When the work W is a transparent member such as a transparent film or a sheet, the pattern light emitted from the lighting device 2 toward the surface of the work W passes through the work W and is condensed by the image pickup unit 3. The positional relationship between the image pickup unit 3 and the lighting device 2 can be set so as to be incident on the system optical system 32. In any of the above cases, the image pickup unit 3 and the lighting device 2 are arranged so that the specular reflection component and the diffuse reflection component reflected on the surface of the work W are incident on the condensing system optical system 32 of the image pickup unit 3. Further, although a line camera in which the light receiving elements are arranged linearly in the Y direction can be used for the image pickup unit 3, an area camera (the light receiving elements are arranged so as to be arranged in the X direction and the Y direction) instead of the line camera. In the case of this area camera, a lighting form called coaxial lighting is also possible.

(Embodiment 2)
FIG. 2 is a diagram schematically showing a schematic configuration and an operating state of the image inspection system 1 according to the second embodiment of the present invention. In the second embodiment, the image pickup control unit 33 is separated from the camera 31 having the image pickup element and the condensing system optical system 32, and the illumination control unit 2b is incorporated into the image pickup control unit 33 and integrated. The image pickup control unit 33 is connected to the setting device 4 via a signal line 100b. The image pickup control unit 33 and the illumination control unit 2b can be separated from each other.

(Embodiment 3)
FIG. 3 is a diagram schematically showing a schematic configuration and an operating state of the image inspection system 1 according to the third embodiment of the present invention. In the third embodiment, the lighting control unit 2b is incorporated into the image pickup control unit 33 and integrated, and the display unit 5 and the mouse 7 are connected to the image pickup control unit 33. The setting device 4 is composed of, for example, a notebook computer or the like, and is connected to the image pickup control unit 33. The setting device 4 has a keyboard 6. Further, a mouse may be connected to the setting device 4. Various user interfaces and the like generated by the setting device 4 can be displayed on the display unit 5 via the image pickup control unit 33.

(Embodiment 4)
FIG. 4 is a diagram schematically showing a schematic configuration and an operating state of the image inspection system 1 according to the fourth embodiment of the present invention. In the fourth embodiment, the lighting device 2 and the image pickup unit 3 are integrated, and the lighting control unit 2b and the image pickup control unit 33 are also integrated. In the second to fourth embodiments, the same parts as those in the first embodiment are designated by the same reference numerals and the description thereof is omitted.

(Deflation metering)
In the first to fourth embodiments, height data is acquired based on a plurality of pattern projection images generated by the image pickup unit 3. Specifically, phase data on the surface of the work W is generated from a plurality of brightness images captured by the imaging unit 3 based on the principle of deflectometry, and the height information of the work W is two based on the phase data. It is configured to perform a deflection metering process to generate height data arranged on the dimensional coordinates, that is, a height image showing the shape of the work W.

As an example shown in FIG. 5, the light emitting unit 2a of the lighting device 2 includes a first light emitting unit 21, a second light emitting unit 22, a third light emitting unit 23, and a fourth light emitting unit 24. Each of the fourth light emitting units 21 to 24 can project a pattern.

Hereinafter, the generation of the height image will be described in detail with reference to FIG. When the first light emitting unit 21 of the lighting device 2 irradiates the work W with four pattern lights according to the spatial code method, the image pickup unit 3 generates a gray code pattern image set consisting of four different images. Further, when the first light emitting unit 21 of the lighting device 2 irradiates the work W with eight pattern lights according to the phase shift method, the image pickup unit 3 generates a phase shift pattern image set consisting of eight different images.

Similarly, when the second light emitting unit 22 of the lighting device 2 irradiates the work W with the pattern light according to the spatial code method, a gray code pattern image set is generated, and the pattern light according to the phase shift method is generated. When W is irradiated, a phase shift pattern image set is generated.

Similarly, when the third light emitting unit 23 of the lighting device 2 irradiates the work W with the pattern light according to the spatial code method, a gray code pattern image set is generated, and the pattern light according to the phase shift method is generated. When W is irradiated, a phase shift pattern image set is generated.

Similarly, when the fourth light emitting unit 24 of the lighting device 2 irradiates the work W with the pattern light according to the spatial code method, a gray code pattern image set is generated, and the pattern light according to the phase shift method is generated. When W is irradiated, a phase shift pattern image set is generated.

After generating the phase shift pattern image set, each image data of the phase shift pattern image set is acquired, and the relative phase calculation process can be performed by using the phase shift method. This makes it possible to acquire a phase image. This phase is a relative phase (pre-Unwrapping phase).

Further, after the gray code pattern image set is generated, each image data of the gray code pattern image set is acquired, and the spatial code calculation process is performed by using the spatial code method to obtain a striped number image. The stripe number image is an image in which a space to be irradiated with light is divided into a large number of small spaces, and a series of space code numbers are attached to the small spaces so that they can be identified.

After that, the absolute phase phase conversion process is performed. In the absolute phase phase conversion process, the phase image and the fringe number image are unwrapped to generate an absolute phase image (intermediate image). That is, since the phase jump can be corrected (phase unwrap) by the phase shift method by the spatial code number obtained by the spatial code method, it is possible to obtain a high-resolution measurement result while ensuring a wide dynamic range of the height. ..

The height may be measured only by the phase shift method. In this case, since the height measurement dynamic range is narrowed, the height cannot be measured correctly in the case of the work W having a large difference in height such that the phase is shifted by one cycle or more. On the contrary, in the case of the work W having a small change in height, since the striped image is not captured or combined by the spatial coding method, there is an advantage that the processing can be speeded up by that amount. For example, when measuring a work W with a small difference in the height direction, it is not necessary to take a large dynamic range, so that the processing time is shortened while maintaining highly accurate height measurement performance even with the phase shift method alone. be able to. Further, since the absolute height is known, the height may be measured only by the space code method. In this case, the accuracy can be improved by increasing the code.

The absolute phase image can also be said to be an angle image in which each pixel has the irradiation angle information of the pattern light on the work W. That is, since the first pattern image set (phase shift pattern image set) includes eight first pattern images taken by shifting the phase of the sinusoidal stripe pattern, the phase shift method should be used. Therefore, each pixel has information on the irradiation angle of the pattern light on the work W. That is, based on the plurality of first pattern images, it is possible to generate a first angle image in which each pixel has irradiation angle information of the first measurement pattern light on the work W. The first angle image is an image that images the angle of the light emitted from the first light emitting unit 21 to the work W.

Similarly, a second angle image in which each pixel has irradiation angle information of the second measurement pattern light on the work W based on a plurality of second pattern images, and each pixel works based on a plurality of third pattern images. A third angle image having the irradiation angle information of the pattern light for the third measurement on the W, and a third angle image in which each pixel has the irradiation angle information of the pattern light for the fourth measurement on the work W based on the plurality of fourth pattern images. A four-angle image can be generated.

The top image of the intermediate image in FIG. 5 is the first angle image, the second image from the top is the second angle image, the third image from the top is the third angle image, and the bottom image. The image is a fourth angle image. The portion of each angle image that appears to be filled in black is the portion that is the shadow of the illumination, and is an invalid pixel without angle information.

The first angle image and the second angle image can generate a first height image, and the third angle image and the fourth angle image can generate a second height image. After that, the first height image and the second height image can be combined to generate a combined height image. As a result, the number of invalid pixels can be reduced in the height image after composition.

(Embodiment 5)
FIG. 6 is a diagram schematically showing a schematic configuration and an operating state of the image inspection system 1 according to the fifth embodiment of the present invention. While the above-described first to fourth embodiments are in the form of generating an inspection image based on the principle of deflation, in the fifth embodiment, the inspection image is generated by using the photometric stereo method. It is configured.

Hereinafter, a specific method for generating an inspection image by using the photometric stereo method will be described with reference to FIG. 6 with the same reference numerals to the same parts as those in the first embodiment, and the description thereof will be omitted. Will be described in detail.

The image inspection system 1 according to the fifth embodiment can be configured in the same manner as the image inspection system disclosed in, for example, Japanese Patent Application Laid-Open No. 2015-232486. That is, the image inspection system 1 includes an image pickup unit 3 that images the work W from a certain direction, and a lighting device 200 for illuminating the work W from three or more different lighting directions, and is similar to the first embodiment. It includes at least a display unit 5, a keyboard 6, and a mouse 7. The acquisition unit A is composed of the image pickup unit 3 and the lighting device 200.

The lighting device 200 is configured to irradiate the work W with light from different directions, and controls the first to fourth light emitting units 201 to 204 and the first to fourth light emitting units 201 to 204. It has a lighting control unit 205 and a lighting control unit 205. The lighting device 200 is a portion that executes multi-directional lighting that irradiates the work W with light from different directions, and is configured to be capable of projecting structured lighting having a predetermined projection pattern. The first to fourth light emitting units 201 to 204 are arranged so as to surround the work W at intervals from each other. Light emitting diodes, incandescent bulbs, fluorescent lamps, and the like can be used for the first to fourth light emitting units 201 to 204. Further, the first to fourth light emitting units 201 to 204 may be separate bodies or may be integrated.

In this embodiment, the first to fourth light emitting units 201 to 204 are sequentially turned on, and when any of the first to fourth light emitting units 201 to 204 is turned on, the image pickup unit 3 takes an image of the work W. For example, when the lighting device 200 receives the first lighting trigger signal, the lighting control unit 205 lights only the first light emitting unit 201. At this time, the image pickup unit 3 receives the image pickup trigger signal and images the work W at the timing when the light is irradiated. When the lighting device 200 receives the second lighting trigger signal, the lighting control unit 205 lights only the second light emitting unit 202, and at this time, the imaging unit 3 images the work W. In this way, four luminance images can be obtained. The number of illuminations is not limited to four, but can be any number as long as it is three or more and the work W can be illuminated from different directions.

Then, the normal vector for the surface of the work W of each pixel is calculated by using the pixel value for each pixel which is in a corresponding relationship between the plurality of luminance images captured by the image pickup unit 3. The calculated normal vector of each pixel is subjected to differential processing in the X direction and the Y direction to generate a contour image showing the contour of the inclination of the surface of the work W. Further, the albedo of each pixel having the same number as the normal vector is calculated from the calculated normal vector of each pixel, and a texture drawing image showing a pattern in which the tilt state of the surface of the work W is removed is generated from the albedo. .. Since this method is a well-known method, detailed description thereof will be omitted. By synthesizing a plurality of luminance images based on the principle of photometric stereo, it is possible to generate a shape image (height image) showing the shape of the work W.

(Multi-spectral lighting)
As another embodiment, the illuminating device 2 capable of multispectral illuminating as structured illuminating having a predetermined projection pattern may be used. The multispectral illumination is to irradiate the work W with light having different wavelengths at different timings, and is suitable for inspecting color unevenness, stains, and the like of a printed matter (inspection object). For example, the lighting device 2 can be configured so that the work W can be irradiated with yellow, blue, and red in order. Specifically, the lighting device 2 may have LEDs of a large number of colors, or may be a liquid crystal display. The lighting device 2 may be configured by a panel, an organic EL panel, or the like.

The image pickup unit 3 captures the work W at the timing of being irradiated with light to obtain a plurality of luminance images. Then, a plurality of luminance images can be combined to obtain an inspection image. This is called multispectral imaging. The light to be irradiated may include ultraviolet rays and infrared rays.

(Connection interface between camera 31 and image inspection application)
The cameras 31 of the first to fifth embodiments are cameras compatible with the GenICam standard. The GenICam standard is a standard that standardizes the connection interface between the PC application and the camera 31, and as shown in FIG. 7, the image inspection application 40 to the camera 31 built mainly on the personal computer which is the main body of the setting device 4 This is a standardized interface for controlling the image and acquiring the image captured by the camera 31 by the image inspection application 40 of the setting device 4. If both the image pickup unit 3 and the image inspection application 40 of the setting device 4 support the GenICam standard, it is possible to connect the camera 31 and the image inspection application 40 of the setting device 4. The image inspection application 40 is composed of software installed in a personal computer. In this embodiment, the case where the camera 31 and the setting device 4 correspond to the GenICam standard as a standardization standard will be described, but the standardization standard is not limited to the GenICam standard, and may be another standardization standard. good.

Assuming that the image inspection application 40 is decomposed into a hierarchical structure, the GenICam layer 41 includes a higher layer (inspection unit 4A, image processing unit 4B) that actually performs image inspection, defect inspection, etc. in the image inspection application 40. It is positioned as an intermediate layer located between the layer 42 and the layer 42 that controls based on a specific network communication standard. The GenICam layer 41 can be roughly classified into two parts, GenApi41a and GenTL (TL: Transport Layer) 41b. The GenApi41a is a part that converts the setting items of the camera 31 and the register address inside the camera 31. With this GenApi41a, the image inspection application 40 does not specify a specific address of the camera 31, but abstractly, if it is an exposure time, it is an argument such as "ExposureTime", and if it is an analog gain, it is an argument such as "AnalogGain". When the setting item is specified in and connected to the camera 31, the file called DeviceXML (details will be described later) acquired from the camera 31 is analyzed to obtain the register address corresponding to the character string (called Shutter). Can be indexed.

GenTL41b defines an interface (API) that controls the transfer of data between the image inspection application 40 and the camera 31, and specifically, the specifications of the API for writing to and reading from the register of the camera 31 and the camera. It defines the specifications of the API for transferring the image data transferred from 31 to the upper layer of the image inspection application 40.

The physical standard for connecting the image inspection application 40 and the camera 31 may be a standard using a high-speed network. For example, a GigE Vision standard 3A using an Ethernet cable 31a and a USB3.0 compatible cable 31b are used. There is USB3Vision standard 3B. Therefore, the image pickup unit 3 and the setting device 4 are connected via a network, and this network may be wired or wireless using a high-speed network cable. Further, the network may be a network using a cable other than the Ethernet cable 31a and the USB3.0 compatible cable 31b, and is not particularly limited.

The GenICam standard does not specifically specify what is used as a physical communication standard, but only defines an abstract specification in the form of GenTL41b. As the lower layer 42 of the GenTL41b, a communication standard is defined by using a specific communication network such as GigE Vision standard 3A or USB3 Vision standard 3B. The specific communication standard is not limited to the GigE Vision standard 3A and the USB3 Vision standard 3B, and may be compatible with the GenICam standard. FIG. 7 shows a state in which cameras 31 and 31 corresponding to each standard are connected to the setting device 4 in order to explain the concepts of the GigE Vision standard 3A and the USB3 Vision standard 3B, but only one camera 31 is set. It can be used by connecting to the device 4.

A file (DeviceXML file) 31c called a DeviceXML is stored in the GenICam standard compatible camera 31. The DeviceXML file 31c is stored in a storage device (internal memory) built in the camera 31. The image inspection application 40 corresponding to the GenICam standard reads the DeviceXML file 31c from the camera 31 when connecting to the camera 31 to be set by the image inspection application 40. To read the DeviceXML file 31c, for example, there is a method of downloading the DeviceXML file 31c from the camera 31. The downloaded DeviceXML file 31c is retained by GenApi41a. The DeviceXML file 31c can be downloaded at the request of the image inspection application 40 or automatically when the camera 31 is connected. By downloading the DeviceXML file 31c, the setting device 4 downloads the DeviceXML file 31c. You can get it. It is also possible to obtain the DeviceXML file 31c corresponding to the camera 31 by another means (for example, download from a website) without downloading from the connected camera 31, and specify it for GenApi41a at the time of connection. be.

In the DeviceXML file 31c, all the setting items held inside the camera 31 and the register address (register information) in which the setting value of each setting item is stored are described in association with each other. The setting item is called a feature, and a character string for specifying an individual feature is assigned to each feature. The node of each feature is described with a reference to the node in which the specific register address is described. The register information also includes the register address and a character string that identifies the register.

In the DeviceXML file 31c, for example, there is a Shutter named "ExposureTime" (exposure time setting), and its attribute is instructed to refer to the register address "ExposureTimeReg", and a certain value is specified as a specific address. It is assumed that it was described. If the cameras 31 are different, the addresses may have different values, but the name of the feature called "ExposureTime" is common. In this way, by managing the setting items that exist as common setting items in many cameras 31 with a unified name, the image inspection application 40 does not need to be aware of the difference in the model of the camera 31 or the difference in the manufacturer. 31 can be set and controlled.

As described above, in the layer of GenApi41a, by analyzing the description contents of the DeviceXML file 31c, the Feature character string passed as an argument from the image inspection application 40, which is a higher layer, is converted into the address of the register. For example, when writing a value (value: 100.5) in a register corresponding to a feature called "ExposureTime", there are two values, an address value and a write value, such as WriteRegister (address value, 100.5). By executing the function with the argument of, the value of the register of the camera 31 can be set via the physical communication standard such as GigE Vision standard 3A or USB3Vision standard 3B.

In addition, although the DeviceXML file 31c defines what should be included as a common feature for each manufacturer, it is also possible to define a vendor's own feature. In the case of the camera 31 having a highly original function, it is possible to use a special function that does not exist in a general-purpose camera by accessing the camera 31 through a dedicated feature.

(Internal processing unit of image pickup unit 3)
FIG. 8 is a flowchart conceptually showing an example of the internal processing of the image pickup unit 3. In this flowchart, the group of the position correction 1 is described so as to be composed of the pattern search unit included in the group and the position correction unit, but the group of the position corrections 2 to 5 is not described. This is because the groups of position corrections 2 to 5 are folded to simplify the figure.

As shown in this flowchart, it is composed of a combination of a plurality of units. The unit is a unit for controlling image pickup and image processing, and by combining the units on a flowchart, the image pickup unit 3 can realize a desired operation. In the flowchart shown in FIG. 8, the unit is described as one step.

For example, if a function to perform a certain process is enabled by the user's setting, the process can be executed by adding a unit corresponding to the function. A unit can be defined as a set of a program for executing a process, parameters required for executing the process, and a storage area for storing the data of the process result. Each process can be performed by the image pickup control unit 33 shown in FIG. 1, and the storage area can be secured in the storage device of the image pickup control unit 33. The concept of the unit itself is not indispensable for realizing the external specifications of the camera 31 corresponding to the GenICam standard.

The flowchart shown in FIG. 8 is a flow chart in which a plurality of units are arranged vertically and horizontally, and may be simply called a flow. It may be a flowchart in which a plurality of units are arranged only in the vertical direction. The processing is executed in order from the start to the end of the flowchart shown in FIG. 8, but it is also possible to branch by providing the branch step SB1 in the middle. In the case of branching, a merging step SB2 can be provided before the end, whereby the merging step SB2 can be provided before proceeding to the end.

(Parameter set)
When the image inspection application 40 is operated in an actual inspection environment, the setting parameters of the image pickup unit 3 may be dynamically changed when the work W is switched or a change in the surrounding environment such as brightness is detected. be. When changing only a very limited parameter such as an exposure time, it can be dealt with by directly writing the corresponding Faceure value from the image inspection application 40.

On the other hand, in the case of the high-performance image pickup unit 3, the number of items that can be set increases, and the number of parameters that should be changed at one time also increases when the work W is switched. In this case, the time required to change the setting increases in correlation with the number of parameters. In an actual image inspection line, it is often the case that the work W is switched and the time from when the new work W reaches the imaging field of view of the image pickup unit 3 to when it goes out of the field of view is short, and a series of setting changes can be performed at high speed. There will be cases you want to do. At such times, a function called a parameter set may be used.

The parameter set holds a plurality of patterns of combinations of various parameters for imaging by the imaging unit 3 in advance so that each pattern can be managed by the parameter set number. For example, if there are three types of work W and you want to perform imaging and subsequent processing with different parameters for each, prepare three parameter sets.

By using the parameter set, a series of setting changes can be completed in a short time only by specifying the parameter set number corresponding to the type of the work W to be imaged next before the image of the work W is performed. From the viewpoint of the image inspection application 40, the feature that specifies the parameter set number can be considered as a kind of selector described later.

The selector that switches the target to be set from the image inspection application 40 side and the register that switches the parameter that the image pickup unit 3 internally refers to during operation can be independent or the same. The upper limit of the number of parameter sets that can be set is limited to the parameter holding space of the internal memory provided inside the image pickup unit 3.

The concept of the parameter set can also be expanded to the imaging unit 3 having a filtering function. For example, when the parameter set Index is 1, the binarization filter is executed, when the parameter set Index is 2, the expansion filter is executed, and when the parameter set Index is 3, the filter is not executed. It is assumed that the parameters corresponding to the set Index have been set. By doing so, it is possible to dynamically switch between imaging and the content to be executed as the filtering process by the parameter set Index.

The GenICam standard supports the function of dynamically switching imaging parameters. The image pickup unit 3 makes it possible to access all the setting parameters by Featur according to the GenICam standard. The image pickup function includes a function of synchronizing the lighting device 2 and the image pickup unit 3 and synthesizing images captured in a plurality of patterns by a dedicated algorithm (generation of an inspection image using the above-mentioned principle of deflectometry), and a wavelength. It has multiple illumination-imaging control modes, such as a multispectral imaging function that irradiates different lights to acquire multiple images. These can be set for each of the above parameter sets, and it is possible to execute image processing, filter processing, composition processing, and image output while dynamically switching according to conditions. The image generated by the image pickup function may be the image itself acquired by the image sensor while switching the lighting pattern of the illumination, or may include a plurality of images synthesized by the above-mentioned dedicated algorithm. ..

In the filtering process, it is possible to set a plurality of patterns in the same parameter set. For example, in the above-mentioned imaging function, a plurality of patterns of images are generated by executing a series of imaging, and it is possible to individually filter a plurality of different images generated there. .. Further, in another pattern, it is also possible to set a range (ROI: Region Of Interest) of a specific region for the same image and then filter only the inside of each region. The range of a specific area can be set by, for example, operating the keyboard 6 or the mouse 7. The specific area may be one or a plurality. The size of a specific area can be set arbitrarily.

The filter processing may be a multi-step filter in which a plurality of types can be repeatedly set in multiple steps for the same image. For example, after performing expansion filtering on a certain image, binarization filtering can be performed on that image. The filtering process is not limited to the multi-step filter process, and may be a single filter process.

By using the function of dynamically switching the imaging parameters of the GenICam standard, it is possible to combine a plurality of parameter sets into one flowchart as shown in FIG. On the internal processing flowchart of the image pickup unit 3, the unit group forming the flowchart from the branch step SB1 to the merging step SB2 is collectively called a parameter set. The example shown in FIG. 8 has four parameter sets, that is, first to fourth parameter sets. That is, there are a plurality of patterns of combinations of values of a plurality of selectors for realizing the process set by the user. The user can set which of the first to fourth parameter sets is selected. For example, when the parameter set number 1 is selected, the first parameter set is automatically selected.

After the start of the flowchart shown in FIG. 8, in the branching step SB1, after passing through each unit constituting any of the first to fourth parameter sets, they can be merged in the merging step SB2 and proceed to the end. When the parameter set number 1 is selected, the branch number becomes 1 in the branch step SB1, and each process of the first parameter set is executed. When the parameter set number 2 is selected, the branch number becomes 2 in the branch step SB1, and each process of the second parameter set is executed. When the parameter set number 3 is selected, the branch number becomes 3 in the branch step SB1, and each process of the third parameter set is executed. When the parameter set number 4 is selected, the branch number becomes 4 in the branch step SB1, and each process of the fourth parameter set is executed. The number of parameter sets is not limited to four and can be set arbitrarily.

Specific examples of the parameter set are shown in FIGS. 9 and 10. FIG. 9 is a parameter set for generating an inspection image using the principle of deflectometry. The unit UA1 executes a plurality of imaging processes with one trigger signal in order to generate an inspection image using the principle of deflection metric, the unit UA2 executes an expansion filter process, and the unit UA3 performs an averaging filter. The processing is executed, and the unit UA4 executes the shading inversion processing. That is, the multi-step processing defined by the parameter set including the filter processing is sequentially executed for the captured image captured by the unit UA1. After that, the unit UA5 transfers the image data in which the multi-step processing is sequentially executed to the PC, that is, outputs the image data to the setting device 4 or the like which is an external device. The image data may be retained internally without being transferred. Depending on the parameter set, it is possible to set not to perform multi-step processing. When transferred to the setting device 4, the inspection unit 4A of the image inspection application 40 shown in FIG. 7 can perform defect inspection and pass / fail determination. Conventionally known algorithms for defect inspection and pass / fail determination can be used.

On the other hand, FIG. 10 is a parameter set when an inspection image is generated by multispectral imaging. The unit UB1 executes a plurality of imaging processes with one trigger signal in order to generate an inspection image by multispectral imaging, and the unit UB2 executes a binarization filter process for a certain region (region 0). In the unit UB3, the expansion filter processing is executed for the region (region 1) different from the region 0. That is, also in this example, the multi-step processing defined by the parameter set including the filter processing is sequentially executed for the captured image captured by the unit UB1. After that, the unit UB4 does not execute the process. The unit UB5 transfers the image data to the PC in the same manner as shown in FIG. As in this example, there may be units that are not processed, that is, disabled units, in the parameter set, and it is a parameter set in which enabled units and disabled units are mixed. You may.

(Following moving objects)
When generating an inspection image based on the principle of photometric stereo or multispectral, a plurality of imaging processes are executed with one trigger signal, and the obtained plurality of images are combined. In this case, if the work W is moved between a plurality of imaging processes, the compositing process after imaging cannot be performed correctly, so it is necessary to detect the amount of deviation for each image. A search for detecting the amount of deviation can be performed by the search unit, and the amount of deviation is corrected based on the detected amount of deviation, and then a combination process of a plurality of images is executed.

The compositing process can be performed by the image calculation unit. A composite setting for synthesizing a plurality of captured images is included in the DeviceXML file as a setting item. Therefore, before outputting the captured image captured by the imaging unit 3 to the outside, the deviceXML file includes a composition setting for synthesizing a plurality of captured images as a setting item related to the processing applied to the captured image. become.

(Unit type)
There are multiple types of units, for example, a pattern search unit that performs pattern search processing to determine the inspection area, a position correction unit, an image calculation unit that internally performs image position correction, color extraction, filtering, etc., or Some of these units that perform relatively simple processing are combined to enhance their functionality. Each unit is a unit for executing a process applied to the captured image before outputting the captured image captured by the imaging unit 3 to the outside.

The pattern search unit is a unit for searching the work W and the inspection target portion in the work W from the images including the work W captured by the image pickup unit 3 and correcting the position of the work W in the captured image. be. For example, when the image inspection system 1 is set, edge detection is performed on an image of the work W by a well-known edge detection method, and the area specified by the detected edge is the work W or the inspection target portion of the work W. It can be stored in the storage device of the image pickup unit 3 as a model (search model image). A conventionally well-known method can be used for the edge detection process itself. For example, the pixel value of each pixel on the luminance image is acquired, and the change in the pixel value on the luminance image becomes equal to or greater than the threshold value for edge detection. If the area exists, the boundary part is extracted as an edge. The threshold value of edge extraction can be arbitrarily adjusted by the user.

After setting the image inspection system 1, when the image inspection system 1 is in operation, the work W that is sequentially conveyed is imaged to obtain an inspection image, and the pattern search unit can perform the work W or the work W on the obtained inspection image. The presence or absence of the inspection target portion of the work W is searched based on the stored model, and the position and angle of the work W are measured by the search process. The position of the work W can be specified by the X coordinate and the Y coordinate. Further, the angle of the work W can be an angle around the image pickup axis of the image pickup unit 3 or an angle around the Z axis shown in FIG.

The position correction unit searches the image including the work W captured by the image pickup unit 3 with the pattern search unit for the work W and the inspection target portion in the work W, measures the position and angle of the work W, and then measures the position and angle of the work W. This is a unit for correcting the position of the work W in the image. When operating the image inspection system 1, a plurality of work Ws are not always transported in the same position and posture, and work Ws in various positions and work Ws in various postures may be transported. .. Since the pattern search unit can search for the reference part of the work W and then the position correction unit can correct the position, the reference part of the work W is always at a constant position and the work W is a predetermined position. Position correction is performed by rotating the image or moving the image in the vertical or horizontal direction so that it is in the posture. As a position correction tool for performing position correction, a plurality of types of tools such as a pattern search can be prepared.

There are a plurality of types of image calculation units, for example, a unit that performs filter processing, a unit that performs synthesis processing that synthesizes a plurality of images captured by the image pickup unit 3, a unit that performs deflection metering processing, and a photometric stereo method. There are units that generate inspection images, units that perform multispectral imaging, and the like. Since there are a plurality of types of filtering, a plurality of types of units for performing filtering can be provided, for example, a binarization filter and an expansion filter. Since the generation of the inspection image by the deflection metering process undergoes a plurality of processes as described above, a unit may be provided for each process, a unit for generating a specular reflection component image, a unit for generating a diffuse reflection component image, and a unit for generating a diffuse reflection component image. A unit or the like for generating a reference phase difference image can be provided.

(Configuration of setting device 4)
The setting device 4 transmits the setting value of each setting item set by the user and the data indicating the register information corresponding to each setting item included in the DeviceXML file 31c to the image pickup unit 3, and sets the image pickup unit 3. It is a device to do.

As shown in FIG. 7, the setting device 4 includes a UI generation unit 4a. The UI generation unit 4a is a part that generates various user interface images. Various user interface images generated by the UI generation unit 4a are displayed on the display unit 5.

On the user interface image, it is possible to switch the image to be edited, display and change the setting items, and display the image to be edited. The setting items include, for example, position correction setting and filter processing setting, and as a feature corresponding to the position correction setting, a part for selecting whether to enable position correction and a type selection of the position correction tool are selected. The part to do is assigned. Further, as a feature corresponding to the filter processing setting, a selected filter type and a portion for selecting and adjusting parameters related to the filter setting such as the extraction size and the extraction color are assigned.

For example, when the user selects an arbitrary image, internally, the value of the selector that specifies the setting target is switched to the index value corresponding to the preprocessing module used to generate that image, thereby. The contents that can be set are switched according to the image. When the value of the selector that specifies the setting target is specified, the feature of one or more preprocessing modules pointed to by the selector corresponding to that value is read, an image that reflects the setting item is generated and displayed, and the position is displayed. Display each parameter value as a correction or filter setting. When the user operates the parameter value of the position correction or filter setting, the operation is accepted and the setting item of the unit corresponding to the preprocessing module corresponding to the value of the selector that specifies the setting target is changed.

The following method can be used to specify the unit to be accessed from the value of the selector. That is, the preprocessing module is composed of a plurality of units, and the selector for switching the index of the preprocessing module can be the one common to these units, and for the plurality of units constituting the preprocessing module. , Which feature belongs to which unit can be named so as to be uniquely determined by the feature name. As a result, the unit to be accessed can be specified from the combination of the value of the selector and the feature selected as the editing target.

Therefore, when the user changes the setting value for the setting item of the imaging unit 3, the data indicating the setting value of each setting item set by the user and the register information corresponding to each setting item included in the DeviceXML file. Is transmitted to the image pickup unit 3 to set the image pickup unit 3. Therefore, if the image pickup unit 3 conforms to the standardization standard, the set value can be changed from the setting device 4 side, and the image pickup unit 3 can change the set value. The degree of freedom in model selection is improved.

Further, since a part of the multi-step processing executed sequentially can be uniquely specified by the combination of the values of the selectors, the image pickup unit 3 conforming to the standardization standard can be used, for example, photometric stereo or deflect. It will be possible to perform multi-step processing such as generation of inspection images using the principle of metric, multispectral imaging, and filtering of the inspection images after generation.

(Specific configuration example of inspection setting unit 400)
A specific configuration example of the setting device 4 will be described below. As shown in FIG. 7, the setting device 4 includes an inspection setting unit 400. The inspection setting unit 400 is a part for performing various inspection settings based on user operations in the setting device 4, and as shown in FIG. 11, the object generation means 401, the display means 402, the posture adjusting means 403, and the area designation. It includes a receiving means 404, an element generating means 405, and an area setting means 406. The inspection setting unit 400 is constructed in the setting device 4 by executing the program installed in the setting device 4, and by executing the program on the setting device 4, each means 401 to 406 functions as shown in the figure. do. In FIG. 11, the means 401 to 406 are described separately for convenience, but one means may have a plurality of functions, and FIG. 11 is conceptual. Further, a part of each means 401 to 406 may be configured by hardware.

(Configuration of object generation means 401)
The object generation means 401 is a means for generating a three-dimensional object that three-dimensionally represents the work W based on the height data acquired by the acquisition unit A. The three-dimensional object may be a three-dimensional polygon display in which height data is connected by a triangular plane, or a three-dimensional point cloud display in which each height data is displayed for each pixel. good.

Here, 2.5D (2.5 dimensional) data will be described. Generally, when displaying a three-dimensional object on a monitor or the like, a 3D (three-dimensional) display is used, and in this three-dimensional display, the three-dimensional coordinates of X, Y, and Z are held and data is handled. There is. On the other hand, 2.5D has some restrictions close to 2D in 3D, and one height data is assigned to one pixel. Although 2.5D has a lower degree of freedom in data than 3D, it has a feature that measurement processing is easy, and even in this example, 2.5D data may be used for measurement processing. In 2.5D, since only the part visible from the camera is measured, the coordinate data on the back side of the object does not exist, but it can be displayed three-dimensionally by the above-mentioned three-dimensional polygon display, three-dimensional point cloud display, or the like.

By using the 2D data, it is possible to measure, for example, shading inspection, pattern search, edge extraction, blob, OCR, etc. for the target area (inspection area) to be image-inspected in the work W. Further, by using the 2.5D data, it is possible to measure the target area, for example, height measurement, 3D search, profile measurement, 3D blob, and the like. Although the types of data differ between 2D data and 2.5D data, the target area data itself does not change because the data is handled in pixel units of the camera 31.

Various filtering processes may be executed on 2D data and 2.5D data. For example, it is possible to perform a blurring process on 2D data, perform smoothing on 2.5D data, or perform 2D image conversion on 2.5D data to convert it into 2D data. The filtering process may be changed according to the type of measurement to be performed.

(Structure of display means 402)
The display means 402 is a means for controlling the display unit 5 which is a monitor of the setting device 4, and causes the display unit 5 to display, for example, a three-dimensional object generated by the object generation means 401. Further, the display means 402 can also display various viewpoint-specific elements generated by the element generation means 405, which will be described later, on the three-dimensional object by superimposing them on the display unit 5.

(Structure of posture adjusting means 403)
The posture adjusting means 403 is a means for adjusting the posture of the three-dimensional object displayed on the display unit 5 by the display means 402. The posture of the three-dimensional object can be adjusted by rotating the three-dimensional object around each of the X-axis, Y-axis, and Z-axis, or by moving the three-dimensional object in any direction of the X-direction, Y-direction, and Z-direction.

Specifically, the posture adjusting means 403 can detect, for example, the operating state of the keyboard 6 and the mouse 7, and clicks the mouse 7 on the three-dimensional object displayed on the display unit 5. By dragging or the like, the three-dimensional object can be rotated by an arbitrary angle around an arbitrary axis of the X axis, the Y axis, and the Z axis. Further, by operating the mouse 7 on the three-dimensional object, the three-dimensional object can be moved in any direction in the X direction, the Y direction, and the Z direction by an arbitrary amount of movement.

Further, by operating the keyboard 6 and the mouse 7, the rotation angle around each axis of the three-dimensional object can be set numerically, and the amount of movement in an arbitrary direction can be set numerically. The set numerical value is accepted by the posture adjusting means 403. The posture adjusting means 403 adjusts the posture of the three-dimensional object by rotating or moving the three-dimensional object so as to correspond to the received numerical value. The above-mentioned method for adjusting the posture of the three-dimensional object is an example, and the posture of the three-dimensional object may be adjusted by another method.

By adjusting the posture of the three-dimensional object, the user can display the three-dimensional object on the display unit 5 in a state of being viewed from a desired direction. This facilitates the operation of designating the target area, which will be described later.

(Specifying the target area at the time of setting)
The area designation receiving means 404 is a means for receiving the designation of the target area to be the target of the image inspection on the three-dimensional object whose posture is adjusted by the posture adjusting means 403. The target area is created by the user's operation as described later. For example, there is a linear area, an area surrounded by a figure, and the like, and if the area has a certain range, its shape and size are You are free. The linear region may be a linear region or a curved region. The curved shape may be, for example, a circle or an arc shape, or may be a free curve shape. As an example of the area surrounded by the figure, for example, the area surrounded by a circle, an ellipse, an arc, a polygon, or the like can be mentioned.

Since the user can see the three-dimensional object representing the work W in three dimensions from a desired direction, the height difference can be easily and surely grasped even in a small step-like portion existing in the work W. .. Since the target area can be specified while grasping the height difference of the work W, the user can easily grasp where and what size or shape the target area is specified on the work W.

The procedure for creating a new target area will be described with reference to the flowchart shown in FIG. This flowchart starts when the user performs an operation of starting a new creation of the target area. Examples of the operation for starting the new creation of the target area include pressing the start button (not shown).

In step SC1 after the start, the click operation of the mouse 7 by the user is detected, and the coordinates of the clicked point are acquired. In step SC1, the XY coordinates of the clicked point will be acquired.

After that, the process proceeds to step SC2, and a hit determination is made with the three-dimensional image data forming the three-dimensional object. Here, a 3D hit determination is performed to determine whether or not the XY coordinates acquired in step SC1 hit the three-dimensional image data. The 3D hit determination will be described with reference to FIG. When the point P1 is clicked on the display unit 5 to acquire the XY coordinates of the point P1, an extension line L1 extending in the Z direction from the point P1 is virtually generated in the three-dimensional space. The Z direction coincides with the image pickup axis direction of the camera 31. If the extension line L1 intersects with the work W1, it is determined that the hit point PH exists, and therefore a hit is determined. On the other hand, if the extension line L1 does not intersect with the work W1, it is determined that the hit point does not hit.

After going through step SC2 in FIG. 12, the process proceeds to step SC3. In step SC3, it is determined whether or not there is a hit point in step SC2. If there is no hit point, it goes to the end and the user clicks another place. If YES is determined in step SC3 and a hit point exists, the process proceeds to step SC4, the XY coordinates of the hit point PH are acquired, and the hit point is treated in the same manner as the two-dimensional display. That is, the point P1 is displayed at a position corresponding to the XY coordinates of the hit point PH in the display area of the display unit 5.

After step SC4, the process proceeds to step SC5, and it is determined whether or not the number of points is equal to or greater than a predetermined number. For example, when creating a linear target area, at least two points for determining a straight line are required, and in this case, it is determined in step SC5 whether or not the number of points is two. .. Further, for example, if three points are required when creating a triangular target area, an arc, or a circular target area, it is determined in step SC5 whether or not the number of points is three. The user can select the shape of the target area to be created before step SC1. The determination in step SC5 is changed according to the shape of the selected target area.

If NO is determined in step SC5 and the number of points is not more than the predetermined number, the process proceeds to step SC1 to accept the click operation of the next point, while YES is determined in step SC5 and the number of points is not more than the predetermined number. Since the target area can be specified in, the creation of the target area is completed. Assuming that the created target area is specified by the user, the process proceeds to step SC6. In step SC6, the target area created by the user as described above is added. In step SC7, the target area data added in step SC6 is saved. The target area data is data related to the shape, position, size, etc. of the target area.

(Creation of a linear target area)
Next, a specific example of creating a target area will be described using an example of a three-dimensional object. First, a specific example of creating a linear target area will be described with reference to FIG. On the upper side of FIG. 14, a three-dimensional object 200 indicating the work W is shown as being displayed on the display unit 5 by the display means 402, and on the lower side of FIG. 14, the display unit 402 displays the work W. A two-dimensional image (height image) 201 of the work W is shown as what is displayed. Such a two-dimensional image 201 can be generated by the acquisition unit A.

The display means 402 is configured so that the three-dimensional object 200 and the two-dimensional image 201 can be switched and displayed on the display unit 5. For example, by a switching operation or a selection operation by the user, only the three-dimensional object 200 can be displayed on the display unit 5, or only the two-dimensional image 201 can be displayed on the display unit 5. Further, both the three-dimensional object 200 and the two-dimensional image 201 can be displayed on the display unit 5. Examples of the switching operation or the selection operation include the operation of the switching button and the operation of the selection button.

When creating a linear target area, the user first selects "straight" from the selection of the shape of the target area. After that, when the pointer 7a of the mouse 7 is placed on the three-dimensional object 200, a line segment L2 extending upward (Z direction) from the tip of the pointer 7a is drawn. When the pointer 7a is moved, the line segment L2 also moves in the same manner. The extending direction of the line segment L2 is the direction in which the image pickup element 31d of the camera 31 is located, and extends by a predetermined length along the image pickup axis of the image pickup element 31d.

When the mouse 7 is pressed, the hit determination in step SC2 shown in FIG. 12 is performed based on the XY coordinates of the point P1 indicated by the pointer 7a at that time. As a result, if the point P1 hits the three-dimensional object 200, the point P1 is set, but if it does not hit, the point P1 is not set. This point P1 is the starting point. Further, when the mouse 7 is pressed, the color of the line segment L2 changes and the line segment L2 is fixed at the position of the point P1 so that the point P1 can be visually recognized. The press operation is to press the button of the mouse 7.

While the user presses the mouse 7, the mouse 7 is dragged to move the pointer 7a away from the point P1. During the drag operation, the line segment L4 extending upward from the tip of the pointer 7a is drawn separately from the line segment L2. After that, when the mouse 7 is released, the hit determination in step SC2 shown in FIG. 12 is performed based on the XY coordinates of the point P3 indicated by the pointer 7a at the time of the release operation. When the point P3 hits the three-dimensional object 200, the point P3 is set as the end point, the color of the line segment L4 changes, and the line segment L4 is fixed at the position of the point P3. If there is no hit, the point P3 is not set. The upper end of the line segment L4 and the upper end of the line segment L2 are at the same height.

When the point P3 is set, a line segment (upper XY direction line) L5 connecting the upper end of the line segment L4 and the upper end of the line segment L2 is drawn. A projection line L6 extending from the point P1 to the point P3 is drawn on the three-dimensional object 200 at a position where the line segment L5 is projected in the height direction. The projection line L6 is superimposed on the three-dimensional object 200 and has a shape along the three-dimensional object 200. The portion on the surface of the three-dimensional object 200 on which the projection line L6 is drawn is the target area.

That is, when the area designation receiving means 404 receives the designation of the start point (point P1) and the end point (point P3) on the three-dimensional object 200, the projection extending from the point P1 to the point P3 in the two-dimensional coordinates of the height data. The target area is set at the coordinates corresponding to the position of the line L6. By performing the operation of designating the target area on the three-dimensional object 200 in this way, it is possible to easily specify the side surface of the wall of the work W, which is difficult to specify on the two-dimensional image 201, as the target area.

The line segment L2 starting from the point P1 and the line segment L4 starting from the point P3 are parallel to each other, and both are line segments extending toward the image pickup element 31d along the image pickup axis of the image pickup element 31d. The user can specify the viewpoint at the position of the image pickup device 31d by referring to the line segment L2 and the line segment L4. The line segment L2 and the line segment L4 are included in the viewpoint specifying element generated by the element generation means 405. The element generation means 405 generates a line segment L2 and a line segment L4 as a viewpoint specifying element for specifying a viewpoint at the position of the image pickup element 31d when the area designation receiving means 404 receives the designation of the target area. It is a means of. Since the viewpoint specifying element generated by the element generation means 405 is displayed on the display unit 5 by the display means 402, the user can grasp the direction of the image pickup element 31d while creating the target area.

When the point P3 is set, the graphic element D1 is superimposed on the three-dimensional object 200 and displayed. The graphic element D1 is included in the viewpoint specifying element, and the graphic element D1 is also generated by the area designation receiving means 404 and displayed on the display unit 5 by the display means 402. The graphic element D1 has a larger area than the line, and in this example, it extends from the projection line L6 along a plane extending in the same direction as the extension direction of the line segment L2 and the line segment L4, that is, along the image pickup axis of the image pickup element 31d. It is composed of planes. Since the graphic element D1 is formed between the line segment L2 and the line segment L4 and both edges of the graphic element D1 are defined by the line segment L2 and the line segment L4, the graphic element D1 is a line segment L2 and a line. It is an element formed based on the line segment L4. Further, although the graphic element D1 is not essential, it is also drawn downward from the projection line L6.

The height of the upper end portion of the graphic element D1 is the same as the height of the upper end portion of the line segment L2 and the line segment L4. The lengths of the line segment L2 and the line segment L4 are determined according to the maximum height and the minimum height in the height image. That is, when determining the lengths of the line segment L2 and the line segment L4, the element generation means 405 first acquires the maximum height and the minimum height in the height image based on the height data. Next, the element generation means 405 increases the lengths of the line segment L2 and the line segment L4 as the difference between the acquired maximum height and the minimum height increases, and if the difference between the acquired maximum height and the minimum height is small, the element generation means 405 increases the length of the line segment L2 and the line segment L4. The smaller the length, the shorter the length of the line segment L2 and the line segment L4. Since the lengths of the line segment L2 and the line segment L4 vary depending on the height of the upper ends of the line segment L2 and the line segment L4, the element generation means 405 determines the line segment L2 and the line segment L2 according to the maximum height in the height image. It can also be said that the height of the upper end portion of the line segment L4 is determined. Similarly, the element generation means 405 can also determine the height of the upper end portion of the graphic element D1 according to the maximum height in the height image.

Since the lengths of the line segments L2 and L4 can be determined according to the maximum height and the minimum height in the height image, the line segments L2 and L4 are too short or too long with respect to the height of the work W. The line segment L2 and the line segment L4 are easy to see. Similarly, since the vertical dimension of the graphic element D1 is also determined, the graphic element D1 is also easy to see. Further, the upper limit and the lower limit of the lengths of the line segment L2 and the line segment L4 may be set in advance.

It is also possible to create a curved target area such as an arc. In this case, the graphic element D1 is composed of a curved surface. Further, the graphic element D1 may be composed of a plane and a curved surface.

The graphic element D1 can be made translucent. Translucency means that the other side of the graphic element D1 can be visually recognized through the graphic element D1. The transparency of the graphic element D1 may be low or high, and the transparency may be such that the contour of the work W located on the other side of the graphic element D1 displayed on the display unit 5 can be visually recognized by the user. Just do it. The graphic element D1 may be colored in a color different from the color of the work W, which facilitates the distinction between the graphic element D1 and the work W. The display form is such that the graphic element D1 and the line segments L2, L4, and L5 can be distinguished from each other. In this example, the graphic element D1 is translucent and the line segments L2, L4, and L5 are opaque, which makes it easy to distinguish between the two. In addition, the distinguishable display form may be a form in which the colors of each other are changed or the color depth is changed.

The viewpoint specifying element may be any one that displays the viewpoint of the work W viewed from the camera 31, and may include, for example, an arrow, a mesh-shaped graphic element, or the like. The graphic element D1 may be an element such as a transparent white wall.

The graphic element D1 does not have to be translucent. In this case, the other side cannot be visually recognized through the graphic element D1, but if the graphic element D1 is colored in a color different from the color of the work W, it can be distinguished from the work W.

When the points P1 and P3 are designated on the three-dimensional object 200, the points P1 and P3 are displayed on the two-dimensional image 201 and the points P1 and P3 are displayed as shown in the lower part of FIG. The line segment L5 connecting the above is drawn. The projection line L6 is located directly below the line segment L5. As a result, the target area can be grasped not only on the three-dimensional object 200 but also on the two-dimensional image 201.

Further, the designation of the target area may be accepted on the two-dimensional image 201. That is, when the point P1 and the point P3 are specified on the two-dimensional image 201, the line segment L5 is drawn on the two-dimensional image 201, the points P1 and the point P3 are displayed on the three-dimensional object 200, and the projection line L6 is displayed. And the graphic element D1 are also drawn. By designating the target area on the two-dimensional image 201 in this way, it is possible to easily specify the center of the hole of the work W or the back of the step, which is difficult to specify on the three-dimensional object 200, as the target area.

(Creating a rectangular target area)
Next, a specific example of creating a rectangular target area will be described with reference to FIG. First, the user selects "rectangular" from the selection of the shape of the target area. After that, when the pointer 7a of the mouse 7 is placed on the three-dimensional object 200, a line segment L2 extending upward from the tip of the pointer 7a is drawn as in the case of a straight line.

When the mouse 7 is pressed, the hit determination in step SC2 shown in FIG. 12 is performed based on the XY coordinates of the point P1 indicated by the pointer 7a at that time. As a result, when the point P1 hits the three-dimensional object 200, the point P1 is set.

While the mouse 7 is being pressed, the mouse 7 is dragged to move the pointer 7a away from the point P1. A line segment L4 extending upward from the tip of the pointer 7a is drawn even during the drag operation. After that, when the mouse 7 is released, the hit determination in step SC2 shown in FIG. 12 is performed based on the XY coordinates of the point P3 indicated by the pointer 7a at the time of the release operation. When the point P3 hits the three-dimensional object 200, the point P3 is set.

When the point P3 is set, a rectangular frame L7 having a virtual line (hidden) extending from the point P1 to the point P3 as a diagonal line is drawn. Further, line segments L10 and L11 extending from two other vertices of the rectangular frame L7 are drawn. The line segments L10 and L11 are parallel to the line segments L2 and L4, and the line segments L2, L4, L10 and L11 are arranged at the vertices of the rectangular frame L7, respectively.

Further, a projection line L8 is drawn on the three-dimensional object 200 at a position where the rectangular frame L7 is projected in the Z direction (height direction). The projection line L8 is superimposed on the three-dimensional object 200 and has a shape along the three-dimensional object 200. The portion surrounded by the projection line L8 on the surface of the three-dimensional object 200 is the target area.

When the point P3 is set, the graphic element D2 is displayed superimposed on the three-dimensional object 200. The graphic element D2 is included in the viewpoint specifying element. The graphic element D2 is composed of a combination of four planes extending from the projection line L8 in the same direction as the extension direction of the line segment L2 and the line segment L4. The four planes are the plane between the line segment L2 and the line segment L10, the plane between the line segment L2 and the line segment L11, the plane between the line segment L4 and the line segment L10, and the line segment L4 and the line segment. It is a plane between L11 and the graphic element D2 is an element formed based on the line segments L2, L4, L10, and L11. Further, although the graphic element D2 is not essential, it is also drawn downward from the projection line L8.

The target area may be highlighted. For example, the target area may be colored in a color different from that of other areas, or may be provided with a predetermined pattern.

(Change of target area)
Next, a case of changing the position, size, and shape of the target area designated as described above will be described with reference to FIGS. 16 and 17. FIG. 16 shows the first stage of the procedure for changing the target area, and proceeds to the second stage shown in FIG. 17 through the first stage. In the example described below, a case where the user changes the target area by using the mouse 7 will be described, but the target area can also be changed by operating a touch operation panel or the like other than the mouse 7.

In step SD1 of the flowchart shown in FIG. 16, the coordinates (XY coordinates) of the pointer at the time when the user clicks the mouse 7 are acquired. After that, in step SD2, a 3D hit determination of a line extending in the Z direction is performed by fixing the XY coordinates from the point specified by the coordinates acquired in step SD1. The method of 3D hit determination is the same as step SC2 in the flowchart of FIG. 12, and it is determined whether or not the line extending in the Z direction hits the three-dimensional image data.

After going through step SD2, the process proceeds to step SD3. In step SD3, it is determined whether or not there is a hit in step SD2. If it doesn't hit, it goes to the end and the user clicks on another part. If YES is determined in step SD3 and a hit is made, the process proceeds to step SD4. In step SD4, a two-step hit determination, that is, a 3D hit determination and then a 2D hit determination is performed.

FIG. 18A is a conceptual diagram of 3D hit determination. The straight line L20 is a line extending in the Z direction with the XY coordinates fixed from the point specified by the coordinates acquired in step SD1, the plane D3 is the Z direction plane, and the plane D4 is the XY direction plane. First, a 3D hit determination with the upper end of the plane D4 is performed. That is, although there are various types of lines in the target area and they are complicated, they are limited to the XY direction lines, and the X coordinates to be drawn are also fixed. Utilizing this, a 3D hit determination with the plane D3 of each Z coordinate is performed, and the XY coordinates of the hit point obtained by the hit determination are acquired. After that, as shown in FIG. 18B, the 2D hit determination is performed using the XY coordinates acquired in the 3D hit determination. This makes it possible to easily implement 3D hit determination.

After going through step SD4 shown in FIG. 16, the process proceeds to step SD5. In step SD5, it is determined whether or not there is a hit in step SD4. If it doesn't hit, it goes to the end and the user clicks on another part. If YES is determined in step SD5 and a hit is made, the process proceeds to step SD6.

In step SD6, a 3D hit determination is performed on the lower end of the XY direction plane in the same manner as in step SD4, and then a 2D hit determination is performed. After going through step SD6, the process proceeds to step SD7. In step SD7, it is determined whether or not there is a hit in step SD6. If it doesn't hit, it goes to the end and the user clicks on another part. If YES is determined in step SD6 and a hit is made, the process proceeds to step SD8. In step SD8, the XYZ coordinates of the hit point are saved. The above steps are automatically executed by the user's click operation.

The target area can be moved by moving (drag operation) the mouse 7 while the user performs the above-mentioned click operation. When the drag operation is performed, the process proceeds to step SE1 of the flowchart shown in FIG. In step SE1, the coordinates (XY coordinates) of the pointer moved by the drag operation are acquired. After that, the process proceeds to step SE2, and a 3D hit determination is performed with the Z coordinate plane of the XYZ coordinates of the hit point saved in step SD8 of the flowchart shown in FIG.

After passing through step SE2, the process proceeds to step SE3. In step SE3, it is determined whether or not there is a hit in step SE2. If there is no hit, it goes to the end and the user drags to another place. If YES is determined in step SE2 and a hit is made, the process proceeds to step SE4.

In step SE4, the viewpoint specific element and the target area are XY translated to the hit coordinates. That is, if a hit is made at the coordinates of the pointer after the drag operation, the target area and the viewpoint specifying element (line segments L2, L4, L10, L11, graphic elements D1 and D2 in FIG. 14) are translated to that coordinate. Area and viewpoint specific elements move in near real time to follow the pointer. As a result, the user can adjust the position of the target area on the three-dimensional object so that the target area is arranged at a desired position.

Further, in step SE4, the viewpoint specifying element is allowed to move only in parallel in the X direction or the Y direction, and the movement in the Z direction (height direction) is prohibited. That is, the position of the viewpoint specific element in the Z direction does not change before and after the movement. Since the viewpoint specific element is an element displayed when the target area to be inspected is set to the height data arranged on the two-dimensional coordinates, it is in the X direction or the Y direction of the three-dimensional object. It is sufficient to move it, and it is not necessary to move it in the direction in which the viewpoint is located (the height direction of the 3D object). If the viewpoint specific element can be moved in the height direction of the 3D object, the setting operation of the target area is performed. May become complicated. In the present embodiment, since the viewpoint specifying element can be moved only in the X direction or the Y direction of the three-dimensional object, the setting operation of the target area becomes easy.

In step SE4, the target area is only translated, and the determination of the position of the target area has not been performed yet. After step SE4, the process proceeds to step SE5, and it is determined whether or not the confirmation operation has been performed by the user. The confirmation operation may be, for example, an operation of releasing the button of the mouse 7 (release operation) or the like, but may be another operation.

If NO is determined in step SE5 and the confirmation operation is not performed, the process returns to step SE1, while if YES is determined in step SE5 and the confirmation operation is performed, the process proceeds to step SE6 and translation is performed. Change to the target area later. In step SE7, the target area data changed in step SE6 is saved.

19 and 20 show specific examples 1 and 2 in the case of adjusting the position of the linear target area. The figure on the left side of FIG. 19 according to the specific example 1 shows before the position adjustment of the target area, and the figure on the right side shows after the position adjustment of the target area. In the figure on the left side of FIG. 19, the pointer 7a of the mouse 7 is moved onto the line segment L5 (after the mouse hover), and then the press operation is performed. As a result, the colors of the line segment L2 and the line segment L4 are changed so that the user can know that the line segment L2 and the line segment L4 have been selected.

After that, when the mouse 7 is dragged, the line segment L2, the line segment L4, the line segment L5, the graphic element D1, and the projection line L6 move together along the movement locus of the pointer 7a, but the movement is prohibited in the Z direction. It is XY translation. The user may place the pointer 7a on any of the line segment L2, the line segment L4, the line segment L5, the graphic element D1, and the projection line L6 and drag the operation. When the line segment L5 is dragged, the line segment L2, the line segment L4, the line segment L5, the graphic element D1, and the projection line L6 are translated XY so that the line segment L5 is located on the pointer 7a even at the destination. Let me. Further, when the projection line L6 is dragged, the line segment L2, the line segment L4, the line segment L5, the graphic element D1, and the projection line L6 are XY so that the projection line L6 is located on the pointer 7a even at the movement destination. Move in parallel. Further, when the line segment L2 or the line segment L4 is dragged, the line segment L2, the line segment L4, the line segment L5, and the figure are arranged so that the line segment L2 or the line segment L4 is located on the pointer 7a even at the destination. The element D1 and the projection line L6 are translated XY. Further, when a certain point of the graphic element D1 is dragged, the line segment L2, the line segment L4, the line segment L5, and the graphic element D1 so that the same point of the graphic element D1 is located on the pointer 7a even at the destination. , The projection line L6 is translated XY.

The line segment L2, the line segment L4, the line segment L5, the graphic element D1, and the projection line L6 can be drawn even during the movement. Next, when the user performs a release operation, the line segment L2, the line segment L4, the line segment L5, the graphic element D1, and the projection line L6 are fixed in a state after being translated XY.

The figure on the left side of FIG. 20 according to the specific example 2 shows before the position adjustment of the target area, and the figure on the right side shows after the position adjustment of the target area. In the figure on the left side of FIG. 20, the pointer 7a of the mouse 7 is moved onto the line segment L4 (after the mouse hover), and then the press operation is performed. As a result, the colors of the line segment L2 and the line segment L4 are changed so that the user can know that the line segment L2 and the line segment L4 have been selected.

After that, when the mouse 7 is dragged, the line segment L2 moves in XY translation while the line segment L2 is fixed. The line segment L5 and the projection line L6 are drawn so as to extend from the line segment L2 to the moved line segment L4. Further, the graphic element D1 is also drawn according to the movement of the line segment L4. Next, when the user performs a release operation, the line segment L4 is fixed in the state after being translated XY. As a result, not only the position of the target area can be changed, but also the length of the target area can be changed.

FIG. 21 is a diagram showing a specific example of adjusting the position of the rectangular target area. The figure on the left side of FIG. 21 shows before the position adjustment of the target area, and the figure on the right side shows after the position adjustment of the target area. In the figure on the left side of FIG. 21, the pointer 7a of the mouse 7 is moved onto, for example, the rectangular frame L7 (after the mouse hover), and then the press operation is performed. As a result, the colors of the line segment L2, the line segment L4, the line segment L10, and the line segment L11 change, and it can be seen that the line segment L2 has been selected by the user. In addition, the intermediate line L9 is also drawn by the press operation. The intermediate line L9 is between the line segment L2 and the line segment L11, between the line segment L2 and the line segment L10, between the line segment L4 and the line segment L10, and between the line segment L4 and the line segment L11, respectively. It is drawn and extends in the Z direction. By dragging the handle of the intermediate line L9 (the handle of the intermediate point) with the mouse 7, only one of the width and the height of the rectangular frame L7 can be changed. The vertex of the rectangular frame L7 also has a handle, and by dragging the handle of this vertex with the mouse 7, both the width and the height of the rectangular frame L7 can be changed at the same time. When the user performs a release operation, the width and height of the rectangular frame L7 are fixed in a changed state. The position of the handle can be set arbitrarily.

(Drawing of effective measurement area)
A part of the work W may be an area that you do not want to measure. The process of removing the area from the measurement target can be called a mask process, and the masked area can be called a mask area. An area other than the mask area can be called an effective measurement area.

By drawing the effective measurement area on the three-dimensional object, the user can easily distinguish between the mask area and the effective measurement area. When drawing an effective measurement area on a three-dimensional object, the following drawing flow can be applied. First, a binary image is created internally, and the inside of the normal area is filled with "1" or the like. After that, the inside of the mask area is filled with "0" to erase the part covered with the normal area. Next, a boundary image is generated from the binary image. The wall surface is semi-transparently drawn three-dimensionally on the boundary part of the boundary image. This reduces the number of semi-transparent objects drawn and makes it easier to see because it can be expressed in a single color.

(Step measurement procedure during operation)
FIG. 22 is a flowchart showing an example of a procedure for measuring a step in a linear target area during operation of the image inspection system 1. Before the step measurement, the user sets the threshold value of the step height in advance. In step SF1 after the start, 2.5D data of the work W is acquired based on the height data newly acquired by the acquisition unit A. At this time, 2D data may be acquired at the same time. In step SF2, the linear target area data saved in step SC6 shown in FIG. 12 and step SF7 shown in FIG. 7 is set in the 2.5D data acquired in step SF1. This step SF2 is a step executed by the area setting means 406 shown in FIG. 11, and by passing through this step, the height of the target area designated by the area designation receiving means 404 is acquired by the acquisition unit A. Can be set for data.

In step SF3, the height profile on the target area set by the area setting means 406 in step SF2 is acquired. In step SF4, the height on the upper side and the height on the lower side of the step are measured in the height profile acquired in step SF3. In step SF5, the step height is calculated from the difference between the height on the upper side and the height on the lower side. Steps SF3 to SF5 are steps in which the image processing unit 4B shown in FIG. 7 performs image processing on the target area set by the area setting means 406. In step SF6, the threshold range set in advance is compared with the step height calculated in step SF5, and if the step height calculated in step SF5 falls within the threshold range, it is determined as "good" and if it does not fit. Is judged to be "defective". This step SF6 is a step executed by the inspection unit 4A shown in FIG. 7.

In step SF7, the setting device 4 outputs the determination value of step SF6 to the external device. When the external device is a defective product discharging mechanism, the defective product is discharged from the transport belt conveyor B by the discharging mechanism in step SF8.

(Scratch inspection procedure during operation)
FIG. 23 is a flowchart showing an example of a procedure for inspecting a scratch in a rectangular target area during operation of the image inspection system 1. Before the scratch inspection, the user sets a threshold value for the amount of scratches in advance. In step SG1 after the start, the acquisition unit A newly acquires 2D data. At this time, 2.5D data may be acquired. In step SG2, rectangular target area data is set in the 2D data acquired in step SG1. This step SG2 is a step executed by the area setting means 406 shown in FIG.

In step SG3, the scratch amount in the target area set by the area setting means 406 in step SG2 is acquired by using the scratch measurement algorithm. The scratch measurement algorithm may be a conventional one. This step SG3 is a step in which the image processing unit 4B shown in FIG. 7 performs image processing on the target area set by the area setting means 406.

In step SG4, the preset threshold range is compared with the scratch amount acquired in step SG3, and if the scratch amount acquired in step SG3 falls within the threshold range, it is determined as "good", and if it does not fit, it is determined. Is determined to be "defective". This step SG4 is a step executed by the inspection unit 4A shown in FIG. 7.

In step SG5, the setting device 4 outputs the determination value of step SG4 to the external device. When the external device is a defective product discharging mechanism, the defective product is discharged from the transport belt conveyor B by the discharging mechanism in step SG6.

(Height inspection procedure during operation)
FIG. 24 is a flowchart showing an example of a procedure for height inspection in a rectangular target area during operation of the image inspection system 1. Before the height inspection, the user sets a height threshold value in advance. In step SH1 after the start, 2.5D data is acquired in the same manner as in step SF1 of FIG. In step SH2, rectangular target area data is set in the 2.5D data acquired in step SH1. This step SH2 is a step executed by the area setting means 406 shown in FIG.

In step SH3, the average height in the target area set by the area setting means 406 in step SH2 is calculated. This step SH3 is a step in which the image processing unit 4B shown in FIG. 7 performs image processing on the target area set by the area setting means 406. In step SH4, the preset threshold range is compared with the average height acquired in step SH3, and if the average height acquired in step SH3 falls within the threshold range, it is determined as "good", and if it does not fit. Is judged to be "defective". This step SH4 is a step executed by the inspection unit 4A shown in FIG. 7.

In step SH5, the setting device 4 outputs the determination value of step SH4 to the external device. When the external device is a defective product discharging mechanism, the defective product is discharged from the transport belt conveyor B by the discharging mechanism in step SH6.

(Expiration date inspection procedure at the time of operation)
FIG. 25 is a flowchart showing an example of a procedure for inspecting the expiration date in an arcuate target area during operation of the image inspection system 1. Before the expiration date inspection, the user sets a threshold value for the date and time in advance. In step SI1 after the start, 2D data is acquired by the acquisition unit A. At this time, 2.5D data may be acquired. In step SI2, arcuate target area data is set in the 2D data acquired in step SI1. This step SI2 is a step executed by the area setting means 406 shown in FIG.

In step SI3, characters in the target area set by the area setting means 406 in step SI2 are cut out. In step SI4, the character cut out in step SI3 is determined by an optical character recognition (OCR) algorithm. In step SI5, characters are converted into date and time. Steps SI3 to SI5 are steps in which the image processing unit 4B shown in FIG. 7 performs image processing on the target area set by the area setting means 406.

In step SI6, the preset threshold range is compared with the date and time converted in step SI5, and if the date and time converted in step SI5 falls within the threshold range, it is determined as "good", and if it does not fit, "good" is determined. It is judged as "defective". This step SI6 is a step executed by the inspection unit 4A shown in FIG. 7.

In step SI7, the setting device 4 outputs the determination value of step SI6 to the external device. When the external device is a defective product discharging mechanism, the defective product is discharged from the transport belt conveyor B by the discharging mechanism in step SI8.

The expiration date is an example, and for example, the serial number, the date of manufacture, and the like can be inspected.

(Procedure for discriminating stamps during operation)
FIG. 26 is a flowchart showing an example of a procedure for marking and discriminating in an arcuate target area during operation of the image inspection system 1. Before the marking determination, the user sets in advance the characters to be stamped on the work W. In step SJ1 after the start, 2.5D data is acquired in the same manner as in step SF1 of FIG. In step SJ2, the 2.5D data acquired in step SJ1 is converted into 2D data by the height extraction algorithm. In step SJ3, the arc-shaped target area data is set in the 2D generated in step SJ2. This step SJ3 is a step executed by the area setting means 406 shown in FIG.

In step SJ4, characters in the target area set by the area setting means 406 in step SJ3 are cut out. In step SJ5, the character cut out in step SJ4 is determined by an optical character recognition (OCR) algorithm. The steps SJ4 and SJ5 are steps in which the image processing unit 4B shown in FIG. 7 performs image processing on the target area set by the area setting means 406.

In step SJ6, the preset characters are compared with the characters determined in step SJ5, and if the characters converted in step SJ5 are the same as the measured characters, it is determined as "good", and if they are different, it is determined as "bad". ". This step SJ6 is a step executed by the inspection unit 4A shown in FIG. 7.

In step SJ7, the setting device 4 outputs the determination value of step SJ6 to the external device. When the external device is a defective product discharging mechanism, the defective product is discharged from the transport belt conveyor B by the discharging mechanism in step SJ8.
(Action and effect of the embodiment)
As described above, according to this embodiment, when the acquisition unit A acquires the height data in which the height information of the work W is arranged on the two-dimensional coordinates at the time of setting, the object is based on the height data. The generation means 401 creates a three-dimensional object. Since the posture of the generated three-dimensional object can be adjusted while being displayed on the display unit 5 of the setting device 4 in a form such as a three-dimensional polygon display or a three-dimensional point cloud display, the work W can be moved in a desired direction. The three-dimensional object seen from the above can be displayed on the display unit 5. The user can specify the target area on the three-dimensional object after the posture adjustment.

In this way, when the target area is specified, the user can see the three-dimensional object that represents the work W in three dimensions from a desired direction, so that the height difference can be easily obtained even in a small step-like portion. And you can surely grasp it. Since the target area can be specified while grasping the height difference of the work W, the user can easily grasp where and what size or shape the target area is specified on the work W.

When the target area is specified, the target area is set for the newly acquired height data at the time of operation. That is, since the information of the target area designated on the three-dimensional object is reflected in the height data at the time of operation, the image inspection is executed for the desired area.

The above embodiments are merely exemplary in all respects and should not be construed in a limited way. Further, all modifications and modifications belonging to the equivalent scope of the claims are within the scope of the present invention.

As described above, the present invention can be used, for example, when inspecting a work or the like.

1 Image inspection system 2 Lighting device (light projector)
4A Inspection unit 4B Image processing unit 5 Display unit (monitor)
31 Camera (image generator)
31d Image sensor 300 Image sensor 400 Inspection setting unit 401 Object generation means 402 Display means 403 Posture adjustment means 404 Area designation reception means 405 Element generation means 406 Area setting means A Acquisition unit D1 Graphic element L2, L4 Line segment (viewpoint identification element)
W work (object to be inspected)

Claims (12)

An image inspection imager provided with an acquisition unit for acquiring height data in which height information of an inspection object is arranged on two-dimensional coordinates, and a setting device connected to the image inspection imager via a network. An image inspection system that includes an inspection setting unit that sets various inspection settings based on user operations.
The inspection setting unit
An object generation means for generating a three-dimensional object that three-dimensionally represents an inspection object based on the height data acquired by the acquisition unit.
A display means for displaying a three-dimensional object generated by the object generation means on the monitor of the setting device, and a display means.
A posture adjusting means for adjusting the posture of a three-dimensional object displayed by the display means, and a posture adjusting means.
An area designation receiving means that accepts the designation of the target area to be the target of the image inspection on the three-dimensional object whose posture is adjusted by the posture adjusting means.
An area setting means for setting a target area designated by the area designation receiving means with respect to the height data acquired by the acquisition unit, and an area setting means.
Image inspection system with. In the image inspection system according to claim 1,
The acquisition unit includes an image pickup element that captures an image of an inspection object.
The inspection setting unit
An element generation means for generating a viewpoint specifying element for specifying a viewpoint at a position of the image pickup element when the designation of a target area is received by the area designation receiving means is provided.
The display means is an image inspection system in which a viewpoint specifying element generated by the element generation means is superimposed on the three-dimensional object and displayed on the monitor. In the image inspection system according to claim 2,
The viewpoint specifying element is formed based on a plurality of line segments extending by a predetermined length from each of the plurality of points designated by the area designation receiving means along the image pickup axis of the image pickup device, and the plurality of line segments. An image inspection system that includes a graphic element consisting of a flat surface or a curved surface. In the image inspection system according to claim 3,
The graphic element is a translucent image inspection system. In the image inspection system according to claim 3 or 4.
An image inspection system in which a predetermined length of the plurality of line segments forming the graphic element is determined according to a maximum height or a minimum height in the height image. In the image inspection system according to any one of claims 2 to 5.
The viewpoint specifying element is an image inspection system whose position can be adjusted on the three-dimensional object after being generated by the element generation means. In the image inspection system according to any one of claims 2 to 6.
The viewpoint specifying element is an image inspection system whose shape can be changed on the three-dimensional object after being generated by the element generation means. In the image inspection system according to claim 6,
The viewpoint specifying element is an image inspection system in which movement of the three-dimensional object in the height direction is prohibited. In the image inspection system according to any one of claims 1 to 8.
When the area designation receiving means receives the designation of the start point and the end point on the three-dimensional object, the target area is the coordinate corresponding to the position of the line segment extending from the start point to the end point in the two-dimensional coordinates of the height data. Image inspection system to set. In the image inspection system according to any one of claims 1 to 9.
The acquisition unit generates a height image based on the acquired height data, and generates a height image.
The display means is configured so that the height image generated by the acquisition unit and the three-dimensional object generated by the object generation means can be switched and displayed on the monitor.
The area designation receiving means is an image inspection system that receives designation of a target area to be image-inspected on a height image generated by the acquisition unit. In the image inspection system according to any one of claims 1 to 10.
The setting device performs an visual inspection based on an image processing unit that performs image processing on a target area set for height data by the area setting means and an image that has been image-processed by the image processing unit. An image inspection system equipped with an inspection unit. In the image inspection system according to any one of claims 1 to 11.
The setting device externally acquires a file in which a setting item consisting of a target area received by the area designation receiving means and register information indicating a place where a setting value of the setting item is stored are stored by a user. It is configured so that the set value of the set item and the register information corresponding to the set item included in the file can be transmitted to the image inspection imaging apparatus.
The image inspection imaging device is an image inspection system that stores the setting values of the setting items received from the setting device in a place indicated by register information corresponding to the setting items.

Images