JP7402745B2 - Imaging device and image inspection system - Google Patents

Description

The present invention relates to, for example, an imaging device that images a workpiece or the like, and an image inspection system that inspects the imaged workpiece or the like.

2. Description of the Related Art Conventionally, as disclosed in Patent Document 1, for example, there has been known an image processing apparatus that uses a pattern projection method to obtain height information of an object to be inspected, such as a workpiece. The image processing device disclosed in Patent Document 1 acquires a plurality of brightness images with a camera while sequentially irradiating a plurality of pattern projection lights onto an object to be inspected, and sends the plurality of brightness images acquired with the camera to a controller. Then, height data is generated in the controller. The controller executes a process of converting height data within a predetermined height range out of the generated height data into a height image with a lower number of gradations than the height data, that is, a height extraction process. After the height extraction process, the controller performs image processing using the height image to determine the quality of the inspection object, and outputs a determination signal to an external device such as a PLC.

JP2019-168285A

By the way, a setting device constituted by a general-purpose personal computer or the like is connected to the controller, and there is a user need to perform image processing using a height image with this setting device. In such cases, conventionally, various processes have been performed on the setting device on the height data output from the controller.

However, if only a two-dimensional image processing library that handles height images is installed in the setting device, the setting device cannot process the height data sent from the controller.

Regarding this point, for example, it is possible to write a program code to convert height data to a height image in the setting device, but writing a program code to convert height data to a height image is difficult. It is difficult to write appropriate code unless you are an expert. Even if the code could be written, it would take time and effort to write it. In particular, when the object to be inspected is tilted from the horizontal plane, it is also necessary to write code to correct the tilt, making it even more difficult.

The present disclosure has been made in view of this point, and its purpose is to enable a setting device connected to an imaging device to easily obtain a height image suitable for image processing from the imaging device and use the setting device to heighten the image. The object of the present invention is to enable image processing using a digital image, and to improve usability for the user.

In order to achieve the above object, in a first disclosure, an imaging device acquires a brightness image according to the amount of reflected light received from an object to be imaged, and also displays height information of the object to be imaged on two-dimensional coordinates. an acquisition unit that acquires height data arranged in the acquisition unit; and a reference position for specifying a reference plane that is a reference for gradation conversion of the height data acquired by the acquisition unit. and a setting unit that sets the reference position, and the acquisition unit acquires a new brightness image and new height data for a new imaging target different from the imaging target from which the height data used for setting the reference position was acquired. a correction unit that searches for a characteristic part of the new luminance image to detect a shift in the position or posture of the new imaged object, and corrects the shift in the reference position with respect to the new height data; , a calculation unit that calculates a reference plane of the new height data using the height information corresponding to the reference position corrected by the correction unit; a conversion unit that converts new height data into a height image having a lower number of gradations than the number of gradations of the new height data; and a standardized standard for converting the height image generated by the conversion unit. The configuration may include an output unit that outputs an output.

According to this configuration, when the brightness image and height data of the imaging target are acquired by the acquisition unit, a reference position can be set for the height data. The reference position is information for specifying the reference plane, and the information may be, for example, a predetermined area set to surround a flat part of the object to be imaged, or a flat area of the object to be imaged. It may also be a point indicating a part. A reference plane is specified based on the set reference position. For example, if a predetermined area is set, a plane specified by the average height of multiple points within the area can be used as the reference plane, and if points are set, A plane including that point can be used as a reference plane. If there is one point set, the plane containing that one point can be used as the reference plane, and if there are three points, the plane specified by the height of the three points can be used as the reference plane. can.

For example, when the imaging target for which height data for setting the reference position has been acquired is used as the setting imaging target, the acquisition unit creates a new brightness image of a new imaging target different from the setting imaging target. When new height data is acquired, the correction unit searches for a characteristic part of the new luminance image. The search by the correction unit may be a pattern search based on the degree of correlation with registered images registered in advance, or may be a method of extracting edges of the imaged object without using registered images. As a result of the search, when a shift in the position or orientation of the new imaging target with respect to the setting imaging target is detected, the shift in the reference position with respect to the new height data is corrected.

After correcting the reference position, a new reference plane for height data is calculated using the height information corresponding to the reference position, and the new height data is calculated based on the reference plane for the new height data. is converted to a lower height image. For example, if the number of gradations of the height data is 16 bits, the height image can be converted to the number of gradations of 8 bits. The generated height image is a two-dimensional image that can be handled by a two-dimensional image processing library.

In addition, since the generated height image is output according to standardized standards, when the imaging device is connected to a setting device such as a general-purpose personal computer, the height image can be easily received by the setting device. can. Furthermore, since the received height image can be handled by the two-dimensional image processing library as described above, the user does not have to write program code for converting it into a two-dimensional image, and the setting device can process the image. It becomes possible to perform processing easily, and time and effort for writing are unnecessary.

For example, even if the object to be imaged is tilted from the horizontal plane, the reference plane is calculated after the deviation of the reference position is corrected as described above, and the number of low gradations is adjusted based on the calculated reference plane. Since the image is converted to a straight image, there is no need for the user to write code to correct the tilt. This also improves usability for the user.

In a second disclosure, the imaging device acquires from the outside a file in which a setting item including the reference position and register information indicating a location where the setting value of the setting item is stored is set by the user. The configuration can be configured to be connectable via a network to a setting device capable of transmitting the set value of the set item and register information included in the file corresponding to the set item to the imaging device. Furthermore, the imaging device can store the setting value of the setting item received from the setting device in a location indicated by register information corresponding to the setting item.

According to this configuration, if both the imaging device and the setting device connected via the network conform to a common standardization standard, the setting item and the location where the setting value of the setting item is stored are indicated. The setting device obtains a file in which register information is written. This file is, for example, a Device XML file when the standardization standard is the GenICam standard. This file may be acquired by the configuration device from the imaging device, or may be downloaded from a website or the like and designated as a separate file to GenApi for reference.

For example, when a user sets a reference position, the coordinates of the reference position become set values, and a setting item called the reference position and its coordinates are transmitted to the imaging device together with register information. As a result, settings related to the reference position of the imaging device are performed, so if the imaging device complies with the standardization specifications, the above settings can be changed freely using the setting device, increasing the degree of freedom in selecting the model of the imaging device. .

In a third disclosure, the acquisition unit includes a light projecting unit that projects structured illumination having a predetermined projection pattern onto the imaging target, and a luminance image according to the amount of reflected light received from the imaging target. and an image generation section that generates a plurality of pattern projection images by receiving reflected light reflected by the structured illumination projected by the light projection section on the imaging target object, and the image generation section generates a plurality of pattern projection images. The height data can be configured to be acquired based on a plurality of pattern projection images generated by the unit.

In other words, methods for acquiring height data include passive methods such as stereo measurement, active methods such as light cutting method and pattern projection method, and any of these methods may be used. Accurate height data can be obtained by using a pattern projection method that projects structured illumination and captures images.

A fourth disclosure is to obtain from an external file that is connected to the imaging device via a network and describes setting items of the imaging device and register information indicating a location where setting values of each setting item are stored. , a setting device that transmits data indicating the setting values of each setting item set by the user and register information corresponding to each setting item included in the file to the imaging device, and configures the imaging device. The image inspection system can be used as a prerequisite.

The setting device includes a display section that displays height information of the new height data, and a height range setting section that sets a predetermined height range using the height information displayed on the display section. It is equipped with The conversion unit converts the new height data into new height data based on the reference plane calculated by the calculation unit and the predetermined height range set by the height range setting unit. The height image can be configured to be converted into a height image having a lower number of gradations than the number of gradations.

According to this configuration, the height information of the acquired new height data is displayed on the display section. By using the height information displayed on the display unit, the user can set the height range that he/she wishes to express as a height image. For example, if expressing the lower part of the object to be imaged in the height image has no meaning in terms of image inspection, the height range can be a range excluding the lower part of the object to be imaged. Thereby, image inspection can be performed using a height image that includes only the height range necessary for image inspection, and it is possible to improve inspection speed and inspection accuracy.

In a fifth disclosure, the setting device calculates a frequency distribution for the height of each pixel included in the height information of the new height data, using the reference plane calculated by the calculation unit as a reference. the display unit displays the frequency distribution calculated by the frequency distribution calculation unit, and the height range setting unit sets a predetermined height on the frequency distribution displayed on the display unit. It may be possible to set a range.

According to this configuration, the frequency distribution of the height of each pixel is displayed on the display, so the user can easily determine, for example, a height range where there are few pixels and a height range where there are many pixels. . For example, a height range where there are few pixels can be excluded from the height range displayed in the height image, or a height image can be generated by narrowing the height range to a height range where there are many pixels.

In a sixth disclosure, the acquisition unit can generate a three-dimensional image of the imaging target based on height data. Further, the display unit may be configured to display the three-dimensional image generated by the acquisition unit and to display the height range set by the height range setting unit on the three-dimensional image. .

According to this configuration, since the height range is shown on the three-dimensional image displayed on the display unit, it can be confirmed whether the height range has been set as intended by the user. For example, lines or planes indicating the height range can be displayed superimposed on the three-dimensional image.

In a seventh disclosure, the display unit may be configured to display the height range set by the height range setting unit on a frequency distribution.

According to this configuration, since the height range is shown on the frequency distribution displayed on the display unit, it can be confirmed whether the height range is set as intended by the user. For example, a line indicating the height range can be displayed superimposed on the frequency distribution.

In an eighth disclosure, the setting device includes an image processing unit that performs image processing on the height image output from the output unit, and performs an appearance inspection based on the image processed by the image processing unit. and an inspection section.

As described above, according to the present disclosure, the setting device can easily acquire a height image suitable for image processing from the imaging device, and can perform image processing using the acquired height image. This eliminates the need to write program code to convert height data to a height image, or to correct the inclination of the object to be imaged during conversion, thereby improving usability for the user.

1 is a diagram schematically showing a schematic configuration and operational state of an image inspection system according to Embodiment 1 of the present invention. 2 is a diagram corresponding to FIG. 1 according to Embodiment 2. FIG. 2 is a diagram corresponding to FIG. 1 according to Embodiment 3. FIG. 2 is a diagram corresponding to FIG. 1 according to Embodiment 4. FIG. It is a flowchart which shows the procedure of generating a height image based on the principle of deflectometry. FIG. 2 is a diagram corresponding to FIG. 1 according to a fifth embodiment of generating a height image using a photometric stereo method. FIG. 2 is a diagram illustrating a connection interface between an image inspection application and a camera. 2 is a flowchart conceptually illustrating an example of internal processing of the imaging device. FIG. 3 is a diagram showing a parameter set when generating an inspection image using the principle of deflectometry. FIG. 3 is a diagram showing a parameter set when generating an inspection image by multispectral imaging. FIG. 3 is a block diagram of an imaging control section. 12 is a flowchart illustrating an example of a procedure for height extraction at the time of setting. FIG. 3 is a diagram showing an example of a 2D image conversion setting dialog. FIG. 6 is a diagram illustrating an example of displaying an image of a workpiece viewed from the side in a 2D image conversion setting dialog. FIG. 14 is a diagram corresponding to FIG. 14 showing a case where the height extraction range is narrowed; 7 is a flowchart illustrating an example of a position correction setting procedure. FIG. 3 is a diagram showing an example of a pattern search setting dialog. 12 is a flowchart illustrating an example of a procedure for height extraction during operation. 12 is a flowchart showing an example of a procedure for specifying a zero plane during operation. It is a flowchart which shows an example of the procedure of taking a three-dimensional difference during operation.

Hereinafter, embodiments of the present invention will be described in detail based on the drawings. Note that the following description of preferred embodiments is essentially just an example, and is not intended to limit the present invention, its applications, or its uses.

(Embodiment 1)
FIG. 1 schematically shows the operating state of an image inspection system 1 according to an embodiment of the present invention. The image inspection system 1 is configured to inspect the presence or absence of defects in the workpiece W using an image taken of the workpiece W, which is an object to be inspected. and a setting device 4 configured with a personal computer (PC) or the like that performs settings for the imaging section 3. Since the workpiece W is an object to be imaged by the imaging unit 3, it can also be called an imaged object. The illumination device 2 and the imaging section 3 constitute an acquisition section A that acquires height data. The imaging device 300 for image testing that constitutes a part of the image testing system 1 is a device that includes an acquisition unit A. Further, the imaging device 300 and the setting device 4 may be separated, and the imaging device 300 alone may be an embodiment of the present invention.

The setting device 4 includes a display section 5, a keyboard 6, and a mouse 7. The display unit 5 includes a monitor made of, for example, an organic EL display or a liquid crystal display, and displays images captured by the imaging device 300, images obtained by performing various processing on the images captured by the imaging device 300, and various users. This is a part that can display an interface image (GUI), etc. Various user interface images and the like are generated by the main body of the setting device 4. The horizontal direction of the display section 5 can be the X direction of the display section 5, and the vertical direction of the display section 5 can be the Y direction of the display section 5.

Further, the keyboard 6 and mouse 7 are conventionally known devices for operating a computer. By operating the keyboard 6 or mouse 7, various information can be input into the setting device 4, various settings can be made, and images and the like displayed on the display section 5 can be selected. Note that instead of or in addition to the keyboard 6 and mouse 7, it is also possible to use a device for operating the computer, such as a voice input device or a pressure-sensitive touch operation panel.

FIG. 1 shows a case where a plurality of workpieces W are placed on the upper surface of a conveyor belt B and are being conveyed in the direction indicated by the white arrow in FIG. An external control device 8 such as a programmable logic controller (PLC) is connected to the setting device 4 as a device for sequentially controlling the conveyor belt B and the image inspection system 1. This external control device 8 may constitute a part of the image inspection system 1. Further, the external control device 8 does not have to be a component of the image inspection system 1.

In the description of this embodiment, the direction in which the workpiece W is transported by the transporting belt conveyor B (the moving direction of the workpiece W) is defined as the Y direction, and the direction orthogonal to the Y direction in a plan view of the transporting belt conveyor B is defined as the X direction. The direction perpendicular to the X direction and the Y direction (the direction perpendicular to the top surface of the transport belt conveyor B) is defined as the Z direction, but this is only defined for the convenience of explanation.

The image inspection system 1 can be used to inspect the appearance of the workpiece W, that is, to inspect the surface of the workpiece W for defects such as scratches, dirt, and dents. It is also possible to make a pass/fail judgment. During operation, the image inspection system 1 receives an inspection start trigger signal that defines the start timing of a defect inspection (pass/fail determination inspection) from the external control device 8 via a signal line. The image inspection system 1 performs imaging and illumination of the workpiece W based on this inspection start trigger signal, and obtains an inspection image after predetermined processing. Thereafter, a visual inspection is performed based on the inspection image, and the inspection results are transmitted to the external control device 8 via the signal line. In this manner, during operation of the image inspection system 1, input of an inspection start trigger signal and output of inspection results are repeatedly performed between the image inspection system 1 and the external control device 8 via the signal line. The input of the inspection start trigger signal and the output of the inspection results may be performed via the signal line between the image inspection system 1 and the external control device 8, as described above, or via other signals (not shown). It may also be done via a line. For example, a sensor for detecting the arrival of the work W may be directly connected to the image inspection system 1, and an inspection start trigger signal may be input from the sensor to the image inspection system 1. Further, the image inspection system 1 may be configured to automatically generate a trigger signal internally and perform an inspection.

Further, the image inspection system 1 may be configured with dedicated hardware, or may be configured with software installed on a general-purpose device, for example, an image inspection program is installed on a general-purpose or dedicated computer. For example, the image inspection program may be installed in a dedicated computer with hardware such as a graphic board specialized for image inspection processing.

(Configuration of acquisition unit A)
The acquisition unit A is configured to acquire a brightness image according to the amount of reflected light received from the workpiece W, and also acquire height data in which height information of the workpiece W is arranged on two-dimensional coordinates. It is. The brightness image can be acquired by imaging the workpiece W illuminated by the illumination device 2 using the imaging unit 3 . When the workpiece W does not need to be illuminated, a brightness image can be obtained by imaging the workpiece W with the imaging section 3 without illuminating it with the illumination device 2.

The method of acquiring height data in which the height information of the workpiece W is arranged on two-dimensional coordinates is not particularly limited, and any method may be used, and hardware and software suitable for the method may be used. You can choose as appropriate. The acquisition unit A can generate a three-dimensional image by acquiring height data.

In other words, there are two main methods for acquiring height data: one is a passive method that acquires height data using images captured under lighting conditions for obtaining normal brightness images; (passive measurement method), and the other is an active method (active measurement method) in which height data is acquired by actively irradiating light for measuring in the height direction. A typical passive method is the stereo measurement method. This involves preparing two imaging units 3, arranging these two imaging units 3 in a predetermined positional relationship, performing imaging respectively, and performing well-known calculations based on the two images generated. You can get height data with .

Representative methods of the active method are the light cutting method and the pattern projection method. In the optical cutting method, in the stereo measurement method described above, one of the imaging units 3 is replaced with a laser projector, and a line-shaped laser beam is projected onto the workpiece W, so that the line light is generated according to the shape of the surface of the workpiece W. The three-dimensional shape of the workpiece W is restored based on the degree of distortion of the image. The optical cutting method does not require the determination of corresponding points, so stable measurement is possible.

The pattern projection method captures multiple images by shifting the shape, phase, etc. of a predetermined projection pattern projected onto the workpiece W, and analyzes the captured multiple images to determine the three-dimensional shape of the workpiece W. This is the method to restore it. There are several types of pattern projection methods, in which the phase of the sine wave striped pattern is shifted, multiple images (at least 3 or more) are taken, and the phase of the sine wave is determined for each pixel from the multiple images. The phase shift method uses the determined phase to find three-dimensional coordinates on the work surface, and the three-dimensional shape is restored using a type of spatial frequency waviness phenomenon that occurs when two regular patterns are combined. In the moiré photography method, the pattern itself projected onto the workpiece W is different for each shot, and for example, at a black and white duty ratio of 50%, the stripe width becomes thinner by half the screen, one-fourth, one-eighth, etc. A spatial coding method in which a striped pattern is sequentially projected, a pattern projection image is taken for each pattern, and the absolute phase of the height of the workpiece W is determined, and multiple thin line pattern illuminations (multi-slits) are projected onto the workpiece. However, a typical example is a multi-slit method in which the pattern is moved at a pitch narrower than the slit period and images are taken multiple times.

Further, although height data can be acquired by combining the above-described phase shift method and spatial coding method, the present invention is not limited to this, and height data may be acquired by other methods. Furthermore, any conceivable method other than the above-mentioned methods, such as optical radar method (time of flight), focused focus method, confocal method, white light interferometry, etc., may be used to obtain height data. I do not care.

The illumination device 2 and the imaging unit 3 are configured to be able to implement a pattern projection method. The specific configurations of the illumination device 2 and the imaging section 3 will be described below.

(Configuration of lighting device 2)
First, the lighting device 2 will be explained. The illumination device 2 is a light projecting section that projects structured illumination having a predetermined projection pattern onto the workpiece W, and includes a light emitting section 2a and a lighting control section 2b that controls the light emitting section 2a. The light emitting section 2a and the illumination control section 2b may be separate bodies or may be integrated. Furthermore, the lighting control section 2b may be incorporated into the setting device 4. The light emitting section 2a can be configured with, for example, a light emitting diode, a projector using a liquid crystal panel, an organic EL panel, a digital micromirror device (DMD), etc., and can also be called a lighting section. Although light emitting diodes, liquid crystal panels, organic EL panels, and DMDs are not shown, those having conventionally known structures can be used. The lighting device 2 is connected to the setting device 4 via a signal line 100a, and can be installed apart from the imaging section 3 and the setting device 4.

The lighting device 2 of Embodiment 1 is configured to be able to perform uniform surface emission. Furthermore, the illumination device 2 is configured to be able to perform illumination that can implement deflectometry processing, as an example of structured illumination having a predetermined projection pattern, and therefore has a periodic illuminance distribution. It has a light emitting section 2a that irradiates the workpiece W with pattern light. That is, the illumination device 2 can be a patterned light illumination unit that performs patterned light illumination in which the workpiece W is sequentially irradiated with a plurality of different patterned lights. The illumination device 2 used when acquiring an inspection image including height data by performing deflectometry processing will be described below.

When using a plurality of light emitting diodes, the plurality of light emitting diodes can be arranged in a dot matrix and a pattern of light having a periodic illuminance distribution can be generated by controlling the current value. For example, in the case of Y-direction pattern light in which brightness and darkness change in the Y-direction, it can also be expressed as pattern light in which a striped pattern is repeated in the Y-direction, and when generating this Y-direction pattern light, the phase of the illuminance distribution is By shifting in the Y direction, it is possible to generate a plurality of Y direction pattern lights with different phases of illuminance distribution. The illuminance distribution of the Y-direction pattern light can also be represented by a waveform that approximates a sinusoidal waveform. In this case, for example, by changing the phase by 90°, the Y-direction pattern light at 0° and the Y-direction pattern light at 90° can be expressed. It is possible to generate pattern light, Y-direction pattern light at 180°, and Y-direction pattern light at 270°.

In addition, in the case of X-direction pattern light in which the brightness and darkness change in the X direction, it can also be expressed as pattern light in which a striped pattern is repeated in the By shifting in the X direction, it is possible to generate a plurality of X direction pattern lights with different phases of illuminance distribution. The illuminance distribution of the X-direction pattern light can also be represented by a waveform that approximates a sinusoidal waveform. In this case, for example, by changing the phase by 90°, the X-direction pattern light at 0° and the X-direction pattern light at 90° It is possible to generate pattern light, X-direction pattern light at 180°, and X-direction pattern light at 270°. That is, the lighting device 2 can illuminate the workpiece W in different lighting modes. When performing deflectometry processing, the pattern light irradiated onto the workpiece W can be not only a sin waveform but also a triangular wave pattern light.

It is also possible to irradiate light with a uniform illuminance distribution within the plane by passing a current having the same current value through all the light emitting diodes. By changing the current value flowing through all the light emitting diodes at the same value, the light emitting state can be changed from a dark side light emitting state to a bright side light emitting state.

Further, in the case of a liquid crystal panel and an organic EL panel, by controlling each panel, the light irradiated from each panel can be made to be patterned light having a periodic illuminance distribution. In the case of a digital micromirror device, by controlling built-in micromirror surfaces, it is possible to generate and irradiate patterned light with a periodic illuminance distribution. Note that the configuration of the illumination device 2 is not limited to that described above, and any device or device that can generate patterned light having a periodic illuminance distribution can be used.

Further, as will be described later, the illumination device 2 used when generating inspection images using the photometric stereo method can be an illumination device that can individually irradiate illumination from at least three or more different directions. Further, the lighting device 2 may be a lighting device configured to enable multispectral illumination, and the configuration of the lighting device 2 is not particularly limited.

(Configuration of imaging unit 3)
Next, the imaging section 3 will be explained. The imaging section 3 includes a camera 31, a condensing optical system 32, and an imaging control section 33. The imaging unit 3 is connected to the setting device 4 via a signal line 100b, and can be installed apart from the lighting device 2 and the setting device 4.

The camera 31 has an image sensor made of an image sensor such as a CCD or CMOS that converts the intensity of light obtained through the condensing optical system 32 into an electrical signal. The camera 31 acquires a brightness image according to the amount of reflected light received from the workpiece W, and also receives the reflected light reflected by the structured illumination device 2 from the workpiece W to create a plurality of patterns. This is an image generation unit that generates a projection image. The imaging control section 33 includes a storage device and a signal processing device, and is a section that controls the start and end of exposure of the camera 31, controls the exposure time, adjusts analog gain, and the like. The internal logic circuit can control the driving of the image sensor and the transfer of image data. Further, the imaging control section 33 is capable of performing various image processing, filter processing, image generation, etc., and the imaging section 3 is a device having a filter processing function.

The condensing optical system 32 is an optical system for condensing light incident from the outside, and typically includes one or more optical lenses. Further, the condensing optical system 32 is configured to be able to perform autofocus. The positional relationship between the imaging unit 3 and the illumination device 2 is set so that the light emitted from the illumination device 2 toward the surface of the workpiece W is reflected on the surface of the workpiece W and enters the condensing optical system 32. can do. When the workpiece W is a translucent member such as a transparent film or sheet, pattern light emitted from the illumination device 2 toward the surface of the workpiece W is transmitted through the workpiece W and collected by the imaging unit 3. The positional relationship between the imaging unit 3 and the illumination device 2 can be set so that the light enters the optical system 32 . In any of the above cases, the imaging unit 3 and the illumination device 2 are arranged so that the specular reflection component and the diffuse reflection component reflected from the surface of the work W enter the condensing optical system 32 of the imaging unit 3. Furthermore, a line camera in which light-receiving elements are arranged linearly in the Y direction can also be used for the imaging unit 3, but instead of a line camera, an area camera (in which the light-receiving elements are arranged in a line in the In the case of this area camera, an illumination form called coaxial illumination is also possible.

(Embodiment 2)
FIG. 2 is a diagram schematically showing a schematic configuration and operational state of an image inspection system 1 according to Embodiment 2 of the present invention. In the second embodiment, the imaging control section 33 is separated from the camera 31 having an image sensor and the condensing optical system 32, and the illumination control section 2b is incorporated into this imaging control section 33 and integrated therewith. The imaging control section 33 is connected to the setting device 4 via a signal line 100b. The imaging control section 33 and the illumination control section 2b can also be separate bodies.

(Embodiment 3)
FIG. 3 is a diagram schematically showing a schematic configuration and operational state of an image inspection system 1 according to Embodiment 3 of the present invention. In the third embodiment, the illumination control section 2b is incorporated into and integrated with the imaging control section 33, and the display section 5 and the mouse 7 are connected to the imaging control section 33. The setting device 4 is composed of, for example, a notebook computer, and is connected to the imaging control section 33. The setting device 4 has a keyboard 6. Further, a mouse may be connected to the setting device 4. Various user interfaces generated by the setting device 4 can be displayed on the display unit 5 via the imaging control unit 33.

(Embodiment 4)
FIG. 4 is a diagram schematically showing a schematic configuration and operational state of an image inspection system 1 according to Embodiment 4 of the present invention. In the fourth embodiment, the lighting device 2 and the imaging section 3 are integrated, and the lighting control section 2b and the imaging control section 33 are also integrated. In Embodiments 2 to 4, the same parts as in Embodiment 1 are given the same reference numerals and explanations are omitted.

(Deflectometry processing)
Embodiments 1 to 4 are configured to acquire height data based on a plurality of pattern projection images generated by the imaging unit 3. Specifically, phase data of the surface of the workpiece W is generated based on the principle of deflectometry from a plurality of brightness images taken by the imaging unit 3, and height information of the workpiece W is calculated based on the phase data. It is configured to perform deflectometry processing to generate height data arranged on dimensional coordinates, that is, a height image indicating the shape of the workpiece W.

As an example shown in FIG. 5, the light emitting section 2a of the illumination device 2 includes a first light emitting section 21, a second light emitting section 22, a third light emitting section 23, and a fourth light emitting section 24. Each of the fourth light emitting units 21 to 24 is capable of projecting a pattern.

Hereinafter, generation of a height image will be explained in detail based on FIG. 5. When the first light emitting unit 21 of the illumination device 2 irradiates the workpiece W with four pattern lights according to the spatial code method, the imaging unit 3 generates a gray code pattern image set consisting of four different images. Further, when the first light emitting unit 21 of the illumination device 2 irradiates the workpiece W with eight pattern lights according to the phase shift method, the imaging unit 3 generates a phase shift pattern image set consisting of eight different images.

Similarly, when the second light emitting unit 22 of the illumination device 2 irradiates the workpiece W with patterned light according to the spatial code method, a gray code pattern image set is generated, and the patterned light according to the phase shift method is irradiated onto the workpiece W. When irradiating W, a phase shift pattern image set is generated.

Similarly, when the third light emitting unit 23 of the illumination device 2 irradiates the workpiece W with patterned light according to the spatial code method, a gray code pattern image set is generated, and the patterned light according to the phase shift method is irradiated onto the workpiece W. When irradiating W, a phase shift pattern image set is generated.

Similarly, when the fourth light emitting unit 24 of the illumination device 2 irradiates the workpiece W with patterned light according to the spatial code method, a gray code pattern image set is generated, and the patterned light according to the phase shift method is irradiated onto the workpiece W. When irradiating W, a phase shift pattern image set is generated.

After generating the phase shift pattern image set, relative phase calculation processing can be performed by acquiring each image data of the phase shift pattern image set and using the phase shift method. Thereby, a phase image can be obtained. This phase is a relative phase (phase before unwrapping).

Further, after generating the gray code pattern image set, each image data of the gray code pattern image set is acquired, and by using the spatial code method, a spatial code calculation process is performed to obtain a stripe number image. A stripe number image is an image in which a space to which light is irradiated is divided into a large number of small spaces, and each of the small spaces is given a series of space code numbers to be identified.

After that, absolute phase phasing processing is performed. In the absolute phase phasing process, the phase image and the fringe number image are combined (unwrapped) to generate an absolute phase image (intermediate image). In other words, the spatial code number obtained by the spatial code method can be used to correct phase jumps (phase unwrapping) by the phase shift method, making it possible to obtain high-resolution measurement results while ensuring a wide dynamic range of height. .

Note that the height may be measured only by the phase shift method. In this case, the height measurement dynamic range is narrowed, so if the work W has a large difference in height such that the phase is shifted by one cycle or more, the height cannot be measured correctly. Conversely, in the case of a workpiece W with a small change in height, striped images are not captured or synthesized using the spatial code method, so there is an advantage that the processing speed can be increased accordingly. For example, when measuring a workpiece W with little height difference, it is not necessary to have a large dynamic range, so the phase shift method alone can maintain highly accurate height measurement performance and shorten processing time. be able to. Furthermore, since the absolute height is known, the height may be measured only by the spatial code method. In this case, accuracy can be increased by increasing the number of codes.

The absolute phase image can also be called an angle image in which each pixel has information on the irradiation angle of the pattern light onto the work W. That is, since the first pattern image set (phase shift pattern image set) includes eight first pattern images obtained by shifting the phase of the sine wave striped pattern, the phase shift method can be used. As a result, each pixel has information on the irradiation angle of the pattern light onto the workpiece W. That is, it is possible to generate a first angle image in which each pixel has information on the irradiation angle of the first measurement pattern light onto the workpiece W, based on the plurality of first pattern images. The first angle image is an image of the angle of light irradiated from the first light emitting unit 21 to the workpiece W.

Similarly, each pixel has information on the irradiation angle of the second measurement pattern light onto the workpiece W based on the plurality of second pattern images, and each pixel has the information on the irradiation angle of the second measurement pattern light on the workpiece W based on the plurality of third pattern images. A third angle image having irradiation angle information of the third measurement pattern light onto the workpiece W, and a third angle image having irradiation angle information of the fourth measurement pattern light onto the workpiece W based on a plurality of fourth pattern images. 4-angle images can be generated.

The uppermost image of the intermediate images in FIG. 5 is the first angle image, the second image from the top is the second angle image, the third image from the top is the third angle image, and the lowermost image is the first angle image. The image is a fourth angle image. The parts of each angle image that appear to be filled with black are the parts that are in the shadow of the illumination, and are invalid pixels with no angle information.

A first height image can be generated using the first angle image and the second angle image, and a second height image can be generated using the third angle image and the fourth angle image. Thereafter, the first height image and the second height image can be combined to generate a combined height image. This allows the number of invalid pixels to be reduced in the combined height image.

(Embodiment 5)
FIG. 6 is a diagram schematically showing a schematic configuration and operational state of an image inspection system 1 according to Embodiment 5 of the present invention. In the first to fourth embodiments described above, the inspection images are generated based on the principle of deflectometry, whereas in the fifth embodiment, the inspection images are generated using the photometric stereo method. It is configured.

Hereinafter, a specific method of generating an inspection image using the photometric stereo method will be described with reference to FIG. will be explained in detail.

The image inspection system 1 according to the fifth embodiment can be configured in the same manner as the image inspection system disclosed in, for example, Japanese Patent Application Publication No. 2015-232486. That is, the image inspection system 1 includes an imaging unit 3 that images the workpiece W from a fixed direction, and an illumination device 200 that illuminates the workpiece W from three or more different illumination directions. It includes at least a display section 5, a keyboard 6, and a mouse 7. An acquisition unit A is configured by the imaging unit 3 and the illumination device 200.

The illumination device 200 is configured to irradiate light onto the workpiece W from different directions, and controls the first to fourth light emitting sections 201 to 204 and the first to fourth light emitting sections 201 to 204. The lighting control section 205 has a lighting control section 205. The illumination device 200 is a part that performs multidirectional illumination that irradiates light from different directions to the workpiece W, and is configured to be able to project structured illumination having a predetermined projection pattern. The first to fourth light emitting units 201 to 204 are arranged so as to surround the workpiece W at intervals from each other. The first to fourth light emitting units 201 to 204 can use light emitting diodes, incandescent bulbs, fluorescent lights, or the like. Further, the first to fourth light emitting sections 201 to 204 may be separate bodies or may be integrated.

In this embodiment, the first to fourth light emitting sections 201 to 204 are turned on in sequence, and the imaging section 3 images the workpiece W when any one of the first to fourth light emitting sections 201 to 204 is turned on. For example, when the lighting device 200 receives the first lighting trigger signal, the lighting control unit 205 lights only the first light emitting unit 201. At this time, the imaging unit 3 receives the imaging trigger signal and images the workpiece W at the timing when the light is irradiated. When the illumination device 200 receives the second illumination trigger signal, the illumination control section 205 lights only the second light emitting section 202, and at this time the imaging section 3 images the workpiece W. In this way, four brightness images can be obtained. Note that the number of lights is not limited to four, but can be any number as long as it is three or more and can illuminate the workpiece W from different directions.

Then, the normal vector of each pixel to the surface of the workpiece W is calculated using the pixel values of the corresponding pixels among the plurality of brightness images captured by the imaging unit 3. Differential processing is performed on the calculated normal vector of each pixel in the X direction and the Y direction to generate a contour image showing the contour of the inclination of the surface of the workpiece W. Also, from the calculated normal vector of each pixel, the albedo of each pixel of the same number as the normal vector is calculated, and from the albedo, a texture depiction image showing a pattern with the tilted state of the surface of the workpiece W removed is generated. . Since this method is a well-known method, detailed explanation will be omitted. By combining a plurality of brightness images based on the principle of photometric stereo, a shape image showing the shape of the workpiece W can be generated.

(Multi-spectral lighting)
In another embodiment, the illumination device 2 may be capable of multispectral illumination as structured illumination with a predetermined projection pattern. Multispectral illumination refers to irradiating the workpiece W with light having different wavelengths at different timings, and is suitable for inspecting printed matter (object to be inspected) for color unevenness, dirt, etc. For example, the lighting device 2 can be configured so that it can sequentially irradiate yellow, blue, and red onto the work W. Specifically, the lighting device 2 may have LEDs of multiple colors, or may have a liquid crystal display. The lighting device 2 may be configured with a panel, an organic EL panel, or the like.

The imaging unit 3 images the workpiece W at the timing when the light is irradiated to obtain a plurality of brightness images. Then, a plurality of luminance images can be combined to obtain an inspection image. This is called multispectral imaging. A shape image showing the shape of the workpiece W can be generated by combining a plurality of brightness images using multispectral imaging. Note that the irradiated light can also include ultraviolet rays and infrared rays.

(Connection interface between camera 31 and image inspection application)
The camera 31 of the first to fifth embodiments is a GenICam standard compatible camera that is compatible with the GenICam standard. The GenICam standard is a standard that standardizes the connection interface between a PC application and the camera 31, and as shown in FIG. This standardizes the interface for controlling the camera 31 and acquiring images captured by the camera 31 using the image inspection application 40 of the setting device 4. If both the imaging unit 3 and the image inspection application 40 of the setting device 4 are compatible with the GenICam standard, the camera 31 and the image inspection application 40 of the setting device 4 can be connected. The image inspection application 40 is composed of software installed on a personal computer. In this embodiment, a case will be described in which the camera 31 and the setting device 4 comply with the GenICam standard as a standardized standard, but the standardized standard is not limited to the GenICam standard, and may be other standardized standards. good.

Assuming that the image inspection application 40 is decomposed into a hierarchical structure, the GenICam layer 41 includes upper layers (inspection section 4A, image processing section 4B) that actually perform image inspection, defect inspection, etc. in the image inspection application 40; It is positioned as an intermediate layer located between the layer 42 that performs control based on specific network communication standards. The GenICam layer 41 can be roughly classified into two parts: GenApi 41a and GenTL (Transport Layer) 41b. GenApi 41a is a part that converts setting items of the camera 31 and register addresses inside the camera 31. This GenApi 41a allows the image inspection application 40 to abstractly input character strings such as "ExposureTime" for exposure time and "AnalogGain" for analog gain, without specifying the specific address of the camera 31. When the camera 31 is connected, the register address corresponding to the character string (called Feature) is determined by analyzing a file called DeviceXML (details will be described later) obtained from the camera 31. can be determined.

GenTL41b specifies an interface (API) that controls data transfer between the image inspection application 40 and the camera 31, and specifically specifies the API specifications for writing and reading the register of the camera 31 and This defines the specifications of an API for passing image data transferred from the image inspection application 31 to the upper layer of the image inspection application 40.

The physical standard for connecting the image inspection application 40 and camera 31 may be any standard that uses a high-speed network; for example, GigE Vision standard 3A using an Ethernet cable 31a and a USB 3.0 compatible cable 31b. There is a USB3Vision standard 3B. Therefore, the imaging unit 3 and the setting device 4 are connected via a network, but this network may be wired using a high-speed network cable or wireless. Further, the network may be a network using a cable other than the Ethernet cable 31a or the USB 3.0 compatible cable 31b, and is not particularly limited.

The GenICam standard does not specifically specify what is to be used as a physical communication standard, and only defines an abstract specification in the form of GenTL41b. As a layer 42 below this GenTL 41b, communication standards are defined using specific communication networks, such as the GigE Vision standard 3A and the USB3Vision standard 3B. The specific communication standard is not limited to the GigE Vision standard 3A or the USB3Vision standard 3B, but may be compatible with the GenICam standard. In FIG. 7, in order to explain the concept of the GigE Vision standard 3A and the USB3Vision standard 3B, cameras 31, 31 compatible with each standard are shown connected to the setting device 4, but only one camera 31 is set. It can be used by connecting to the device 4.

The GenICam standard compatible camera 31 stores a file called DeviceXML (DeviceXML file) 31c. The DeviceXML file 31c is stored in a storage device (internal memory) built into the camera 31. The image inspection application 40 compliant with the GenICam standard reads the DeviceXML file 31c from the camera 31 when the image inspection application 40 is connected to the camera 31 to be set. To read the DeviceXML file 31c, for example, there is a method of downloading the DeviceXML file 31c from the camera 31. The downloaded DeviceXML file 31c is held in the GenApi 41a. The DeviceXML file 31c can be downloaded in response to a request from the image inspection application 40 or automatically when connected to the camera 31. By downloading the DeviceXML file 31c, the setting device 4 downloads the DeviceXML file 31c. Can be obtained. Furthermore, without downloading the DeviceXML file 31c from the camera 31 to be connected, it is also possible to obtain the DeviceXML file 31c corresponding to the camera 31 by another means (for example, downloading from a website) and specify it to the GenApi 41a at the time of connection. be.

In the DeviceXML file 31c, all setting items held inside the camera 31 and register addresses (register information) in which the setting values of each setting item are stored are described in association with each other. The setting items are called features, and each feature is assigned a character string for specifying the individual feature. A reference to a node in which a specific register address is written is written in each Feature node. The register information also includes a register address and a character string that specifies the register.

In the DeviceXML file 31c, for example, there is a Feature named "ExposureTime" (exposure time setting), and its attribute is instructed to refer to a register address "ExposureTimeReg", and a certain value is specified as a specific address. Suppose it was written. If the camera 31 is different, the address may have a different value, but the name of the Feature "ExposureTime" is common. In this way, by managing common setting items for many cameras 31 using unified names, the image inspection application 40 can easily control the camera 31 without being aware of differences in model or manufacturer. 31 can be set and controlled.

As described above, the GenApi 41a layer converts the Feature string passed as an argument from the image inspection application 40, which is a higher layer, into a register address by analyzing the contents of the DeviceXML file 31c. For example, when writing a value (value: 100.5) to a register corresponding to a Feature called "ExposureTime", there are two values: the address value and the value to be written, such as WriteRegister (address value, 100.5). By executing a function with the argument , it is possible to set the value of the register of the camera 31 via a physical communication standard such as the GigE Vision standard 3A or the USB3Vision standard 3B.

Note that although the DeviceXML file 31c defines features that are common to each manufacturer, it is also possible to define features unique to each vendor. In the case of the camera 31 having highly unique functions, by accessing the camera 31 through a dedicated Feature, it is also possible to utilize special functions that do not exist in general-purpose cameras.

(Internal processing unit of imaging section 3)
FIG. 8 is a flowchart conceptually showing an example of internal processing of the imaging section 3. In this flowchart, the group for position correction 1 is described as consisting of a pattern search unit and a position correction unit included in the group, but the groups for position corrections 2 to 5 are not described. This is because the groups of position corrections 2 to 5 are collapsed to simplify the diagram.

As shown in this flowchart, it is composed of a combination of multiple units. A unit is a unit that controls imaging and image processing, and by combining the units on a flowchart, the imaging section 3 can realize a desired operation. In the flowchart shown in FIG. 8, a unit is described as one step.

For example, if a function that performs a certain process is enabled through user settings, the process can be made executable by adding a unit corresponding to that function. A unit can be defined as a combination of a program for executing processing, parameters necessary for executing processing, and a storage area for storing data of processing results. Each process can be performed by the imaging control section 33 shown in FIG. 1, and the storage area can be secured in the storage device of the imaging control section 33. Note that the concept of the unit itself is not essential when realizing the external specifications of the camera 31 that conforms to the GenICam standard.

The flowchart shown in FIG. 8 is a flowchart in which a plurality of units are arranged vertically and horizontally, and can also simply be called a flow. It may be a flowchart in which a plurality of units are arranged only in the vertical direction. Although the processing is sequentially executed from the start to the end of the flowchart shown in FIG. 8, it is also possible to branch by providing a branching step SB1 in the middle. In the case of branching, a merging step SB2 can be provided before reaching the end, so that it is possible to proceed to the end after merging.

(parameter set)
When operating the image inspection application 40 in an actual inspection environment, the setting parameters of the imaging unit 3 can be dynamically changed when the workpiece W is switched or changes in the surrounding environment such as brightness are detected. be. If only a very limited parameter, such as exposure time, is to be changed, this can be done by directly writing the corresponding Feature value from the image inspection application 40.

On the other hand, in the case of a highly functional imaging unit 3, the number of items that can be set increases, and the number of parameters that must be changed at once when switching the workpiece W also increases. In this case, the time required to change settings increases in correlation with the number of parameters. In an actual image inspection line, when the workpiece W is switched, the time from when the new workpiece W reaches the imaging field of view of the imaging unit 3 to when it leaves the field of view is often short, making it possible to perform a series of setting changes at high speed. There will be cases where you want to do it. In such cases, a function called parameter set may be used.

The parameter set is a set in which a plurality of patterns of combinations of various parameters for imaging by the imaging unit 3 are stored in advance, and each pattern can be managed by a parameter set number. For example, if there are three types of work W and it is desired to perform imaging and subsequent processing using different parameters for each type, three parameter sets are prepared.

When a parameter set is used, a series of setting changes can be completed in a short time by simply specifying the parameter set number corresponding to the type of workpiece W to be imaged next before imaging the workpiece W. When viewed from the image inspection application 40, a Feature that specifies a parameter set number can be considered as a type of selector that will be described later.

The selector for switching the target set from the image inspection application 40 side and the register for switching the parameter internally referenced by the imaging unit 3 during operation can be independent or can be the same. The upper limit of the number of parameter sets that can be set is limited to the parameter holding space of the internal memory provided inside the imaging section 3.

The concept of a parameter set can also be applied to the imaging unit 3 having a filter processing function. For example, when the parameter set Index is 1, the binarization filter is executed, when the parameter set Index is 2, the expansion filter is executed, and when the parameter set Index is 3, the filter is not executed. Assume that parameters corresponding to the set index have been set. By doing so, the content to be executed as imaging and filter processing can be dynamically switched using the parameter set Index.

The GenICam standard supports a function to dynamically switch imaging parameters. The imaging unit 3 conforms to the GenICam standard and allows all setting parameters to be accessed using Features. The imaging function includes a function that synchronizes the illumination device 2 and the imaging unit 3 and synthesizes images taken in multiple patterns using a dedicated algorithm (generation of an inspection image using the principle of deflectometry described above), It has multiple illumination-imaging control modes, including a multispectral imaging function that captures multiple images by irradiating different lights. These can be set for each of the above parameter sets, and it is possible to execute imaging processing, filter processing, composition processing, and image output while dynamically switching according to conditions. The image generated by the imaging function may be the image itself obtained by the image sensor while switching the illumination pattern, or may include multiple images synthesized by the above-mentioned dedicated algorithm. .

In filter processing, it is possible to set multiple patterns within the same parameter set. For example, in the above-mentioned imaging function, multiple patterns of images are generated by performing a series of imaging operations, but it is possible to individually apply filter processing to the multiple different images generated. . In another pattern, it is also possible to set a specific region of interest (ROI) for the same image and then filter only the inside of each region. The range of a specific area can be set by operating the keyboard 6 or mouse 7, for example. There may be one or more specific areas. The size of the specific area can be set arbitrarily.

The filter processing may be a multi-stage filter that can repeatedly set multiple types of filters in multiple stages for the same image. For example, after performing dilation filter processing on an image, it is possible to perform binarization filter processing on that image. The filter processing is not limited to a multi-stage filter, but may be a single filter processing.

Using the GenICam standard's function of dynamically switching imaging parameters, multiple parameter sets can be combined into one flowchart as shown in FIG. On the internal processing flowchart of the imaging unit 3, a group of units forming the flowchart from branching step SB1 to merging step SB2 is collectively referred to as a parameter set. The example shown in FIG. 8 has four parameter sets, ie, first to fourth parameter sets. In other words, there are multiple patterns of combinations of multiple selector values to implement the processing set by the user. The user can set which of the first to fourth parameter sets to select. For example, when parameter set number 1 is selected, the first parameter set is automatically selected.

After the flowchart shown in FIG. 8 starts, each unit constituting any one of the first to fourth parameter sets is passed through in a branching step SB1, and then merged in a merging step SB2 to proceed to the end. If parameter set number 1 is selected, the branch number becomes 1 in branch step SB1, and each process of the first parameter set is executed. If parameter set number 2 is selected, the branch number becomes 2 in branch step SB1, and each process of the second parameter set is executed. If parameter set number 3 is selected, the branch number becomes 3 in branch step SB1, and each process of the third parameter set is executed. If parameter set number 4 is selected, the branch number becomes 4 in branch step SB1, and each process of the fourth parameter set is executed. The number of parameter sets is not limited to four and can be set arbitrarily.

Specific examples of parameter sets are shown in FIGS. 9 and 10. FIG. 9 shows a parameter set for generating an inspection image using the principle of deflectometry. The unit UA1 executes multiple imaging processes with a single trigger signal in order to generate an inspection image using the principle of deflectometry, the unit UA2 executes an expansion filter process, and the unit UA3 executes an averaging filter. The process is executed, and the unit UA4 executes a grayscale inversion process. That is, multi-step processing defined by a parameter set including filter processing is sequentially performed on the captured image captured by unit UA1. Thereafter, in the unit UA5, the image data on which the multi-step processing has been sequentially executed is transferred to the PC, that is, outputted to the setting device 4 or the like which is an external device. Note that the image data may be held internally without being transferred. Depending on the parameter set, settings may be made so that multi-step processing is not performed. When transferred to the setting device 4, defect inspection and pass/fail determination can be performed in the inspection section 4A of the image inspection application 40 shown in FIG. Conventionally known algorithms can be used for defect inspection and pass/fail determination.

On the other hand, FIG. 10 shows a parameter set when generating an inspection image by multispectral imaging. Unit UB1 executes multiple imaging processes with a single trigger signal in order to generate an inspection image by multispectral imaging, and unit UB2 executes binarization filter processing on a certain area (area 0). , unit UB3 performs expansion filter processing on a region (region 1) different from region 0. That is, in this example as well, multi-step processing defined by a parameter set including filter processing is sequentially performed on the captured image captured by unit UB1. Thereafter, the process is not executed in unit UB4. Unit UB5 transfers the image data to the PC in the same way as shown in FIG. As in this example, a parameter set may include a unit that does not perform processing, that is, a disabled unit, and a parameter set may include a mixture of enabled units and disabled units. It's okay.

(moving object tracking)
When generating inspection images based on photometric stereo or multispectral principles, multiple imaging processes are performed using a single trigger signal, and the resulting multiple images are combined. In this case, if the workpiece W is moving during a plurality of imaging processes, the combining process after imaging cannot be performed correctly, so it is necessary to detect the amount of shift for each image. A search for detecting the amount of deviation can be performed by a search unit, and after correcting the amount of deviation based on the detected amount of deviation, a process of combining a plurality of images is executed.

The compositing process can be performed by an image calculation unit. Combination settings for combining a plurality of captured images are included as setting items in the DeviceXML file. Therefore, before outputting the captured image captured by the imaging unit 3 to the outside, the DeviceXML file must include a composition setting for compositing multiple captured images as a setting item regarding the processing applied to the captured image. become.

(Type of unit)
There are multiple types of units, such as a pattern search unit that performs pattern search processing to determine the inspection area, a position correction unit, an image calculation unit that internally performs image position correction, color extraction, filter processing, etc. There are devices that are highly functional by combining units that perform these relatively simple processes. Each unit is a unit for executing processing applied to the captured image captured by the imaging section 3 before outputting the captured image to the outside.

The pattern search unit is a unit that searches for the workpiece W or the inspection target part of the workpiece W from among the images including the workpiece W imaged by the imaging unit 3, and corrects the position of the workpiece W in the captured image. be. For example, when setting up the image inspection system 1, edge detection is performed using a well-known edge detection method on an image of the workpiece W, and an area specified by the detected edge is detected on the workpiece W or a portion of the workpiece W to be inspected. It can be stored in the storage device of the imaging unit 3 as a model (model image for search). The edge detection process itself can be performed using a conventionally known method. For example, the pixel value of each pixel on the brightness image is acquired, and the change in pixel value on the brightness image is equal to or greater than the threshold for edge detection. If a region exists, its boundary portion is extracted as an edge. The edge extraction threshold can be adjusted arbitrarily by the user.

After the image inspection system 1 is set, when the image inspection system 1 is operated, images of the workpieces W that are sequentially conveyed are obtained to obtain inspection images, and the pattern search unit searches the workpieces W or the like on the obtained inspection images. A search process is performed to determine the presence or absence of the inspection target portion of the workpiece W based on the stored model, and the position and angle of the workpiece W are measured by the search process. The position of the workpiece W can be specified using the X and Y coordinates. Further, the angle of the workpiece W can be an angle around the imaging axis of the imaging unit 3 or an angle around the Z axis shown in FIG.

The position correction unit uses a pattern search unit to search for the work W or the inspection target part of the work W from among the images including the work W captured by the imaging unit 3, and after measuring the position and angle of the work W, This is a unit for correcting the position of the workpiece W in the image. During operation of the image inspection system 1, a plurality of workpieces W are not always transported in the same position and orientation, and workpieces W in various positions or in various postures may be transported. . Since the pattern search unit searches for the reference part of the workpiece W and the position correction unit can correct the position, the reference part of the workpiece W is always at a constant position and the workpiece W is kept at a predetermined position. Position correction is performed by rotating the image or moving the image vertically or horizontally so that the image has the correct orientation. A plurality of types of position correction tools, such as a pattern search tool, can be prepared for position correction.

There are multiple types of image processing units, such as a unit that performs filter processing, a unit that performs synthesis processing that combines multiple images captured by the imaging unit 3, a unit that performs deflectometry processing, and a unit that performs photometric stereo method. There are units that use this to generate inspection images, units that perform multispectral imaging, etc. Since there are multiple types of filter processing, multiple types of units that perform filter processing can be provided, such as a binarization filter, an expansion filter, and the like. Generation of an inspection image by deflectometry processing goes through multiple processes as described above, so a unit may be provided for each process, such as a unit that generates a specular reflection component image, a unit that generates a diffuse reflection component image, A unit or the like that generates a reference phase difference image can be provided.

(Configuration of setting device 4)
The setting device 4 transmits data indicating the setting values of each setting item set by the user and register information corresponding to each setting item included in the DeviceXML file 31c to the imaging unit 3, and changes the settings of the imaging unit 3. It is a device for doing this.

As shown in FIG. 7, the setting device 4 includes a UI generation section 4a. The UI generation section 4a is a section that generates various user interface images. Various user interface images generated by the UI generation section 4a are displayed on the display section 5.

On the user interface image, it is possible to switch images to be edited, display and change setting items, and display images to be edited. Setting items include, for example, position correction settings and filter processing settings, and the Feature corresponding to the position correction settings includes a part for selecting whether to enable position correction and a part for selecting the type of position correction tool. The parts to be performed are assigned. Further, as a Feature corresponding to the filter processing settings, the type of the selected filter and a part for selecting and adjusting parameters related to filter settings such as extraction size and extraction color are assigned.

For example, when a user selects an arbitrary image, the selector value that specifies the setting target is internally switched to the index value corresponding to the preprocessing module used to generate that image. The configurable contents change depending on the image. When the value of the selector that specifies the setting target is specified, one or more Features of the preprocessing module pointed to by the selector corresponding to that value are read, an image reflecting the setting items is generated and displayed, and the position Display each parameter value as a correction or filter setting. When the user operates parameter values for position correction or filter settings, the operation is accepted and the setting items of the unit corresponding to the preprocessing module corresponding to the value of the selector specifying the setting target are changed.

The following method can be used to specify the unit to be accessed from the selector value. In other words, the preprocessing module is composed of multiple units, and the selector that switches the index of the preprocessing module can be common to these units. , which Feature belongs to which unit can be named such that it is uniquely determined by the Feature name. Thereby, the unit to be accessed can be specified from the combination of the selector value and the Feature selected as the editing target.

Therefore, when the user changes the setting values for the setting items of the imaging unit 3, the setting values of each setting item set by the user and data indicating register information corresponding to each setting item included in the DeviceXML file are is sent to the imaging unit 3 and the settings of the imaging unit 3 can be performed. Therefore, if the imaging unit 3 complies with the standardization specifications, the setting value can be changed from the setting device 4 side, and the setting value of the imaging unit 3 can be changed. Increased freedom in model selection.

In addition, since it is possible to uniquely specify a part of multi-step processing that is executed sequentially by a combination of selector values, it is possible to uniquely specify a part of multi-step processing that is executed sequentially. It will be possible to perform multiple steps of processing, such as generation of inspection images using the principle of metrology, multispectral imaging, and filter processing of generated inspection images.

(Processing executed when setting up image inspection system 1)
As shown in the block diagram of FIG. 11, the imaging control section 33 includes a setting section 33a, a frequency distribution calculation section 33b, a height range setting section 33c, and a correction section as sections that execute various processes including height extraction processing. 33d, a calculation section 33e, a conversion section 33f, and an output section 33g. Hereinafter, the setting performed before operation of the image inspection system 1 will be described with reference to the setting procedure for height extraction based on the flowchart shown in FIG. 12.

The flowchart shown in FIG. 12 shows the setting procedure for height extraction performed when setting up the image inspection system 1, and the steps described as "internal processing" are steps executed within the image inspection system 1. .

In step SC1 after the start, a filtered image that has been subjected to filter processing is added to the image acquired by the acquisition unit A. This step SC1 is executed by the user. After adding the desired filter image, the process proceeds to step SC2, where a three-dimensional image (3D image) is set as the target image, and a two-dimensional image (2D image) is set as the output image type.

In step SC3, a 2D image conversion setting dialog 50 shown in FIG. 13 is opened. When the user performs a predetermined operation, the UI generation unit 4a generates a 2D image conversion setting dialog 50 and displays it on the display unit 5, for example. The 2D image conversion setting dialog 50 includes a converted image display area 50a that displays an image obtained by converting a 3D image into a 2D image, and a through display area 50b that displays a 3D image before being converted to a 2D image. is provided. Further, the 2D image conversion setting dialog 50 includes a detailed setting area 51. FIG. 13 shows an example in which a three-dimensional image of the workpiece W viewed from above is displayed in the through display area 50b. On the other hand, FIG. 14 shows an example in which a three-dimensional image of the workpiece W viewed from the side (horizontal direction) is displayed in the through display area 50b.

After displaying the 2D image conversion setting dialog 50 in step SC3, the process advances to step SC4. In step SC4, operate the detailed setting area 51 of the 2D image conversion setting dialog 50 to set the conversion mode to "height shading" and select "plane reference" as the specification method for specifying the reference position for specifying the reference plane. Set. As an option for the specification method, for example, there is a "point-based" method. Thereafter, the process proceeds to step SC5, and on the image displayed in the post-conversion image display area 50a, an area is set by a rectangular frame line 52 at a plane location for specifying the reference plane.

That is, the conversion unit 33f shown in FIG. 11 is a part that converts height data in a certain height range into a low gradation height image using a reference plane as a reference as described later. It is necessary to specify a reference plane that will be used as a reference when converting the key. In the processes of steps SC4 and SC5, a reference position for specifying the reference plane can be set with respect to the height data acquired by the acquisition unit A. The processing in steps SC4 and SC5 is executed by the setting unit 33a.

In the setting unit 33a, when the "plane reference" is set in step SC4, the frame line 52 is displayed superimposed on the image displayed in the converted image display area 50a. The position, size, and shape of the frame line 52 can be freely changed by the user operating the mouse 7 or the like. Since the user knows the rough shape of the workpiece W in advance, the user can arrange the frame line 52 so as to surround the flat part of the workpiece W. After arranging and determining the frame line 52 at a desired location, the process proceeds to step SC6, where a reference plane is specified from the height information within the area surrounded by the frame line 52. This process can be executed by the setting unit 33a, and the setting unit 33a specifies, for example, a plane specified by the average height of a plurality of points within the area surrounded by the frame line 52 as the reference plane.

The setting of the reference position by the setting unit 33a is not limited to setting a region. For example, if "point reference" is selected as the specification method in step SC4, the user can operate the mouse 7 or the like to set the reference position as a point in step SC5. If only one point is set in step SC5, the plane containing that one point is set as the reference plane in step SC6. Furthermore, if three points are set in step SC5, the plane specified by the heights of those three points can be used as the reference plane in step SC6. The reference plane setting method is not particularly limited, and setting methods other than those described above may be used.

In step SC7, the height data is converted so that the inclination of the reference plane specified in step SC6 is set to 0. As a result, the tilt of the three-dimensional image displayed in the through display area 50b shown in FIG. 13 is corrected.

Thereafter, the process proceeds to step SC8, where the extraction range is specified using the height histogram. The extraction range, which will be described later, is a predetermined height range in which gradation conversion is performed. In step SC8, first, the frequency distribution calculation unit 33b shown in FIG. 11 calculates a frequency distribution for the height of each pixel included in the height information of the height data. When the frequency distribution calculation unit 33b calculates the frequency distribution, it generates a histogram representing the calculated frequency distribution and displays it in the histogram display area 53 of the 2D image conversion setting dialog 50 shown in FIG. The vertical axis of the histogram displayed in the histogram display area 53 is the height, and the horizontal axis is the frequency. Therefore, by viewing the histogram, the user can easily grasp the height range with a high frequency, the height range with a low frequency, the height range with no frequency, etc.

In the histogram display area 53, as an example of means for displaying the height range, a center display line 53a, an upper display line 53b, and a lower display line 53c are displayed in a state overlapping with the above-mentioned histogram. The center display line 53a is a line indicating the center of the extraction range, and indicates the position of the reference plane on which an offset amount, which will be described later, is reflected. As shown in FIG. 14, when the image of the work W seen from the side is displayed in the through display area 50b, the center display surface 54a is displayed in a state overlapping with the image of the work W.

The upper display line 53b and the lower display line 53c are lines indicating the upper and lower limits of the extraction range, respectively. The extraction range is lower than the upper display line 53b and higher than the lower display line 53c. By using the upper display line 53b and the lower display line 53c, the height range set by the height range setting section can be displayed on the histogram. The upper display line 53b and the lower display line 53c may have the same color or line type, and only the center display line 53a may have a different color or line type. Since the height range is between the upper display line 53b and the lower display line 53c, the area between the upper display line 53b and the lower display line 53c is colored in order to clearly distinguish it from other areas. etc., the display form may be changed.

As shown in FIG. 14, when the image of the workpiece W seen from the side is displayed in the through display area 50b, the upper display surface 54b corresponds to the upper display line 53b, and the lower display line 53c corresponds to the upper display surface 54b. The lower display surface 54c is displayed superimposed on the image of the workpiece W. Therefore, by using the upper display surface 54b and the lower display surface 54c, the height range set by the height range setting section 33c can be displayed on the three-dimensional image. The upper display surface 54b and the lower display surface 54c may have the same color or line type, and only the center display surface 54a may have a different color or line type.

The 2D image conversion setting dialog 50 is provided with an extraction range setting area 55. The extraction range setting area 55 includes a Z offset adjustment section 55a and a gain adjustment section 55b. The height range setting unit 33c generates an extraction range setting area 55, and receives user operations performed in each adjustment unit 55a, 55b. The Z offset adjustment section 55a and the gain adjustment section 55b can be provided with an input section for increasing or decreasing the value by operating a mouse or the like, an input section for directly inputting numerical values, and a display area where the input values can be confirmed. can be provided.

The Z offset adjustment section 55a is a section for specifying the amount of offset in the height direction (Z direction) of the reference plane by a user's operation. After the value specified by the Z offset adjustment section 55a is received by the height range setting section 33c, the process proceeds to step SC9, where the offset amount of the reference plane is set. The display positions of the center display line 53a and the center display surface 54a (shown in FIG. 14) are updated to correspond to the set offset amount. When the offset amount is adjusted in the positive direction, the center display line 53a and the center display surface 54a move upward, and when the offset amount is adjusted in the negative direction, the center display line 53a and the center display surface 54a move downward. Move to. As the display position of the center display line 53a is updated, the display positions of the upper display line 53b and the lower display line 53c are also updated, and as the display position of the center display surface 54a is updated, the display positions of the upper display surface 54b and the lower display line 53a are updated. The display position of the display surface 54c is also updated.

The gain adjustment section 55b is a section for specifying the extraction range by user operation (step SC8). The gain value is a value indicating how many mm corresponds to one gradation, and can be expressed in gradation/mm. Increasing the gain value results in gradation conversion with clear contrast. For example, when the gain value is set to 100 gradations/mm, gradation conversion is performed such that each gradation is 0.010 mm.

The gain value specified by the gain adjustment section 55b is received by the height range setting section 33c, and then set to a gain value based on the extraction range (step SC9). The display positions of the upper display line 53b, the lower display line 53c, and the upper display surface 54b and lower display surface 54c (shown in FIG. 14) are updated to correspond to the specified extraction range. When the extraction range is narrowed, as shown in FIG. 15, the interval between the upper display line 53b and the lower display line 53c on the histogram becomes narrower, and the interval between the upper display surface 54b and the lower display surface 54c of the through display area 50b decreases. It gets narrower. As described above, step SC9 of the flowchart shown in FIG. 12 is completed.

In step SC10, the three-dimensional image is converted into a two-dimensional image according to the offset amount and gain value of the reference plane set in step SC9. Specifically, the conversion unit 33f shown in FIG. 11 converts the height data into a height image having a lower number of gradations than the number of gradations of the height data based on the offset amount and gain value of the reference plane. Convert to If the height data is 16 bits, the height image can be converted to an 8-bit gradation number. The above is the setting procedure for height extraction.

As described above, height extraction settings include area setting parameters for calculating a reference plane, height extraction setting parameters, and the like. All of these setting parameters are prepared as features, and are sent from the setting device 4 to the imaging unit 3.

Next, the procedure for setting position correction performed when setting up the image inspection system 1 will be described based on the flowchart shown in FIG. 16. In step SD1 after the start, position correction is enabled. The subsequent steps can be performed by the correction section 33d shown in FIG. 11, for example. In step SD2, when the user operates a pattern search setting button (not shown), the UI generation unit 4a generates a pattern search setting dialog 56 shown in FIG. 17, and displays it on the display unit 5, for example. The pattern search setting dialog 56 is provided with an image display area 56a and an image registration button 56b. An image captured by the imaging section 3 is displayed in the image display area 56a. The user checks the image displayed in the image display area 56a, and if it is the desired image, operates the image registration button 56b (step SD3). When the image registration button 56b is operated, the process proceeds to step SD4, and the image currently displayed in the image display area 56a is stored in the imaging control unit 33 as a search model image.

In step SD5, a search area is set. The search area is an area in which a search for a pattern to be searched is performed. When setting a search area, a frame line 57 for setting the search area is displayed in the image display area 56a as shown in FIG. 17, and the user changes the position, size, and shape of this frame line 57. The area surrounded by the changed frame line 57 is set as the search area.

Further, in step SD6, a pattern area is set. The pattern area is an area to be searched by pattern search, and can be a characteristic part of the model image. When setting a pattern area, a frame line 58 for pattern area setting is displayed in the image display area 56a as shown in FIG. 17, and the user changes the position, size, and shape of this frame line 58. The area surrounded by the changed frame line 58 is set as a pattern area. Either step SD5 or step SD6 may be performed first.

In step SD7, pattern search parameters are set. The pattern search setting dialog 56 shown in FIG. 17 is provided with a pattern search parameter setting area 56c. Pattern search parameters include, for example, color extraction settings, search angle range, search sensitivity, search accuracy, and the like. The user can arbitrarily set pattern search parameters, and the set parameters are stored.

As mentioned above, in the position correction settings, there are setting parameters that represent the position and size of the search area and pattern area, pattern search setting parameters, etc., and these setting parameters are internally changed according to the user's setting operations. Be changed. All of these setting parameters are prepared as features, and are sent from the setting device 4 to the imaging unit 3. Furthermore, in step SD4, the image that serves as the reference for the pattern search is registered as a model image, and a "model image registration command" for this registration is also prepared as a Feature. By issuing a model image registration command from the setting device 4 to the imaging unit 3, registration of the model image can be executed.

(Processing executed during operation of image inspection system 1)
Next, the processing executed during operation of the image inspection system 1 will be described with reference to the height extraction setting procedure based on the flowchart shown in FIG. 18. When the setting device 4 issues an Acquisition command to the imaging unit 3, the following processes are sequentially executed inside the imaging unit 3.

In step SE1 after the start, the imaging unit 3 generates a three-dimensional image and a two-dimensional image of the workpiece W. Thereby, the acquisition unit A can acquire a brightness image according to the amount of reflected light received from the workpiece W, and can also acquire height data in which height information is arranged on two-dimensional coordinates. The height data acquired during operation is height data for a new workpiece W that is different from the workpiece W from which height data was acquired at the time of setting, that is, for setting the reference position. The brightness image is also a brightness image of a new workpiece W different from the workpiece W at the time of setting. The three-dimensional image and two-dimensional image acquired in step SE1 can be displayed on the display unit 5.

In step SE2, the correction unit 33d shown in FIG. 11 executes a pattern search using the two-dimensional image newly acquired in step SE1. In the pattern search, a new shift in the position or posture of the workpiece W is detected by searching for a characteristic part of a new two-dimensional image according to the pattern area, search area, and search parameters set in the position correction settings. Thereby, the amount of deviation in the X direction, the amount of deviation in the Y direction, and the rotation angle of the new workpiece W with respect to the model image can be obtained as the search results. Thereafter, the process proceeds to step SE3, and the correction unit 33d shown in FIG. 11 uses the search result obtained in step SE2 to rotate and correct the position of the new three-dimensional image obtained in step SE1. Thereby, it is possible to correct the deviation of the reference position with respect to the new height data.

Note that as a method for correcting the deviation of the reference position, in addition to pattern search, for example, an edge extraction method that does not use a model image may be used.

After correcting the deviation of the reference position, the process proceeds to step SE4. In step SE4, the calculation unit 33e shown in FIG. 11 calculates a reference plane for new height data using the height information corresponding to the corrected reference position. If "Plane reference" is set, the plane specified by the average height of multiple points will be specified as the reference plane, and if "Point reference" is set, the plane containing the set one point will be specified. The plane specified by the heights of the three points is the reference plane.

After calculating the reference plane, the process proceeds to step SE5. In step SE5, the converter 33f corrects the inclination of the three-dimensional image by converting the inclination of the reference plane to zero. Here, it is assumed that the above step SE3 has not been performed. In step SE3, the position of the three-dimensional image is corrected, but if the position is not corrected and the workpiece W is imaged in a rotated state with respect to the model image, the rotated image will be The reference surface will be calculated using When calculating the reference plane, as described above, information within a predetermined area set by the frame line 52 shown in FIG. 14 is used, but the area within this frame line 52 is The image of the rotated workpiece W is different from the image of the rotated workpiece W. As a result, if the reference plane is calculated using the image of the rotated workpiece W, the tilt correction will not be performed as expected and the result will not be correct. There is a risk that it will not be possible to obtain

On the other hand, by going through step SE3 as in this example, even if the image shows the rotated workpiece W, the image is rotated so that it is in the same direction as the model image, and the frame line shown in FIG. The area within 52 can be the same area as the model image. As a result, the tilt correction is performed as expected. Note that if there is no possibility that the workpiece W will rotate, the step of position correction may be omitted.

Thereafter, the process proceeds to step SE6, where gradation conversion is performed on the new height data. That is, the converting unit 33f converts the new height data into a number of gradations (e.g., 16 bits) lower than the number of gradations (e.g., 16 bits) of the new height data, using the reference plane calculated by the calculation unit 33e as a reference. :8bit).

At the time of gradation conversion, the converting unit 33f can also perform conversion based on the reference plane calculated by the calculating unit 33e and a predetermined height range set by the height range setting unit 33c. For example, the area between the upper display surface 54b and the lower display surface 54c (between the upper display line 53b and the lower display line 53c) of the 2D image conversion setting dialog 50 shown in FIG. 14 is set as the predetermined height range. If so, only a predetermined height range in the new height data is extracted and gradation-converted. The narrower the predetermined height range, the higher the gain during gradation conversion.

After the gradation conversion, three-dimensional display processing is executed in step SE7. The three-dimensional display process may be, for example, a three-dimensional polygon display or a three-dimensional point cloud display.

Thereafter, the process proceeds to step SE8, and the output unit 33g outputs the height image generated by the conversion unit 33f to the setting device 4 using a connection interface standardized by the above-mentioned GenICam standard or the like.

Height extraction result data, pattern search result data, etc. are all prepared as features, and by issuing an Acquisition command from the setting device 4 to the imaging section 3, the setting device 4 acquires the above result data from the imaging section 3. Can be read.

Further, the height image generated by the converter 33f is a two-dimensional image that can be handled by a two-dimensional image processing library. In other words, even if only a two-dimensional image processing library that handles height images is installed on the setting device 4, since the image sent from the imaging unit 3 is a two-dimensional image, the setting device 4 Image inspection, etc. can be performed without writing program code to convert height data to a height image.

(Zero surface specified)
Next, the procedure for specifying the zero plane during operation of the image inspection system 1 will be described based on the flowchart shown in FIG. 19. Steps SF1, SF2, and SF3 after the start are the same as steps SE1, SE2, and SE3 in the flowchart shown in FIG. 18, respectively. In step SF4, preprocessing is performed on the three-dimensional image. After that, the process proceeds to steps SF5 and SF6. Steps SF5 and SF6 are the same as steps SE4 and SE5 in the flowchart shown in FIG. 18, respectively.

In step SF7, the output unit 33g outputs the three-dimensional image corrected in step SF6 to the setting device 4 using the standardized connection interface described above. In this example, the setting device 4 can use a three-dimensional image in which the inclination of the reference plane has been corrected.

(3D difference)
Next, a procedure for acquiring a three-dimensional difference during operation of the image inspection system 1 will be described based on the flowchart shown in FIG. 20. Steps SG1, SG2, and SG3 after the start are the same as steps SE1, SE2, and SE3 in the flowchart shown in FIG. 18, respectively. In step SG4, preprocessing is performed on the three-dimensional image in the same way as when specifying the zero plane. Thereafter, the process proceeds to step SG5. In step SG5, the difference between the three-dimensional image and the model image registered at the time of setting is obtained. Thereby, the position of the workpiece W on the image can be adjusted accurately.

In step SG6, the output unit 33g outputs the three-dimensional image after taking the difference in step SG5 to the setting device 4 through the standardized connection interface described above.

(Visual inspection)
In this example, as shown in FIG. 7, the setting device 4 includes an inspection section 4A and an image processing section 4B. The image processing section 4B is a section that performs image processing on the height image output from the output section 33g of the imaging section 3. The image processing performed by the image processing section 4B is a process for increasing the accuracy of the visual inspection performed by the inspection section 4A. Since the inspection section 4A performs an appearance inspection based on the image processed by the image processing section 4B, inspection accuracy can be improved.

(Operations and effects of embodiments)
As explained above, according to this embodiment, when the acquisition unit A acquires the brightness image and height data of the workpiece W, the user sets the reference position and identifies the reference plane based on the set reference position. be able to.

During operation of the image inspection system 1, the acquisition unit A can acquire a new brightness image and new height data for a new workpiece W different from the workpiece W at the time of setting. The characteristic portion of the new luminance image is searched by the correction unit 33d of the imaging unit 3, and if the search results show that the reference position has shifted, the deviation can be corrected within the imaging unit 3.

After correcting the reference position, a new reference plane for height data is calculated using the height information corresponding to the reference position, and the new height data is calculated based on the reference plane for the new height data. is converted into a low-height image to generate a 2D image that can be handled by a 2D image processing library.

Since the generated height image is output according to a standardized standard, when the imaging unit 3 is used by connecting to the setting device 4 such as a general-purpose personal computer, the height image can be easily received by the setting device 4. Can be done. Furthermore, since the received height image can be handled by a two-dimensional image processing library, image processing can be easily performed using the setting device 4 without the user having to write program code for converting it into a two-dimensional image. This makes it possible to do so without the need for time and effort for writing.

For example, even if the workpiece W is tilted from the horizontal plane, the reference plane is calculated after the deviation of the reference position is corrected as described above, and the height of the low gradation number is calculated based on the calculated reference plane. Since it is converted into an image, there is no need for the user to write code to correct the tilt. Therefore, user-friendliness is improved.

The embodiments described above are merely illustrative in all respects and should not be interpreted in a limiting manner. Furthermore, all modifications and changes that come within the scope of equivalents of the claims are intended to be within the scope of the present invention.

As described above, the present invention can be used, for example, when imaging a workpiece or the like or when inspecting an imaged workpiece or the like.

1 Image inspection system 2 Illumination device (light projector)
4A Inspection section 4B Image processing section 31 Camera (image generation section)
33a Setting section 33b Frequency distribution calculation section 33c Height range setting section 33d Correction section 33e Calculation section 33f Conversion section 33g Output section 300 Imaging device A Acquisition section W Work (imaged object)

Claims (8)

an acquisition unit that acquires a brightness image according to the amount of reflected light received from the imaged object and acquires height data in which height information of the imaged object is arranged on two-dimensional coordinates;
a setting unit that sets a reference position for specifying a reference plane that is a reference for gradation conversion of the height data acquired by the acquisition unit;
When the acquisition unit acquires a new luminance image and new height data for a new imaging target different from the imaging target from which the height data used for setting the reference position was acquired, the new a correction unit that searches for a characteristic part of a brightness image to detect a shift in the position or posture of a new object to be imaged, and corrects a shift in the reference position with respect to the new height data;
a calculation unit that calculates a reference plane of the new height data using height information corresponding to the reference position corrected by the correction unit;
a conversion unit that converts the new height data into a height image having a lower number of gradations than the number of gradations of the new height data, using the reference plane calculated by the calculation unit as a reference;
An imaging device comprising: an output section that outputs the height image generated by the conversion section according to a standardized standard. The imaging device according to claim 1,
The imaging device includes:
A file in which a setting item consisting of the reference position and register information indicating a location where the setting value of the setting item is stored is obtained from an external source, and the setting value of the setting item set by the user, and the setting value of the setting item set by the user are obtained. The device is configured to be connectable via a network to a setting device capable of transmitting register information corresponding to the setting items included in the file to the imaging device;
An imaging device that stores a setting value of the setting item received from the setting device in a location indicated by register information corresponding to the setting item. The imaging device according to claim 1 or 2,
The acquisition unit includes:
a light projecting unit that projects structured illumination having a predetermined projection pattern onto an imaging target;
In addition to acquiring a brightness image according to the amount of reflected light received from the imaging target, the structured illumination projected by the light projecting unit receives the reflected light reflected by the imaging target to project a plurality of patterns. and an image generation unit that generates an image,
An imaging device configured to acquire height data based on a plurality of pattern projection images generated by the image generation section. An imaging device according to any one of claims 1 to 3,
A file that is connected to the imaging device via a network and that describes setting items of the imaging device and register information indicating a location where the setting values of each setting item are stored is obtained from an external source, and a file that is set by the user is obtained. An image inspection system comprising: a setting device that transmits data indicating setting values of each setting item and register information corresponding to each setting item included in the file to the imaging device, and configures settings of the imaging device. hand,
The setting device includes:
a display unit that displays height information of the new height data;
a height range setting section that sets a predetermined height range using the height information displayed on the display section;
The conversion unit converts the new height data into new height data based on the reference plane calculated by the calculation unit and the predetermined height range set by the height range setting unit. An image inspection system configured to convert a height image into a height image having a lower number of tones than a number of tones. The image inspection system according to claim 4,
The setting device includes a frequency distribution calculation unit that calculates a frequency distribution for the height of each pixel included in the height information of the new height data, using the reference plane calculated by the calculation unit as a reference;
The display unit displays the frequency distribution calculated by the frequency distribution calculation unit,
The image inspection system is configured such that the height range setting section is capable of setting a predetermined height range on the frequency distribution displayed on the display section. The image inspection system according to claim 4 or 5,
The acquisition unit generates a three-dimensional image of the object to be imaged based on the height data,
The image inspection system is configured such that the display unit displays the three-dimensional image generated by the acquisition unit and also displays the height range set by the height range setting unit on the three-dimensional image. . The image inspection system according to claim 5,
The display unit is an image inspection system configured to display the height range set by the height range setting unit on a frequency distribution. The image inspection system according to any one of claims 4 to 7,
The setting device includes an image processing unit that performs image processing on the height image output from the output unit, and an inspection unit that performs an appearance inspection based on the image processed by the image processing unit. image inspection system.

Images