JP7231432B2 - Image processing device - Google Patents

Description

The present invention relates to an image processing apparatus equipped with an illumination device for illuminating an object to be measured and an imaging device for receiving light reflected from the object to be measured, and more particularly to a technical field of a structure capable of acquiring a three-dimensional shape of the object to be measured. belongs to

Conventionally, as this type of image processing apparatus, a pattern light having a light intensity distribution that varies depending on the position is projected onto an object to be measured, the light reflected from the object to be measured is received, and the height obtained based on the amount of received light is measured. A so-called pattern projection method is known that obtains the three-dimensional shape of an object to be measured using information. As a pattern projection method, there is a phase shift method in which pattern light whose illuminance distribution is varied, for example, sinusoidally is projected a plurality of times with different phases and an image is captured each time.

Patent Document 1 discloses an illumination device that generates and projects pattern light, including a light source, a condenser lens that collects light emitted from the light source, and a liquid crystal panel that receives the light converged by the condenser lens. and configured to project a pattern formed on a liquid crystal panel onto an object to be measured.

Japanese Patent No. 4011561

By the way, when a liquid crystal panel is used, the transmittance of light becomes highest when light is incident parallel to the normal to the display surface (output surface) of the liquid crystal panel. The greater the angle, the lower the light transmission. In other words, since the liquid crystal panel has angular characteristics in which the light transmittance varies depending on the direction of light incidence, when pattern light is generated using the liquid crystal panel, luminance unevenness occurs according to the position of the pattern light. There is a risk.

Therefore, in general, as disclosed in Patent Document 1, the light source and the liquid crystal panel are positioned so that the incident direction of the light emitted from the light source is parallel to the normal line of the display surface of the liquid crystal panel. .

In addition, at a site where a large number of objects to be measured are transported, it is desirable to minimize the time required to acquire the three-dimensional shape of one object. The pattern light may be projected sequentially, and the time required to acquire the three-dimensional shape may be prolonged. Therefore, it is conceivable to use a TN system liquid crystal panel with a high response speed to switch between multiple types of patterned light at high speed. In this case, the above-described unevenness in luminance becomes conspicuous, and the positional relationship between the incident direction of light and the liquid crystal panel is more strictly defined.

Therefore, when using a liquid crystal panel, it is necessary to arrange the light source and the liquid crystal panel so that the incident direction of light is parallel to the normal line of the display surface of the liquid crystal panel. In many cases, the lighting device is installed with the liquid crystal panel tilted with respect to the screen. In addition, it is difficult to reduce the size of the illuminating device because a condensing lens is required between the light source and the liquid crystal panel to condense the light. Furthermore, if it is attempted to project pattern light onto the object to be measured from a plurality of directions in order to reduce the non-measurable region, the size of the illumination device will be further increased.

SUMMARY OF THE INVENTION The present invention has been made in view of the above points, and an object of the present invention is to reduce the size of an illuminating device while satisfying the requirements for the angular characteristics of a liquid crystal panel.

In order to achieve the above object, a first invention is an image processing apparatus for measuring the height of an object to be measured. a first light source and a second light source that are provided apart from each other in a direction and emit diffused light; , a first liquid crystal panel provided such that an emission surface is positioned in a plane perpendicular to the central axis of the opening; a second liquid crystal panel arranged so as to have an exit surface positioned within the plane; and projecting different patterns of light from the first liquid crystal panel onto the object to be measured a plurality of times, A lighting device comprising: a light projection control section that controls the first light source, the second light source, the first liquid crystal panel, and the second liquid crystal panel so that the second liquid crystal panel projects different pattern lights a plurality of times. and a first pattern image set obtained by receiving the reflected light of the plurality of pattern lights emitted by the first liquid crystal panel through the opening of the illumination housing and the light emitted by the second liquid crystal panel. an imaging device that generates a second pattern image set obtained by receiving reflected light of a plurality of pattern lights; and the first pattern image set and the second pattern image set generated by the imaging device. an inspection object image generation unit for generating an inspection object image including height information of the measurement object in the central axis direction of the illumination device; and an inspection process based on the inspection object image generated by the inspection object image generation unit. and an inspection unit that performs

According to this configuration, when diffused light emitted from the first light source is incident on the first liquid crystal panel having a pattern formed on the emission surface, patterned light is generated and projected onto the measurement object. Since the first light source emits diffused light, diffused light is incident on the first liquid crystal panel. Therefore, unevenness in luminance according to the position of the pattern light due to the angular characteristics of the liquid crystal panel is less likely to occur. In addition, since the second light source and the second liquid crystal panel project the pattern light onto the measurement object from a direction different from that of the first light source and the first liquid crystal panel, the non-measurable area on the measurement object is reduced. This configuration eliminates the need for a condensing structure using a condensing lens, so that it is possible to reduce the size of the illumination device when projecting pattern light onto the object to be measured from a plurality of directions.

The imaging device receives the reflected light from the measurement target of the pattern light projected from each of the first liquid crystal panel and the second liquid crystal panel to generate a first pattern image set and a second pattern image set. Based on the first pattern image set and the second pattern image set generated by the imaging device, the inspection target image generation unit generates an inspection target image including height information of the measurement target, and based on the inspection target image, the inspection unit performs the inspection process.

In addition, since both the exit surfaces of the first liquid crystal panel and the second liquid crystal panel are positioned on the same plane perpendicular to the central axis of the opening of the illumination housing, the pattern light is emitted from a plurality of directions to the object to be measured. can be compactly summarized.

Pattern light includes both pattern light having a periodic light intensity distribution for the phase shift method and coded light for the spatial code method.

A second aspect of the invention is characterized in that a driving method of the first liquid crystal panel and the second liquid crystal panel is a TN method.

That is, by adopting the TN driving method for the liquid crystal panel, when a plurality of types of pattern light are sequentially projected, the pattern light can be switched at high speed. Since it is large, it is conceivable that luminance unevenness according to the position of the pattern light due to the angular characteristics of the liquid crystal panel is likely to occur. In this configuration, diffused light is incident on the liquid crystal panel, so even if the change in light transmittance depending on the incident direction of the liquid crystal panel light is large, luminance unevenness according to the position of the pattern light is less likely to occur.

In a third aspect of the invention, the illumination device comprises a third light source provided between the first light source and the second light source within the illumination housing for emitting diffused light, and the opening portion within the illumination housing. a fourth light source that is provided point-symmetrically with the third light source with respect to the central axis of the third light source and emits diffused light; a third liquid crystal panel arranged correspondingly and having an emission surface positioned in a plane perpendicular to the central axis of the opening; and a fourth liquid crystal panel arranged so as to correspond to the fourth light source, and provided so that the emission surface is positioned within the plane, wherein the light projection control unit is arranged to correspond to the measurement object and controlling the third light source, the fourth light source, the third liquid crystal panel, and the fourth liquid crystal panel to project pattern light multiple times, wherein the imaging device is configured to project the opening of the illumination housing. light reflected from the measurement object of the pattern light projected from each of the third liquid crystal panel and the fourth liquid crystal panel through a portion to generate a third pattern image set and a fourth pattern image set; It is characterized in that it is configured to generate

According to this configuration, the third liquid crystal panel and the fourth liquid crystal panel project the pattern light for measurement onto the object to be measured from a direction different from the direction in which the pattern light is projected by the first liquid crystal panel and the second liquid crystal panel. be able to. Therefore, the pattern light can be projected by the third liquid crystal panel and the fourth liquid crystal panel to a portion that would be shadowed in the projection direction of the pattern light by the first liquid crystal panel and the second liquid crystal panel, and the non-measurable area can be reduced. .

Also, the first liquid crystal panel and the second liquid crystal panel can be paired, and the third liquid crystal panel and the fourth liquid crystal panel can be paired. Further, the fifth liquid crystal panel and the sixth liquid crystal panel can be similarly paired, and the seventh liquid crystal panel and the eighth liquid crystal panel can also be similarly paired in the illumination housing, in which case the fifth light source, the sixth light source, A seventh light source, an eighth light source may be provided within the illumination housing.

In a fourth invention, the pattern light has a periodic illuminance distribution that varies in a one-dimensional direction, and the first light sources are arranged in a direction in which the illuminance of the pattern light does not change. It is characterized by comprising a plurality of light emitting diodes.

According to this configuration, a linear light source can be formed by a plurality of light emitting diodes. Incidentally, the second light source, the third light source and the fourth light source can be similarly configured by a plurality of light emitting diodes.

In a fifth aspect of the invention, the first light source is arranged above an outer end portion of the first liquid crystal panel located radially outward of the illumination housing, and the light emitted from the first light source is A relative position between the first liquid crystal panel and the second light source is set so that light is incident on the first liquid crystal panel within an effective angle range of the first liquid crystal panel.

According to this configuration, the diffused light emitted from the first light source is incident on the first liquid crystal panel within the effective angle range of the first liquid crystal panel. Brightness unevenness according to position is less likely to occur. Incidentally, the first light source and the second liquid crystal panel can be similarly arranged.

According to the present invention, the first liquid crystal panel is provided with the first light source and the second light source spaced apart from each other in the circumferential direction of the opening of the lighting housing, and the diffused light emitted from the first light source and the second light source are respectively incident on the first liquid crystal panel. Since the second liquid crystal panel and the second liquid crystal panel are provided in the same plane perpendicular to the central axis of the opening of the illumination housing, while suppressing the occurrence of luminance unevenness according to the position of the pattern light due to the angular characteristics of the liquid crystal panel, It is possible to reduce the size of the illumination device capable of projecting pattern light from a plurality of directions onto the object to be measured, and to increase the degree of freedom in installing the illumination device.

1 is a diagram showing a system configuration example of an image processing apparatus according to an embodiment of the present invention; FIG. 3 is a block diagram of a controller unit; FIG. 1 is a block diagram of an imaging device; FIG. It is a top view of an illuminating device. FIG. 10 is a bottom view of the lighting device according to Embodiment 2; FIG. 6 is a sectional view taken along the line VI-VI in FIG. 5; It is a figure explaining the positional relationship of a light emitting diode and a liquid crystal panel. It is a figure explaining the positional relationship of the light emitting diode which concerns on a modification, and a liquid crystal panel. It is a figure explaining the generation point of pattern light. It is a figure explaining the example of arrangement|positioning of a 1st light emitting diode. 4 is a graph showing the relationship between the ratio of the size of a light-emitting diode and the half period of a wave, and the pixel undulation. It is a figure explaining another example of arrangement of the 1st light emitting diode. It is a block diagram of an illuminating device. FIG. 10 is a diagram showing a procedure for generating intermediate images and reliability images from a pattern image set; FIG. 10 is a diagram illustrating the procedure for generating an intermediate image and a reliability image from a pattern image set using example images; FIG. 10 is a diagram showing a gray code pattern and a phase shift pattern forming process, and the relationship between the relative phase and the absolute phase; It is a figure explaining the height measurement method by a height measurement part. It is a figure explaining the flow of correction processing. It is a figure which shows the position of the light emitting diode before correction|amendment and after correction|amendment. It is a figure which shows the presence or absence of tilt. FIG. 10 is a schematic diagram showing a case where height deviation occurs between light emitting diodes that form a pair. FIG. 4 is a diagram for explaining the estimation of deviation and the flow of adjustment of each unit; It is a schematic diagram explaining the estimation method of a shift|offset|difference. FIG. 4 is a conceptual diagram of camera calibration; FIG. 4 is a schematic diagram showing how camera parameters are set by changing the height of the object to be measured; FIG. 4 is a diagram showing mathematical expressions of a camera parameter matrix and a distortion model; It is a flow chart which shows an estimation procedure. It is a side view which shows the structural example of the 1st light projection part which has an adjustment mechanism. It is a top view which shows the structural example of the 1st light projection part which has an adjustment mechanism. It is a perspective view which shows the structural example of the 1st light projection part which has another adjustment mechanism. It is a figure explaining before and behind change of a use segment. FIG. 10 is a diagram showing a set of phase shift pattern images obtained by projecting pattern light by the first light projecting unit when the measurement object is a rectangular parallelepiped box; FIG. 33 shows a relative phase image obtained based on the phase shift pattern image set shown in FIG. 32 and an intermediate image corresponding to the phase shift pattern image set shown in FIG. 32; FIG. 10 is a diagram showing a set of phase-shifted pattern images obtained by projecting pattern light by the second light projecting section when the measurement object is a rectangular parallelepiped box; 35 shows a relative phase image obtained based on the phase shift pattern image set shown in FIG. 34 and an intermediate image corresponding to the phase shift pattern image set shown in FIG. 34; FIG. FIG. 10 is a diagram showing a user interface displaying a height image and a cross-sectional profile when the object to be measured is a rectangular parallelepiped box;

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described in detail based on the drawings. It should be noted that the following description of preferred embodiments is essentially merely illustrative, and is not intended to limit the invention, its applications, or its uses.

FIG. 1 is a diagram showing a system configuration example of an image processing apparatus 1 according to an embodiment of the present invention. The image processing apparatus 1 includes an imaging device 2, an illumination device 3, a controller section 4, a display section 5, a console section 6, and a mouse 7, and obtains a height image of the object W to be measured. , the height of the object W to be measured can be measured based on this height image, and various inspections can be performed on the object W to be measured.

The object W to be measured is placed on a mounting surface 100 of a conveying device such as a belt conveyor, for example, and the height of the object W placed on the mounting surface 100 is measured and various inspections are performed. I do. The object W to be measured is preferably kept stationary during the height measurement.

The image processing device 1 can be wired to a programmable logic controller (PLC) 101 via a signal line 101a. The image processing device 1 and the PLC 101 may be wirelessly connected. The PLC 101 is a control device for sequence-controlling the conveying device and the image processing device 1, and can use a general-purpose PLC. The image processing device 1 can also be used without being connected to the PLC 101 .

The display unit 5 is a display device such as a liquid crystal display panel, and constitutes display means. The display unit 5 includes, for example, an operation user interface for operating the image processing apparatus 1, a setting user interface for setting the image processing apparatus 1, and a height display for displaying the height measurement result of the object to be measured. A user interface for displaying measurement results, a user interface for displaying inspection results for displaying various inspection results of the object to be measured, and the like can be displayed. A user of the image processing apparatus 1 can operate and set the image processing apparatus 1 by viewing the display unit 5, and can grasp the measurement results, inspection results, and the like of the measurement object W. Furthermore, the operating state of the image processing apparatus 1 can also be grasped.

As shown in FIG. 2, the display unit 5 is connected to a display control unit 46 included in the controller unit 4, and can be controlled by the display control unit 46 to display the above-described user interface, height image, and the like. is configured as

The console section 6 is input means for the user to operate the image processing apparatus 1 and input various information, and is connected to the controller section 4 . The mouse 7 is also input means for the user to operate the image processing apparatus 1 and input various information, and is connected to the controller section 4 . The console unit 6 and the mouse 7 are examples of input means, and the input means may be, for example, a touch panel screen provided in the display unit 5 or a voice input device. Can also be configured. In the case of a touch panel screen, display means and input means can be realized by one device.

A general-purpose personal computer PC for generating and storing a control program for the controller section 4 can also be connected to the controller section 4 . Also, an image processing program for performing various settings related to image processing can be installed in the personal computer PC, and various settings for image processing performed by the controller unit 4 can be performed. Alternatively, a processing order program that defines the processing order of image processing can be generated by software operating on this personal computer PC. In the controller section 4, each image processing is sequentially executed according to the processing order. The personal computer PC and the controller unit 4 are connected via a communication network. transferred to Conversely, the processing order program, layout information, etc. can be read from the controller unit 4 and edited on the personal computer PC. Note that the above program may be generated not only by the personal computer PC but also by the controller section 4 .

Also, the controller unit 4 can be constructed with dedicated hardware, but the present invention is not limited to this configuration. For example, a general-purpose personal computer PC, a work station, or the like installed with a dedicated image processing program, an inspection processing program, a height measurement program, or the like may function as the controller unit. In this case, the imaging device 2, the lighting device 3, the display section 5, the console section 6, and the mouse 7 may be connected to a personal computer PC, workstation, or the like.

Further, although the functions of the image processing apparatus 1 will be described later, all the functions of the image processing apparatus 1 may be realized by the controller section 4 or may be realized by a general-purpose personal computer PC. Alternatively, part of the functions of the image processing apparatus 1 may be realized by the controller unit 4, and the remaining functions may be realized by a general-purpose personal computer PC. The functions of the image processing apparatus 1 can be realized by software, or by a combination of hardware.

The interface for connecting the imaging device 2, lighting device 3, display unit 5, console unit 6, and mouse 7 to the controller unit 4 may be a dedicated interface, or an existing communication standard such as Ethernet ( product name), USB, RS-232C, etc. can also be used.

The height image representing the height of the measurement object W is an image representing the height of the measurement object W in the direction of the central axis A (shown in FIG. 1) of the opening 30a of the illumination device 3 shown in FIG. It can also be called a distance image. The height image can be displayed as a height with reference to the placement surface (also referred to as a reference surface) 100 of the object W to be measured, or as a relative distance to the illumination device 3 in the direction of the central axis A. It is an image in which the grayscale value of each pixel changes according to the height. In other words, the height image can be said to be an image whose gradation value is determined based on the height of the object W to be measured with respect to the placement surface 100. It can also be said that it is an image whose grayscale value is determined based on the relative distance. Further, the height image can be said to be a multivalued image having gradation values corresponding to the height of the measurement object W with respect to the mounting surface 100, and the relative distance from the illumination device 3 in the direction of the central axis A It can also be said to be a multivalued image having grayscale values corresponding to . Furthermore, the height image can be said to be a multivalued image obtained by converting the height of the measurement object W with respect to the placement surface 100 of the measurement object W into a gray value for each pixel of the luminance image. It can also be said to be a multivalued image obtained by converting the relative distance to the device 3 in the direction of the central axis A into a grayscale value.

A height image is an image containing height information of the object W to be measured. For example, a three-dimensional composite image obtained by synthesizing and pasting an optical luminance image as texture information to a distance image is also regarded as a height image. be able to. Height images are not limited to those displayed three-dimensionally, and those displayed two-dimensionally are also included.

Methods for obtaining height images as described above are broadly divided into two methods. One is a passive method ( passive measurement method), and the other is an active method (active measurement method) in which a distance image is generated by actively irradiating the measurement object W with light for measuring in the height direction. In this embodiment, a height image is obtained by an active method, and more specifically, a pattern projection method is used.

In the pattern projection method, a plurality of images are acquired by shifting the shape, phase, etc. of the pattern of the measurement pattern light (simply referred to as pattern light) projected onto the measurement object W, and the acquired plurality of images are analyzed. is a method of obtaining the three-dimensional shape of the object W to be measured. There are several types of pattern projection methods. A phase shift method that obtains the three-dimensional coordinates of the surface of the measurement object W using the phase obtained by the measurement, or a pattern that is projected onto the measurement object W is changed for each photographing, for example, the black-and-white duty ratio is 50%, and the fringe width is the entire width. 1/4, 1/8, 1/16, . There is a spatial code method for obtaining the absolute phase of . The sinusoidal pattern light and the striped pattern light are pattern light having a periodic illumination distribution that changes in one dimension. Note that “projecting” the measurement pattern light onto the measurement object W and “irradiating” the measurement object W with the measurement pattern light are synonymous.

In the image processing apparatus 1 according to the present embodiment, the height image is generated by combining the phase shift method and the spatial code method described above, but the height image is not limited to this, and the height image is generated only by the phase shift method. Alternatively, the height image may be generated only by the spatial coding method. Also, the height image of the object W to be measured may be generated using another known active method.

The outline of the method for measuring the height of the object W to be measured by the image processing apparatus 1 is as follows. First, the measurement object W is irradiated with the first measurement pattern light and the second measurement pattern light respectively generated by the first light projection unit 31 and the second light projection unit 32 of the illumination device 3 from different directions, and the measurement is performed. The imaging device 2 receives the first measurement pattern light reflected from the object W to generate a first pattern image set including a plurality of first pattern images, and the second measurement pattern reflected from the measurement object W. The light is received by imaging device 2 to generate a second pattern image set comprising a plurality of second pattern images. After that, based on the plurality of first pattern image sets, each pixel generates a first angle image having irradiation angle information of the first measurement pattern light on the measurement object W, and based on the plurality of second pattern images to generate a second angle image in which each pixel has irradiation angle information of the second pattern light for measurement onto the object to be measured. Next, according to the irradiation angle information of each pixel of the first angle image, the irradiation angle information of each pixel of the second angle image, and the relative position information of the first light projection unit 31 and the second light projection unit 32, A height image representing the height of the object W is generated, and the height of the object W to be measured is obtained from this height image.

Although not essential, in the image processing apparatus 1, as shown in FIG. A fourth light projecting section 34 is provided. Therefore, it is also possible to irradiate the measurement target W with the third measurement pattern light and the fourth measurement pattern light respectively generated by the third light projecting section 33 and the fourth light projecting section 34 of the illumination device 3 from different directions. can. In this case, the imaging device 2 receives the third measurement pattern light reflected from the measurement object W to generate a third pattern image set including a plurality of third pattern images, and the third pattern light reflected from the measurement object W is generated. 4 The imaging device 2 receives the pattern light for measurement and generates a fourth pattern image set composed of a plurality of fourth pattern images. After that, based on the plurality of third pattern images, each pixel generates a third angle image having irradiation angle information of the third measurement pattern light onto the measurement object W, and based on the plurality of fourth pattern images, Each pixel generates a fourth angle image having irradiation angle information of the fourth measurement pattern light onto the measurement object. Next, according to the irradiation angle information of each pixel of the third angle image, the irradiation angle information of each pixel of the fourth angle image, and the relative position information of the third light projecting unit 33 and the fourth light projecting unit 34, A height image representing the height of the object W is generated, and the height of the object W to be measured is obtained from this height image.

(Phase shift method)
Here, the phase shift method will be explained. In the phase shift method, when pattern light having patterns with sinusoidally varied illuminance distributions is sequentially projected onto the object to be measured, three or more patterns of pattern light with different phases of the sinusoidal waves are projected. Each lightness value of the height measurement point is obtained from an image captured for each pattern from an angle different from the projection direction of the pattern light, and the phase value of the pattern light is calculated from each lightness value. The phase of the pattern light projected onto the measurement point changes according to the height of the measurement point, and light with a phase different from the phase observed by the pattern light reflected at the reference position is observed. become. Therefore, the phase of the pattern light at the measurement point is calculated, and the height of the measurement point is measured by substituting it into the geometric relational expression using the principle of triangulation. can be asked for. According to the phase shift method, the height of the object W to be measured can be measured with high resolution by reducing the period of the pattern light, but the measurable height range is within 2π in phase shift amount. Only low height objects (objects with a small height difference) can be measured. Therefore, the spatial code method is also used.

(Spatial code method)
According to the space code method, a space irradiated with light can be divided into a large number of small spaces each having a substantially fan-shaped cross section, and a series of space code numbers can be assigned to the small spaces. Therefore, even if the height of the object W to be measured is high, that is, even if the height difference is large, the height can be calculated from the space code number as long as it is in the space irradiated with light. Therefore, it is possible to measure the shape of the entire measuring object W having a high height. Thus, according to the spatial code method, the allowable height range (dynamic range) is widened.

(Detailed configuration of lighting device 3)
As shown in FIG. 4, the lighting device 3 according to the first embodiment includes a lighting housing 30, a first light projecting section 31, a second light projecting section 32, a third light projecting section 33, a fourth light projecting section 34 and a projection control unit 39 . As shown in FIG. 1, the illumination device 3 and the controller section 4 are connected by a connection line 3a, but the illumination device 3 and the controller section 4 may be wirelessly connected.

The illumination device 3 may be a pattern light projection dedicated device only for projecting pattern light, or may be a device also used as observation illumination for observing the measurement object W. FIG. When the illumination device 3 is used as a pattern light projection device, an observation illumination device can be provided separately from the pattern light projection device or integrally with the pattern light projection device. A light-emitting diode, a semiconductor laser, a halogen light, an HID, or the like can be used as the observation illumination device.

The lighting housing 30 has an opening 30a in the center in plan view, and the first side 30A, the second side 30B, the third side 30C, and the fourth side 30D are continuous in plan view. It has a shape close to a rectangle. Since the first side portion 30A, the second side portion 30B, the third side portion 30C, and the fourth side portion 30D extend linearly, the opening portion 30a also has a substantially rectangular shape in plan view.

The outer shape of the lighting housing 30 and the shape of the opening 30a are not limited to the illustrated shapes, and may be, for example, circular. A central axis A of the opening 30a shown in FIG. When the illumination device 3 is installed so that the lower surface of the illumination housing 30 is horizontal, the central axis A of the opening 30a extends vertically. It can also be installed in such a way that it is in a folded state, in which case the central axis A of the opening 30a is inclined.

The central axis A of the opening 30a may not strictly pass through the center of the opening 30a. An axis passing through the place can also be the central axis A. That is, the central axis A can be defined as an axis passing through the center of the opening 30a and its vicinity. An extension line of the central axis A intersects the mounting surface 100 .

In the following description, for convenience, as shown in FIG. The upper side of the illumination device 3 is referred to as the upper side of the illumination device 3, and the fourth side portion 30D side is referred to as the lower side of the illumination device 3. It can be in any direction.

The interiors of the first side portion 30A, the second side portion 30B, the third side portion 30C and the fourth side portion 30D of the lighting housing 30 are hollow. The first light projecting section 31 is housed inside the first side section 30A. A second light projecting section 32, a third light projecting section 33 and a fourth light projecting section 34 are housed inside the second side section 30B, the third side section 30C and the fourth side section 30D, respectively. The first light projecting section 31 and the second light projecting section 32 form a pair, and the third light projecting section 33 and the fourth light projecting section 34 form a pair. A light projection control unit 39 is also housed inside the illumination housing 30 .

Since the first side portion 30A and the second side portion 30B are arranged to face each other across the central axis A, the first light projecting portion 31 and the second light projecting portion 32 are symmetrical about the central axis A. They are arranged so as to be bilaterally symmetrical (point symmetrical) with respect to the center, and are separated from each other in the circumferential direction of the central axis A.

Further, the third side portion 30C and the fourth side portion 30D are also arranged so as to face each other with the central axis A interposed therebetween. They are arranged so as to be vertically symmetrical (point symmetrical) with respect to the center, and are separated from each other in the circumferential direction of the central axis A. In a plan view, four light projecting units are arranged in the order of a first light projecting unit 31, a third light projecting unit 33, a second light projecting unit 33, and a fourth light projecting unit 34 clockwise around the central axis A. will be

FIG. 5 is a bottom view of the illumination device 3 according to Embodiment 2 of the present invention. In this second embodiment, eight light projection units 31 to 38 are provided inside the illumination housing 30 . A fifth light projecting portion 35 is provided between the first light projecting portion 31 and the third light projecting portion 33 , and a sixth light projecting portion 36 is provided between the second light projecting portion 32 and the fourth light projecting portion 34 . is provided, a seventh light projecting portion 37 is provided between the second light projecting portion 32 and the third light projecting portion 33, and an eighth light projecting portion 37 is provided between the first light projecting portion 33 and the fourth light projecting portion 34 A light projecting section 38 is provided. The fifth light projecting part 35 and the sixth light projecting part 36 are a pair, and are arranged so as to be symmetrical with each other with the central axis A as the center of symmetry. Also, the seventh light projecting part 37 and the eighth light projecting part 38 are a pair, and are arranged so as to be symmetrical with each other with the central axis A as the center of symmetry. In a plan view, a first light projecting part 31, a fifth light projecting part 35, a third light projecting part 33, a seventh light projecting part 37, a second light projecting part 32, a The 6 light projecting section 36, the 4th light projecting section 34, and the 8th light projecting section 38 are arranged in this order.

Further, in the illumination device 3 of the second embodiment, an observation illumination 50 for illuminating and observing the object W to be measured is provided separately from the light projection units 31 to 38 . The observation illumination 50 is provided on the outer peripheral portion of the bottom of the illumination housing 30 and has an annular shape surrounding the first to eighth light projection portions 31 to 38 . As shown in FIG. 6, the observation illumination 50 has a substrate 51a, a plurality of observation light emitting diodes 51b mounted on the substrate 51a and emitting observation illumination light, and a cover member 52. As shown in FIG. The substrate 51a is arranged so as to surround the first to eighth light projecting sections 31-38. The plurality of observation light-emitting diodes 51b are provided at intervals in the circumferential direction so as to surround the first to eighth light projecting sections 31-38. The cover member 52 is provided so as to cover the observation light-emitting diode 41b from the light emitting surface side, and is made of a material that is translucent and has the property of diffusing light.

(Configuration of first light projecting section 31)
The first to fourth light projecting sections 31 to 34 of the lighting device 1 of the first embodiment are the same as the first to fourth light projecting sections 31 to 34 of the lighting device 1 of the second embodiment.

As shown in FIG. 6, the first light projecting portion 31 includes a casing 31a, a first LED (light emitting diode) 31b serving as a first light source for emitting diffused light, and receiving diffused light emitted from the first LED 31b. A first LCD (first pattern light generator) 31d that sequentially generates a plurality of first measurement pattern lights having patterns different from each other and irradiates the object W to be measured is provided. LCD is a liquid crystal display, ie liquid crystal panel, so the first LCD 31d is the first liquid crystal panel. The light source is not limited to a light-emitting diode, and may be any light-emitting body that emits diffused light.

As shown in FIG. 7, the first LCD 31d is arranged corresponding to the first LED 31b, and the light exit surface of the first LED 31b faces the first LCD 31d. This ensures that the light emitted from the first LED 31b enters the first LCD 31d. For example, a white light emitting diode can be used as the light emitting diode. The first LCD 31d is provided so that the display surface (emission surface) of the first LCD 31d is positioned within a plane (indicated by a virtual line 200 in FIG. 6) perpendicular to the central axis A of the opening 30a of the illumination housing 30. The plane 200 and the display surface of the first LCD 31d are located on the same plane.

The second light projecting section 32 is configured similarly to the first light projecting section 31 . Specifically, as shown in FIG. 6, the second light projecting unit 32 includes a casing 32a, a second LED 32b mounted on a substrate 32c, and a second LCD (second pattern light) arranged corresponding to the second LED 32b. generator) 32d. The first LED 31b and the second LED 32b are paired. The first LED 31b and the second LED 32b are attached to the lighting housing 30 so that their relative positions can be corrected. Details of relative position correction will be described later.

The configuration of the first light projecting section 31 will be described in detail below. As shown in FIGS. 6 and 7 , the first LEDs 31 b of the first light projecting section 31 are provided at the top inside the lighting housing 30 while being plural. The arrangement direction of the first LEDs 31b is a direction intersecting with the light emission direction.

That is, the substrate 31c is disposed above the casing 31a inside the casing 31a. A plurality of first LEDs 31b are mounted on the downward facing surface of the substrate 31c. The plurality of first LEDs 31b can be arranged in a straight line, or can be arranged such that adjacent first LEDs 31b are shifted in the vertical direction. When generating pattern light having a periodic illuminance distribution that varies in one dimension, the first LEDs 31b are arranged in a direction in which the illuminance of the pattern light does not change. By arranging the plurality of first LEDs 31b along the longitudinal direction of the first side portion 30A of the lighting housing 30 shown in FIG. Become.

The light emission directions of the plurality of first LEDs 31b can be the same, and in this embodiment, as indicated by the diagonal lines sloping to the left in FIG. It is set to reach the second side portion 30B side (the right side of the illumination device 3) from A. The light irradiation range of the plurality of first LEDs 31b is set wider than the imaging field of view of the imaging device 2 .

A specific description will be given of the light emission range of the plurality of first LEDs 31b. As shown in FIG. 17, the separation direction between the first light projecting section 31 and the second light projecting section 32 is defined as the X direction, and the vertical direction is defined as the Z direction. The illumination device 3 is arranged so that the lower surface of the illumination device 3 is horizontal (parallel to the mounting surface 100), and is installed at a distance of "1" above the mounting surface 100 of the object W to be measured. shall be A straight line D extending from the first LED 31b to a point C at (X, Z)=(0, 0) is drawn with X=0 directly below the first LED 31b. Also, a straight line F extending from the first LED 31b to a point E of (X, Z)=(1, 0) is drawn. At this time, the direction of the first LEDs 31b is set and the light source lens of the first LEDs 31b is designed so that the light emission range of the plurality of first LEDs 31b is an area sandwiched between the straight line D and the straight line F. there is

As shown in FIG. 6, the first LCD 31d is provided in the lighting housing 30 in a state separated downward from the first LED 31b. The driving method of the first LCD 31d is a TN (Twisted Nematic) method. Therefore, when the voltage applied to the first LCD 31d is 0, the liquid crystal composition (liquid crystal molecules) is aligned parallel to the display surface and allows the light of the first LED 31b to pass through. rises perpendicularly to the display surface, and blocks the light from the first LED 31b when the voltage reaches the maximum voltage.

(Relative position of LED and LED)
A relative positional relationship between the first LED 31b and the first LCD 31d will be described. As shown in FIG. 9, the display surface of the first LCD 31d is located on the same plane as a plane 200 orthogonal to the central axis A of the opening 30a of the illumination housing 30, and The other end is defined as the outer end side, and the radially inner end of the illumination housing 30 in the first LCD 31d is defined as the inner end side. The first LED 31b is arranged above the outer end side of the first LCD 31d. Symbol SEG in FIG. 9 denotes a segment that constitutes the first LCD 31d. A black segment SEG is a segment that does not transmit light, and a white segment SEG is a segment that transmits light. By alternately forming the black segment SEG and the white segment SEG on the display surface of the first LCD 31d in this way, the light transmitted through the first LCD 31d has a periodic intensity as shown below. It becomes a sinusoidal pattern light that changes to . This is the principle of pattern light generation. The number, interval, and formation position of the black segment SEG and the white segment SEG can be arbitrarily changed, and multiple types of pattern light with different wave periods and phases can be generated.

The number of black segments SEG and the number of white segments SEG can be the same. Therefore, when the pattern light is generated, the first LCD 31d can alternately form the light-transmitting region and the light-not-transmitting region with the same width. This is performed by a light projection control section 39, which will be described later.

Here, let us explain the general characteristics of liquid crystal panels. Light transmittance is highest when light is incident parallel to the normal to the display surface of the liquid crystal panel. The greater the angle formed by the two, the lower the light transmittance. Liquid crystal panels have angular characteristics in which light transmittance varies depending on the direction of light incidence. Therefore, when pattern light is generated using a liquid crystal panel, luminance unevenness occurs according to the position of the pattern light. There is concern that

In contrast, in the present embodiment, the relative positions of the first LCD 31d and the first LED 31b are set so that the light emitted from the first LED 31b is incident on the first LCD 31d within the effective angle range of the first LCD 31d. It is As a result, the diffused light emitted from the first LED 31b is incident on the first LCD 31d within the effective angle range of the first LCD 31d, so that uneven brightness according to the position of the pattern light due to the angular characteristics of the first LCD 31d is less likely to occur. Become.

The effective angular range of the first LCD 31d can be an angular range in which a predetermined or more contrast of pattern light can be secured. The fact that the contrast of the pattern light is equal to or higher than a predetermined level means that when the imaging device 2 receives the pattern light reflected from the object W to be measured, the imaging device 2 can obtain a pattern image capable of generating an image to be inspected. be. That is, due to the angular characteristics of the first LCD 31d, the greater the angle formed between the normal to the display surface of the first LCD 31d and the light incident direction, the lower the light transmittance. Although it is conceivable that the contrast of the pattern light is lowered, in this embodiment, the contrast is at least the level at which the height information of the measurement object W can be obtained based on the pattern image captured by the imaging device 2. The relative positions of the first LCD 31d and the first LED 31b are set such that

Also, the effective angle range of the first LCD 31d can be defined as follows. For example, when the light emitted from the first LED 31b passes through the first LCD 31d in which the liquid crystal molecules are aligned most easily, the angle range in which the light is attenuated by 10% or less is defined as the effective angle range of the first LCD 31d. Angle range. By setting the relative positions of the first LCD 31d and the first LED 31b so that the light attenuation rate is 10% or less, the height information of the measurement object W is obtained based on the pattern image captured by the imaging device 2. It is possible to secure the contrast to the extent possible.

The liquid crystal molecule alignment state that allows light to pass through most is the state in which the liquid crystal molecules are aligned in the direction that does not block the light path the least (when the voltage is 0 as described above). The effective angle range of the first LCD 31d can also be set so that the attenuation rate is 5% or less.

Further, as shown in FIG. 9, a normal line 201 drawn from the center of the first LED 31b (the center of the light emitting surface) toward the display surface of the first LCD 31d, and illumination in the effective angle range of the first LCD 31d from the center of the first LED 31b The relative positions of the first LED 31b and the first LCD 31d are set so that the angle α formed by the inner virtual line 202 drawn toward the boundary on the inner end side located radially inward of the housing 30 is 50° or less. can do. As a result, the angle range in which the light emitted from the first LED 31b is incident on the first LCD 31d becomes the optimal range for the TN liquid crystal panel, and as a result, the contrast of the pattern light can be ensured above a predetermined level. The angle α between the normal 201 and the inner virtual line 202 can be 45° or less, and can also be 40° or less.

Also, a normal line 201 drawn from the center of the first LED 31b toward the display surface of the first LCD 31d, and an outer virtual line 203 drawn from the center of the first LED 31b toward the boundary on the outer end side of the effective angle range of the first LCD 31d. is set to be 10° or less. The angle formed by the normal 201 and the outer virtual line 203 can be 5° or less, or can be 0°. As a result, the angle range in which the light emitted from the first LED 31b is incident on the first LCD 31d becomes the optimal range for the TN liquid crystal panel, and as a result, the contrast of the pattern light can be ensured above a predetermined level.

The distance between the first LED 31b and the surface (display surface) of the first LCD 31d and the width of the effective angle range of the first LCD 31d (the distance between the outer end and the inner end of the first LCD 31d) may be substantially matched.

(LED size)
As shown in FIG. 9, the dimension K in the direction in which waves continue on the light exit surface of the first LED 31b is set to be equal to or less than the dimension of the half cycle of the waves formed on the first LCD 31d. In this embodiment, one wave is formed by eight segment SEGs arranged in the direction in which the wave in the first LCD 31d continues, so four segment SEGs correspond to a half cycle of the wave. That is, the dimension J of the four segments SEG arranged in the wave-continuous direction is equal to the dimension K of the light exit surface of the first LED 31b, or the dimension J is longer.

From the results of experiments, it was found that the dimension K of the light emitting surface of the first LED 31b is equal to the dimension of the half period of the wave formed on the display surface of the first LCD 31d (same as the dimension of the portion through which light is transmitted), that is, the dimension J or less. It has been found that if there is, the pattern light may not have a neat sinusoidal illumination intensity distribution. If the sinusoidal pattern light is not properly generated, when the height image is obtained by image processing, high-frequency undulations appear in the height image. There are things that can be done, and it will be an obstacle to the inspection. This is the same when generating wave pattern light other than sinusoidal wave.

In response to this, it is conceivable to select a light-emitting diode having a size in which the dimension K of the light emitting surface of the first LED 31b exceeds the dimension J of the half wave period, but generally available light-emitting diode sizes are limited. Therefore, it is not possible to freely set the dimension K of the light emitting surface. As in this embodiment, there is no choice but to use a light emitting diode in which the dimension K of the light emitting surface is equal to or less than the dimension J of the half wave period. sometimes not.

FIG. 10 shows an arrangement form of the first LEDs 31b. The horizontal direction in FIG. 10 is the direction in which the illuminance of the pattern light does not change, and the 64 first LEDs 31b are arranged in order from left to right in FIG. The number of first LEDs 31b is an example and can be set arbitrarily.

The first LED 31b positioned at the left end in FIG. 10 is referred to as the first LED 31b-1, followed by the first LED 31b-2, the first LED 31b-3, . . . The first LED 31b positioned at the right end in FIG. 10 is designated as the first LED 31b-64. The first LEDs 31b-1 to 31b-64 are arranged in a direction in which the illuminance of the pattern light does not change, and are arranged so as to be shifted from each other in a direction in which the illuminance of the pattern light changes (vertical direction in FIG. 10). The optical axes irradiated by the first LEDs 31b-1 to 31b-64 are positioned on the straight line 205. As shown in FIG. A straight line 205 extends in a direction in which the illuminance of the pattern light does not change and passes through a portion of the LED array where the amount of light is greatest.

The first LEDs 31b-1 to 64 are arranged so that the center of the light emitting surface thereof is positioned within a predetermined range in the direction in which the illuminance of the pattern light changes (vertical direction in FIG. 10). The center of the light exit surface of the first LED 31b-1 and the center of the light exit surface of the first LED 31b-64 are located on a straight line 206 parallel to the straight line 205. The center is arranged at one end of the predetermined range. The center of the light exit surface of the first LED 31b-32 and the center of the light exit surface of the first LED 31b-33 are located on a straight line 207 parallel to the straight line 205. The center is arranged at the other end of the predetermined range.

The distance 208 between the straight line 205 on which the optical axis is located and the straight line 206 passing through one end of the predetermined range is the distance 209 between the straight line 205 on which the optical axis is located and the straight line 207 passing through the other end of the predetermined range. set equal. A dimension 210 obtained by combining the dimension 208 and the dimension 209 is a dimension representing the predetermined range. The first LED 31b-1, 64 is the one end light emitting diode whose light emitting surface center is located at one end of the predetermined range, and the other end light emitting diode whose light emitting surface center is located at the other end of the predetermined range is , the first LEDs 31b-32, 33. Each of the first LEDs 31b-2 to 31b-34 to 31b-63 is an intermediate light-emitting diode whose light emitting surface is centered in the middle of the predetermined range. These first LEDs 31b-2 to 63 are arranged so as to be shifted from each other in the direction in which the illuminance of the pattern light changes. Specifically, the first LEDs 31b-2 to 31b-31 and 34 to 63 are arranged so as to shift in multiple stages from near one end of the predetermined range to near the other end.

Since the first LEDs 31b-1 to 64 are arranged such that the centers of their light emitting surfaces are shifted within the predetermined range, as shown in FIG. -32 and 33 are shifted upward, and the first LEDs 31b-1 and 64 are shifted downward. As described above, the dimension of the light emitting surface of each of the first LEDs 31b is K, and the first LEDs 31b-1 to 64 are arranged so as to be shifted within the predetermined range. The apparent dimension of the light exit surface is K', which is longer than K. This dimension K' is longer than the dimension J of the half cycle of the wave formed on the first LCD 31d. As a result, high-frequency undulations are less likely to occur in the image to be inspected.

The ratio between the dimension J of the half period of the wave formed on the first LCD 31d and the dimension K' in the direction in which the wave continues on the apparent light emitting surface composed of the first LEDs 31b-1 and 64 is 1:1. It is set within the range of 2 to 1:1.4. FIG. 11 is a graph showing the relationship between the ratio of dimension J to dimension K' and pixel waviness. The horizontal axis in FIG. The pixel undulation becomes stronger as it goes up. As shown in this graph, when the ratio of dimension J to dimension K' is 1 or less, the pixel undulation becomes stronger, while when it exceeds 1.2, the pixel undulation becomes smaller, and a low level is maintained up to 1.4. there is Therefore, it is preferable to set the above ratio within the range of 1:1.2 to 1:1.4.

FIG. 12 is a diagram showing another arrangement example of the first LEDs 31b-1 to 64. In FIG. In this example, the centers of the light emitting surfaces of the first LEDs 31b-32 and 33 are located at the center of the predetermined range, the centers of the light emitting surfaces of the first LEDs 31b-1 and 64 are located at one end of the predetermined range, and the first LED 31b The center of the light exit surface of -2 and 63 is positioned at the other end of the predetermined range. Each first LED 31b is arranged such that the center of the light exit surface approaches the center of the predetermined range as the center of the arrangement direction of the first LEDs 31b is approached. Also in this example, as shown in FIG. 9, the apparent dimension of the light exit surface formed by the first LEDs 31b-1 to 64 is K' longer than K. As shown in FIG.

As shown in FIG. 8, a diffusing unit 31g can be provided between the light emitting surface of the first LED 31b and the first LCD 31d to diffuse the light emitted from the light emitting surface and enter the first LCD 31d. As a result, even if the dimension K of the light exit surface of the first LED 31b is equal to or less than the dimension of the half cycle of the wave formed on the first LCD 31d, the light incident form on the first LCD 31d is such that the dimension of the light exit surface of the first LCD 31d is the same as that of the wave. Since the incident pattern is longer than the half-cycle dimension, high-frequency undulations are less likely to occur in the image to be inspected. The diffusing means 31g can be, for example, a diffusing plate or a diffusing lens. When the diffusing means 31g is provided, pixel waviness is suppressed by the diffusing effect. may be placed.

(LED drive circuit and LCD drive circuit)
As shown in FIG. 13, the first light projection unit 31 includes a first LED drive circuit (light source drive circuit) 31e that drives the first LED 31b, and a first LCD drive circuit (liquid crystal panel drive circuit) 31f that drives the first LCD 31d. is provided. The first LED drive circuit 31 e is a circuit for changing the supply current value to the first LED 31 b and is controlled by the light projection control section 39 . Therefore, the first LED 31b is controlled by the light projection control section 39 via the first LED driving circuit 31e. Current value control by the first LED drive circuit 31e is DAC control.

The first LCD drive circuit 31f is a circuit for changing the orientation of the liquid crystal composition of each segment SEG by changing the voltage applied to each segment SEG (shown in FIG. 9) constituting the first LCD 31d. In this embodiment, as an example is shown in FIG. 16, there are 64 segments SEG constituting the first LCD 31d, and the voltage applied to each of the 64 segments SEG can be individually changed. It is possible to switch between a state in which the light emitted from the first LED 31b is allowed to pass and a state in which the light emitted from the first LED 31b is not allowed to pass for each segment SEG. The first LCD drive circuit 31f is controlled by a light projection control section 39 shared with the first LED drive circuit 31e. Therefore, the first LCD 31d is controlled by the light projection control section 39 via the first LCD drive circuit 31f. Further, since the first LED driving circuit 31e and the first LCD driving circuit 31f are controlled by the common light projection control section 39, the first LED driving circuit 31e and the first LCD driving circuit 31f can be driven precisely.

The first LCD driving circuit 31f controlled by the light projection control unit 39 drives the first LCD 31d to receive diffused light emitted from the first LED 31b and sequentially generate a plurality of first measurement pattern lights having different patterns. Then, the object W to be measured can be irradiated with the light. The plurality of first measurement pattern lights include pattern light for a spatial code (gray code) used in the spatial code method and pattern light having a periodic illuminance distribution used in the phase shift method.

The upper part of FIG. 16 shows the case where the pattern light for the spatial code is generated by the first LCD 31d, and the lower part of FIG. 16 shows the pattern light having a periodic illuminance distribution used in the phase shift method. indicates the case. In FIG. 16, the black portion is the segment SEG that does not pass the diffused light emitted from the first LED 31b, and the white portion is the segment SEG that allows the diffused light emitted from the first LED 31b to pass. Also, FIG. 16 shows a case where 64 segments SEG constituting the first LCD 31d are arranged in the horizontal direction of the figure.

In the case of the generation of the pattern light for the spatial code on the upper side of FIG. 16, a stripe pattern is generated in which the stripe width becomes finer to half, 1/4, . . . there is By controlling the first LCD 31d in this manner, it is possible to sequentially generate the pattern light for the spatial code.

In the case of generating pattern light for the phase shift method in the lower part of FIG. 16, a plurality of sinusoidal fringe patterns are generated with the phases shifted. In the case of this example, the LCD display is binary controlled to generate a rectangular wave pattern. However, as shown in FIG. A sinusoidal pattern can be obtained. More specifically, a pattern close to a sine wave can be obtained by combining the rectangular wave pattern formed on the liquid crystal panel and the light emitting pattern of the light emitting diodes having an area. If we had an ideal point or line light source instead of an LED, we would get a binary pattern instead of a sinusoidal pattern. Therefore, in order to obtain a sinusoidal pattern, the balance between the LED light source size and the LCD aperture size is important. Also, in this example, eight pattern lights are generated. By controlling the first LCD 31d in this manner, it is possible to sequentially generate pattern light for the phase shift method.

That is, the light projection control section 39 controls the first LED 31b and the first LCD 31d so as to sequentially generate a plurality of pattern lights according to the phase shift method and/or the spatial code method. When the projection of one pattern light among the plurality of pattern lights is completed, the next pattern light is projected. By repeating this process, all the pattern lights are projected. The pattern forming process by the first LCD 31d will be described later.

The number of pattern lights for the spatial code and the number of pattern lights for the phase shift method are not limited to the numbers shown.

(Configuration of the second light projecting section 32)
As shown in FIG. 1, the light emission range of the second LED 32b of the second light projecting portion 32 is at least on the first side portion 30A side of the central axis A of the opening portion 30a of the illumination housing 30 from directly below the second LED 32b (illumination left side of device 3). That is, the light emission range of the second LED 32b of the second light projecting section 32 is the light emission range of the first LED 31b of the first light projecting section 31 with the central axis A of the opening 30a of the illumination housing 30 as the center of symmetry. It is designed to be bilaterally symmetrical. The light emission range of the second LED 32b is indicated by diagonal lines sloping downward to the right in FIG.

As shown in FIG. 13, the second light projection unit 32 includes a second LED drive circuit (light source drive circuit) 32e that drives the second LED 32b, and a second LCD drive circuit (liquid crystal panel drive circuit) 32f that drives the second LCD 32d. are provided, and these are controlled by the light projection control section 39 . Since the second LCD 32d is driven in the same manner as the first LCD 31d, the second LCD 32d receives the diffused light emitted from the second LED 32b, sequentially generates a plurality of second measurement pattern lights having different patterns, and irradiates the object W to be measured. be able to. The plurality of second pattern lights for measurement include pattern light for the spatial code and pattern light for the phase shift method.

The pattern light emitted from the first light projection part 31 and the pattern light emitted from the second light projection part 32 are substantially the same so that they intersect on the central axis A of the opening 30 a of the illumination housing 30 . The first light projecting part 31 and the second light projecting part 32 are integrally supported by the lighting housing 30 while being separated from each other in the circumferential direction of the central axis A so as to have a spread angle. “Integratedly supported” means that the first light projecting unit 31 and the second light projecting unit 32 are supported so that the relative positional relationship between the first light projecting unit 31 and the second light projecting unit 32 does not change during installation or use. That is, the second light projecting part 32 is fixed with respect to the lighting housing 30 . Therefore, during operation, since the relative positions of the first light projecting part 31 and the second light projecting part 32 in the illumination housing 30 do not change, for example, as shown in FIG. If l is set as the distance between the center of the first LED 31b and the center of the second LED 32b during operation, the distance between the center of the first LED 31b and the center of the second LED 32b is fixed to l. The distance between the central portion of the first LED 31b and the central portion of the second LED 32b is the relative position information of the first light projecting portion 31 and the second light projecting portion 32 in the lighting housing 30. 2 can be stored. It should be noted that the separation distance between the central portion of the first LED 31b and the central portion of the second LED 32b can be changed when not in operation.

Further, the relative positional information of the first light projecting portion 31 and the second light projecting portion 32 in the lighting housing 30 may be a straight line distance between the center of the first LED 31b and the center of the second LED 32b. When the light emitted from the LED is turned back by a mirror or the like to irradiate the object W to be measured, the distance can be set in consideration of the path length of the light.

Since the first LCD 31 d is arranged on the left side of the illumination device 3 , it projects pattern light from the left side onto the measurement object W placed on the placement surface 100 . Further, since the second LCD 32d is arranged on the right side of the lighting device 3, it projects pattern light onto the measurement object W placed on the placement surface 100 from the right side. The first LCD 31d and the second LCD 32d are liquid crystal panels that project pattern light onto the object W to be measured from different directions.

(Configuration of the third light projecting section 33 and the fourth light projecting section 34)
The third light projecting section 33 and the fourth light projecting section 34 are configured in the same manner as the first light projecting section 31. As shown in FIG. ) 33b, and a third LCD (third pattern light generator) 33d arranged corresponding to the third LED 33b. and a fourth LCD (fourth pattern light generating section) 34d arranged in a line. The third LED 33b and the fourth LED 34b are paired. The third LED 33b and the fourth LED 34b are attached to the lighting housing 30 so that their relative positions can be corrected. Details of relative position correction will be described later.

The display surfaces of the third LCD 33d and the fourth LCD 34d are positioned on the same plane as the plane perpendicular to the central axis A of the opening 30a of the illumination housing 30 (the plane denoted by reference numeral 200 in FIG. 6).

The light emission range of the third LED 33b of the third light projecting section 33 and the light emission range of the fourth LED 34b of the fourth light projecting section 34 are defined by the light emission range of the first LED 31b of the first light projecting section 31 and the light emission range of the fourth LED 34b. It is set to be the same as the relationship with the light emission range of the second LED 32 b of the second light projecting section 32 . Specifically, the light emission range of the third LED 33b of the third light projecting portion 33 is set so as to reach at least the fourth side portion 30D side of the central axis A of the opening 30a of the lighting housing 30 from directly below the third LED 33b. is set. The light emission range of the fourth LED 34b of the fourth light projecting portion 34 is set to reach at least the third side portion 30C side of the central axis A of the opening 30a of the lighting housing 30 from directly below the fourth LED 34b. Therefore, when the central axis A of the opening 30a of the illumination housing 30 is taken as the center of symmetry, the light emission range of the third LED 33b of the third light projecting part 33 and the light emission range of the fourth light projecting part 34 are vertically symmetrical. and the light emission range of the fourth LED 34b are set.

As shown in FIG. 13, the third light projection unit 33 includes a third LED drive circuit (light source drive circuit) 33e that drives the third LED 33b, and a third LCD drive circuit (liquid crystal panel drive circuit) 33f that drives the third LCD 33d. are provided, and these are controlled by the light projection control section 39 . Since the third LCD 33d is driven in the same manner as the first LCD 31d, the third LCD 33d receives the diffused light emitted from the third LED 33b, sequentially generates a plurality of third measurement pattern lights having mutually different patterns, and irradiates the object W to be measured. be able to. The plurality of third measurement pattern lights include pattern light for spatial code and pattern light for phase shift method.

The fourth light projecting section 34 is provided with a fourth LED driving circuit (light source driving circuit) 34e for driving the fourth LED 34b and a fourth LCD driving circuit (liquid crystal panel driving circuit) 34f for driving the fourth LCD 34d. , are controlled by the projection control unit 39 . Since the fourth LCD 34d is driven in the same manner as the first LCD 31d, the fourth LCD 34d receives diffused light emitted from the fourth LED 34b, sequentially generates a plurality of fourth measurement pattern lights having different patterns, and irradiates the object W to be measured. be able to. The plurality of fourth measurement pattern lights includes pattern light for spatial code and pattern light for phase shift method.

The pattern light emitted from the third light projecting part 33 and the pattern light emitted from the fourth light projecting part 34 are substantially the same so that they intersect on the central axis A of the opening 30 a of the illumination housing 30 . The third light projecting portion 33 and the fourth light projecting portion 34 are integrally supported by the lighting housing 30 while being separated from each other in the circumferential direction of the central axis A so as to have a spread angle. Therefore, since the relative positions of the third light projecting portion 33 and the fourth light projecting portion 34 in the lighting housing 30 do not change, the distance between the center of the third LED 33b and the center of the fourth LED 34b is set to a predetermined value in advance. If set, the separation distance between the center of the third LED 33b and the center of the fourth LED 34b is fixed at a predetermined value during operation. The distance between the central portion of the third LED 33b and the central portion of the fourth LED 34b is the relative position information of the third light projecting portion 33 and the fourth light projecting portion 34 in the lighting housing 30. 2 can be stored.

Since the third LCD 33d is arranged above the illumination device 3, it projects pattern light onto the measurement object W placed on the placement surface 100 from that direction. Further, since the fourth LCD 34d is arranged below the illumination device 3, it projects pattern light onto the measurement object W placed on the placement surface 100 from that direction. The third LCD 33d and the fourth LCD 34d are liquid crystal panels that project pattern light onto the object W to be measured from different directions.

The fifth to eighth light projecting sections 35 to 38 shown in FIG. 5 are constructed similarly to the first to fourth light projecting sections 31 to 34. As shown in FIG.

(Control by light projection control unit 39)
As shown in FIG. 13, in this embodiment, a first LED driving circuit 31e, a second LED driving circuit 32e, a third LED driving circuit 33e, a fourth LED driving circuit 34e, a first LCD driving circuit 31f, a second LCD driving circuit 32f, and a third LCD Since the drive circuit 33f and the fourth LCD drive circuit 34f are controlled by the common light projection control section 39, these drive circuits can be precisely synchronized. Similarly, the fifth to eighth light projection units 35 to 38 also have LED drive circuits and LCD drive circuits, which can be precisely synchronized.

The light projection control unit 39 performs at least the following operations until projection of one of the plurality of pattern lights is completed from any one of the first LCD 31d, the second LCD 32d, the third LCD 33d, and the fourth LCD 34d. A pattern to be projected next is completed on another liquid crystal panel that projects pattern light, and after the projection of the pattern light by the one liquid crystal panel is completed, the next pattern is projected onto the other liquid crystal panel. The first LCD 31d, the second LCD 32d, the third LCD 33d, and the fourth LCD 34d are controlled so as to repeat the process of projecting light.

Specifically, the light projection control unit 39 of the illumination device 3 receives from the controller unit 4 a trigger signal for starting projection of the pattern light and a replay signal for synchronizing with the imaging device 2 during projection of the pattern light. A synchronous trigger signal is input. A trigger signal may be input from the PLC 101 . For example, a trigger signal can be input to the light projection control unit 39 based on the detection result of a photoelectric sensor or the like connected to the PLC 101 . The device that generates the trigger signal may not be the PLC 101, but may be a photoelectric sensor or the like. In this case, a photoelectric sensor or the like can be directly connected to the light projection control section 39 or connected via the controller section 4 .

When the trigger signal is input, the light projection control unit 39 switches the pattern currently formed on the first LCD 31d to a pattern different from the current display mode via the first LCD driving circuit 31f. 1 LCD 31d is controlled. Here, in order to switch the pattern on the first LCD 31d, the first LED driving circuit 31e changes the voltage applied to the liquid crystal composition of each segment constituting the first LCD 31d by a well-known technique. The time from when the voltage applied to the liquid crystal composition is changed until the orientation of the liquid crystal composition is changed is longer than the imaging interval of the imaging device 2, which will be described later. In other words, in order to switch the pattern currently formed on the first LCD 31d to a different pattern, a predetermined pattern switching time that is longer than the imaging interval of the imaging device 2 is required, and thus switching from one pattern to another pattern is required. become. Similarly, the second LCD 32d, the third LCD 33d and the fourth LCD 34d also require time for sequentially switching patterns.

While the pattern on the first LCD 31d is completely formed, control is performed so that light is emitted from the first LED 31b in synchronization with the formation of the pattern, and light is not emitted from the second LED 32b, the third LED 33b and the fourth LED 34b. . As a result, only the pattern formed on the first LCD 31d is projected onto the measurement object W as pattern light. will not be

The time during which the pattern is formed on the first LCD 31d is part of the pattern switching time for forming the pattern on the second LCD 32d. The time for forming the pattern on the second LCD 32d is longer than the time for forming the pattern on the first LCD 31d. ing.

When the image of the pattern light of the pattern projected onto the object W to be measured is completed, while the pattern is completely formed on the second LCD 32d, light is emitted from the second LED 32b in synchronization with the formation of the pattern. 1 LED 31b, third LED 33b and fourth LED 34b are controlled so as not to emit light. As a result, only the pattern formed on the second LCD 32d is projected onto the measurement object W as pattern light.

The time during which the pattern is formed on the second LCD 32d is part of the switching time for forming the pattern on the third LCD 33d. The time to form the pattern on the third LCD 33d is longer than the time to form the pattern on the second LCD 32d. ing.

When the image of the pattern light of the pattern projected onto the object W to be measured is completed, while the pattern is completely formed on the third LCD 33d, light is emitted from the third LED 33b in synchronization with the formation of the pattern. 1 LED 31b, second LED 32b and fourth LED 34b are controlled so as not to emit light. As a result, only the pattern formed on the third LCD 33d is projected onto the measuring object W as pattern light.

The time during which the pattern is formed on the third LCD 33d is part of the switching time for forming the pattern on the fourth LCD 34d. The time to form the pattern on the fourth LCD 34d is longer than the time to form the pattern on the third LCD 33d. ing.

When the image of the pattern light of the pattern projected onto the object W to be measured is completed, while the pattern is completely formed on the fourth LCD 34d, light is emitted from the fourth LED 34b in synchronization with the formation of the pattern. 1 LED 31b, second LED 32b and third LED 33b are controlled so as not to emit light. As a result, only the pattern formed on the fourth LCD 34d is projected onto the measuring object W as pattern light. Part of the time during which this pattern is formed is part of the switching time for forming the next pattern on the first LCD 31d.

That is, in this embodiment, one of the first LCD 31d, the second LCD 32d, the third LCD 33d, and the fourth LCD 34d does not successively project a plurality of pattern lights by one liquid crystal panel. When the projection of the pattern light is completed, the first pattern light is projected by another liquid crystal panel, and when the projection of the first pattern light by the another liquid crystal panel is completed, the first pattern is projected by another liquid crystal panel. When the light is projected and the projection of the first pattern light is completed in all the liquid crystal panels, the second pattern light is projected by the one liquid crystal panel, and the second pattern light is projected by the one liquid crystal panel. When the projection of the second pattern light is completed, the second pattern light is projected by another liquid crystal panel, and when the projection of the second pattern light by the another liquid crystal panel is completed, the second pattern light is projected by another liquid crystal panel. The first LCD 31d, the second LCD 32d, the third LCD 33d, and the fourth LCD 34d are controlled so as to project the pattern light. Therefore, the liquid crystal panel on which pattern light is not projected can be prepared for forming a pattern to be projected next, so that the slow response speed of the liquid crystal panel can be compensated for.

In the above example, the case where the pattern light is projected by all of the first LCD 31d, the second LCD 32d, the third LCD 33d, and the fourth LCD 34d has been described. The pattern light can also be projected only by the 3LCD 33d and the 4th LCD 34d. When the pattern light is projected only by the first LCD 31d and the second LCD 32d, the pattern light may be projected alternately. For example, while the first pattern light is being projected by the first LCD 31d, A first pattern formation process is performed, and then a second pattern formation process is performed on the first LCD 31d while the first pattern light is being projected on the second LCD 32d, and this is repeated. The same is true when pattern light is projected only by the third LCD 33d and the fourth LCD 34d.

In addition to the trigger signal and the resynchronization trigger signal, pattern light formation information is also transmitted from the controller 4 to the light projection control section 39 , and the transmitted pattern light formation information is temporarily sent to the light projection control section 39 . The first LED 31b, the second LED 32b, the third LED 33b and the fourth LED 34b, and the first LCD 31d, the second LCD 32d, the third LCD 33d and the fourth LCD 34d are controlled based on the stored pattern light formation information.

The pattern light formation information includes, for example, the irradiation mode, the presence or absence of irradiation of the pattern light for the spatial code, the specific pattern and number of the pattern light for the spatial code, the presence or absence of the irradiation of the pattern light for the phase shift method, The specific pattern and number of pattern lights for the phase shift method, the irradiation order of the pattern lights, and the like are included. The irradiation modes include a first irradiation mode in which only the first LCD 31d and the second LCD 32d irradiate the pattern light and project it onto the measurement object W, and a first irradiation mode in which the pattern light is projected on the measurement object W, and all of the first LCD 31d, the second LCD 32d, the third LCD 33d, and the fourth LCD 34d project the pattern light. A second irradiation mode and a third irradiation mode in which pattern light is irradiated only by the third LCD 33d and the fourth LCD 34d and projected onto the measurement object W are included.

(Configuration of imaging device 2)
As shown in FIG. 1 and the like, the imaging device 2 is provided separately from the illumination device 3 . As shown in FIG. 1, the imaging device 2 and the controller section 4 are connected by a connection line 2a, but the imaging device 2 and the controller section 4 may be wirelessly connected.

Since the imaging device 2 constitutes a part of the image processing device 1, it can also be called an imaging unit. Since the imaging device 2 is provided separately from the lighting device 3, the imaging device 2 and the lighting device 3 can be installed separately. Therefore, the installation location of the imaging device 2 and the installation location of the lighting device 3 can be changed, and the installation location of the imaging device 2 and the installation location of the lighting device 3 can be separated. As a result, the degree of freedom in installing the imaging device 2 and the lighting device 3 can be greatly improved, and the image processing device 1 can be introduced to any production site or the like.

In addition, if the installation location of the imaging device 2 and the installation location of the lighting device 3 can be the same at the site, the imaging device 2 and the lighting device 3 can be attached to the same member, and the installation state can be determined by the user. It can be arbitrarily changed according to the site. Also, the imaging device 2 and the lighting device 3 can be attached to the same member and used as an integrated unit.

The imaging device 2 is arranged above the lighting housing 30 of the lighting device 3, that is, on the side opposite to the direction in which the pattern light is emitted, so as to look into the opening 30a of the lighting housing 30. As shown in FIG. Therefore, the imaging device 2 receives the first measurement pattern light reflected from the measurement object W through the opening 30a of the illumination housing 30 of the illumination device 3 to generate a plurality of first pattern images, A plurality of second pattern images can be generated by receiving the second measurement pattern light reflected from the measurement object W. FIG. Further, when the lighting device 3 has the third light projecting section 33 and the fourth light projecting section 34 , the image capturing device 2 projects the measurement target through the opening 30 a of the lighting housing 30 of the lighting device 3 . A plurality of third pattern images are generated by receiving the third pattern light for measurement reflected from the object W, and a plurality of fourth pattern images are generated by receiving the fourth pattern light for measurement reflected from the object W to be measured. can be generated. Similarly, when the fifth to eighth light projection units 35 to 38 are provided, the fifth to eighth pattern images can be generated.

As shown in FIG. 3, the imaging device 2 includes a lens 21 that forms an optical system, and an imaging device 22 that is a light-receiving device that receives light incident from the lens 21 . , a so-called camera is configured. The lens 21 is a member for forming an image of at least a height measurement area or an inspection target area on the measurement object W on the imaging device 22 . The optical axis of the lens 21 may be aligned with the central axis A of the opening 30a of the illumination housing 30 of the lighting device 3, but they do not have to be aligned. In addition, the distance between the imaging device 2 and the lighting device 3 in the direction of the central axis A can be set arbitrarily within a range in which the lighting device 3 does not interfere with the imaging by the imaging device 2, and is designed with a high degree of freedom in installation. there is

Also, as the imaging element 22, a CCD, a CMOS sensor, or the like can be used. The imaging device 22 receives the reflected light from the object W to be measured, acquires an image, and outputs the acquired image data to the data processing unit 24 . In this example, a high resolution CMOS sensor is used as the imaging device 22 . It is also possible to use an imaging device capable of capturing images in color. The imaging device 22 can also capture a normal luminance image in addition to the pattern projection image. When capturing a normal luminance image, all the LEDs 31b, 32b, 33b, and 34b of the illumination device 3 are turned on, and all the LCDs 31d, 32d, 33d, and 34d are controlled so as not to form pattern light. good. If there is an observation illumination 50 as shown in FIGS. 5 and 6, it is also possible to take a normal luminance image with the imaging device 2 using it.

The imaging device 2 further includes an exposure control unit 23, a data processing unit 24, a phase calculation unit 26, an image processing unit 27, an image storage unit 28, and an output control unit 29, in addition to the camera. . The data processing unit 24, the phase calculation unit 26, the image processing unit 27, and the image storage unit 28 are connected to a common bus line 25 built in the imaging device 2, and are capable of transmitting and receiving data to each other. . The exposure control section 23, the data processing section 24, the phase calculation section 26, the image processing section 27, the image storage section 28 and the output control section 29 can be configured by hardware or by software.

(Configuration of Exposure Control Unit 23)
A trigger signal for starting imaging and a resynchronization trigger signal for synchronizing with the illumination device 3 during imaging are input to the exposure control section 23 from the controller section 4 . Input timings of the trigger signal and the resynchronization trigger signal input to the exposure control unit 23 are set so as to be the same timings as the trigger signal and the resynchronization trigger signal input to the illumination device 3 .

The exposure control unit 23 is a part that directly controls the imaging device 22, and controls the imaging timing and exposure time of the imaging device 22 by the trigger signal and the resynchronization trigger signal input to the exposure control unit 23. The exposure control unit 23 is configured to receive and store information about imaging conditions from the controller unit 4 . The information about the imaging conditions includes, for example, the number of imaging times, the imaging interval (the time after imaging until the next imaging), the exposure time (shutter speed) at the time of imaging, and the like.

The exposure control section 23 causes the imaging device 22 to start imaging in response to the input of the trigger signal sent from the controller section 4 . In this embodiment, since it is necessary to generate a plurality of pattern images by inputting a single trigger signal, a resynchronization trigger signal is input from the controller unit 4 during imaging. Synchronization with the device 3 is possible.

Specifically, while the pattern completely formed on the first LCD 31 d is projected onto the measurement object W as pattern light, the exposure control unit 23 takes an image so that the image sensor 22 takes an image (exposure). Control the element 22 . The exposure time and the time during which the pattern is projected onto the object W to be measured as pattern light can be the same, but the timing at which the exposure is started can be slightly later than the timing at which the pattern light is started to be projected. good too.

After that, while the pattern formed on the second LCD 32d is projected onto the object W to be measured as pattern light, the exposure control unit 23 controls the imaging device 22 so that the imaging device 22 takes an image. When this imaging is completed, the exposure control unit 23 controls the imaging device 22 so that the imaging device 22 takes an image while the pattern formed on the third LCD 33d is projected onto the measurement object W as pattern light. do. After that, while the pattern formed on the fourth LCD 34d is projected onto the object W to be measured as pattern light, the exposure control unit 23 controls the imaging device 22 so that the imaging device 22 takes an image. By repeating this, a plurality of first pattern images, a plurality of second pattern images, a plurality of third pattern images, and a plurality of fourth pattern images are generated.

The imaging device 22 transfers the image data to the data processing unit 24 each time it completes imaging. The image data can be stored in the image storage section 28 shown in FIG. That is, since the image pickup timing by the image pickup device 22 and the image request timing of the controller unit 4 do not match, the image storage unit 28 functions as a buffer that absorbs this discrepancy.

The image data is transferred to the data processing unit 24 shown in FIG. 3 between the imaging and the next imaging. . When the image of the measurement object W irradiated with a certain pattern of light is completed, the image data of the previous pattern is stored while the image of the measurement object W irradiated with the next pattern of light is being imaged. Transfer to the processing unit 24 . In this manner, the image data captured last time can be transferred to the data processing unit 24 at the time of the next image capturing.

Further, it is also possible to image the measurement object W irradiated with pattern light of a certain pattern a plurality of times. In this case, the first LED 31b can be lit only when the imaging element 22 takes an image. The exposure time of the imaging element 22 can be set so that the first imaging is longer than the second imaging, and the second imaging is set to be longer than the first imaging. You can also When capturing an image of the measurement object W irradiated with pattern light of other patterns, the image can be captured multiple times. Thereby, while one pattern light among the plurality of pattern lights is being projected onto the measurement object W, a plurality of images with different exposure times can be generated. A plurality of images with different exposure times are used when performing high dynamic range processing, which will be described later. It should be noted that the first LED 31b may be kept on while the measurement object W irradiated with the pattern light of a certain pattern is imaged a plurality of times.

(Configuration of data processing unit 24)
The data processing unit 24 shown in FIG. 3 generates a plurality of pattern image sets based on the image data output from the imaging element 22. FIG. When the imaging device 22 generates a plurality of first pattern images, the data processing section 24 generates a first pattern image set consisting of a plurality of first pattern images. Similarly, generating a second pattern image set consisting of a plurality of second pattern images, generating a third pattern image set consisting of a plurality of third pattern images, and generating a fourth pattern image set consisting of a plurality of fourth pattern images to generate Therefore, the imaging device 2 can receive the reflected light from the measurement object W of the plurality of pattern lights projected from each liquid crystal panel, and generate a plurality of pattern image sets corresponding to each liquid crystal panel. .

When pattern light is projected only by the first LCD 31d and the second LCD 32d, a first pattern image set and a second pattern image set are generated. When pattern light is projected only by the third LCD 33d and the fourth LCD 34d, a third pattern image set and a fourth pattern image set are generated.

The data processing unit 24 can generate a phase shift pattern image set by projecting pattern light according to the phase shift method, and can generate a gray code pattern image set by projecting pattern light according to the spatial code method.

The pattern light according to the phase shift method is pattern light whose illuminance distribution is varied, for example, in a sinusoidal shape, but other pattern light may be used. In this embodiment, the number of pattern lights according to the phase shift method is eight, but the number is not limited to this. On the other hand, the pattern light according to the spatial code method is a striped pattern in which the striped width is reduced to half, quarter, . In this embodiment, the number of pattern lights according to the spatial code method is four, but the number is not limited to this. The pattern explained in this example is a case where the gray code is used as a spatial code, and it is not the purpose of the gray code to form pattern light by dividing the stripe width in half, but as a result It just happens. Also, the Gray code is a kind of code system in which noise resistance is taken into account by setting the Hamming distance to 1 between adjacent codes.

As shown in FIG. 15, when the first light projecting unit 31 of the illumination device 3 irradiates the measurement object W with four pattern lights according to the spatial code method, the data processing unit 24 is composed of four different images. Generate a set of gray code pattern images. Further, when the first light projecting unit 31 of the illumination device 3 irradiates the measuring object W with eight pattern lights according to the phase shift method, the data processing unit 24 generates a phase shift pattern image set consisting of eight different images. to generate Both the gray code pattern image set and the phase shift pattern image set obtained by irradiation of the pattern light by the first light projecting section 31 are the first pattern image set.

Similarly, when the second light projecting unit 32 of the illumination device 3 irradiates the object W to be measured with pattern light according to the spatial code method, a gray code pattern image set is generated and a pattern according to the phase shift method is generated. When light is applied to the measurement object W, a phase shift pattern image set is generated. Both the gray code pattern image set and the phase shift pattern image set obtained by irradiation of the pattern light by the second light projecting section 32 are the second pattern image set.

Similarly, when the third light projecting unit 33 of the illumination device 3 irradiates the object W to be measured with pattern light according to the spatial code method, a gray code pattern image set is generated and a pattern according to the phase shift method is generated. When light is applied to the measurement object W, a phase shift pattern image set is generated. Both the gray code pattern image set and the phase shift pattern image set obtained by irradiation of the pattern light by the third light projecting section 33 are the third pattern image set.

Similarly, when the fourth light projecting unit 34 of the illumination device 3 irradiates the object W to be measured with pattern light according to the spatial code method, a gray code pattern image set is generated and a pattern according to the phase shift method is generated. When light is applied to the measurement object W, a phase shift pattern image set is generated. Both the gray code pattern image set and the phase shift pattern image set obtained by irradiation of the pattern light by the fourth light projecting section 34 are the fourth pattern image set.

Each pattern image set can be stored in the image storage unit 28 shown in FIG.

As shown in FIG. 3, the data processing section 24 has an HDR processing section 24a. HDR processing is high dynamic range imaging synthesis processing, and the HDR processing unit 24a synthesizes a plurality of images with different exposure times. That is, as described above, when the measurement object W irradiated with pattern light of a certain pattern is imaged a plurality of times with different exposure times, a plurality of luminance images with different exposure times are obtained. By synthesizing these luminance images, an image having a dynamic range wider than that of each luminance image can be generated. A conventionally well-known method can be used for the HDR synthesis method. Instead of changing the exposure time, a plurality of luminance images with different brightness may be obtained by changing the intensity of the radiated light, and these luminance images may be synthesized.

(Configuration of Phase Calculation Unit 26)
The phase calculator 26 shown in FIG. 3 is a part that calculates an absolute phase image that is the original data of the height image. As shown in FIG. 14, in step SA1, each image data of the phase shift pattern image set is acquired and relative phase calculation processing is performed by using the phase shift method. This can be expressed as a relative phase (pre-unwrapping phase) in FIG. 16, and a phase image is obtained by the relative phase calculation processing in step SA1.

On the other hand, at step SA3 in FIG. 14, each image data of the gray code pattern image set is obtained, and the spatial code calculation process is performed by using the spatial code method to obtain the stripe number image. The fringe number image is an image that is made identifiable by assigning a series of space code numbers to the small spaces when the space irradiated with light is divided into a large number of small spaces. FIG. 16 shows how to assign a series of spatial code numbers.

At step SA4 in FIG. 14, absolute phase phasing processing is performed. In the absolute phase phasing process, the phase image obtained in step SA1 and the fringe number image obtained in step SA3 are synthesized (unwrapped) to generate an absolute phase image (intermediate image). In other words, the spatial code number obtained by the spatial code method can be used to correct the phase jump (phase unwrap) by the phase shift method, so it is possible to obtain high-resolution measurement results while ensuring a wide dynamic range of height. .

Height measurement may be performed only by the phase shift method. In this case, the height measurement dynamic range is narrowed, so that the height cannot be measured correctly in the case of an object W having a large difference in height such that the phase shifts by one period or more. Conversely, in the case of the object W to be measured that has a small change in height, the striped image is not picked up or synthesized by the spatial code method, so there is the advantage of speeding up the processing accordingly. For example, when measuring an object W that has little difference in the height direction, it is not necessary to take a large dynamic range. can be shortened. Also, since the absolute height is known, the height may be measured only by the spatial code method. In this case, the accuracy can be improved by increasing the code.

Further, at step SA2 in FIG. 14, each image data of the phase shift pattern image set is acquired, and reliability image calculation processing is performed. In the reliability image calculation process, a reliability image indicating phase reliability is calculated. This is an image that can be used to determine invalid pixels.

The phase image, fringe number image and reliability image can be stored in the image storage unit 28 shown in FIG.

The absolute phase image generated by the phase calculator 26 can also be said to be an angle image in which each pixel has irradiation angle information of the pattern light for measurement onto the object W to be measured. That is, since the first pattern image set (phase shift pattern image set) includes eight first pattern images obtained by shifting the phase of the sinusoidal fringe pattern, the phase shift method can be used. Thus, each pixel has irradiation angle information of the pattern light for measurement to the object W to be measured. In other words, the phase calculator 26 is a part that generates a first angle image in which each pixel has irradiation angle information of the first measurement pattern light onto the measurement object W based on a plurality of first pattern images. , can also be called an angle image generator. The first angle image is an image of the angle of the light emitted from the first LED 31b to the object W to be measured.

Similarly, the phase calculator 26 calculates a second angle image in which each pixel has irradiation angle information of the second measurement pattern light on the measurement object W based on the plurality of second pattern images, and a plurality of third pattern images. A third angle image in which each pixel has irradiation angle information of the third measurement pattern light to the measurement object W based on the image, A fourth angle image having irradiation angle information of the four measurement pattern lights can be generated. The second angle image is an image of the angle of the light emitted from the second LED 32b to the object W to be measured. The third angle image is an image obtained by imaging the angle of the light emitted from the third LED 33b to the object W to be measured. The fourth angle image is an image of the angle of the light emitted from the fourth LED 34b to the object W to be measured. The uppermost image of the intermediate images in FIG. 15 is the first angle image, the second image from the top is the second angle image, the third image from the top is the third angle image, and the bottom image is the third angle image. The image is the fourth angle image. The parts of each angle image that appear to be blacked out are the parts that are shaded by the lighting (each of the LEDs), and are invalid pixels without angle information.

(Configuration of image processing unit 27)
The image processing unit 27 is a part that performs image processing such as gamma correction, white balance adjustment, and gain correction on each of the pattern images, phase images, stripe number images, and reliability images. Each pattern image, phase image, fringe number image, and reliability image after image processing can be stored in the image storage unit 28 . Image processing is not limited to the processing described above.

(Configuration of output control unit 29)
Upon receiving the image output request signal output from the controller unit 4, the output control unit 29 selects the image instructed by the image output request signal among the images stored in the image storage unit 28 according to the image output request signal. is a part that outputs to the controller unit 4 via the image processing unit 27 . In this example, each pattern image, phase image, fringe number image and reliability image before image processing are stored in the image storage unit 28, and an image requested by an image output request signal from the controller unit 4 is processed. Only the image processing section 27 performs image processing and outputs the data to the controller section 4 . The image output request signal can be output when the user performs various measurement operations or inspection operations.

In this embodiment, the data processing section 24 , the phase calculation section 26 and the image processing section 27 are provided in the imaging device 2 , but they may also be provided in the controller section 4 . In this case, the image data output from the imaging device 22 is output to the controller section 4 and processed.

(Configuration of controller section 4)
As shown in FIG. 2, the controller unit 4 includes an imaging projection control unit 41, a height measurement unit 42, an image synthesizing unit 43, an inspection unit 45, a display control unit 46, and a history storage unit 47. I have. The controller unit 4 is provided separately from the imaging device 2 and the lighting device 3 .

(Structure of imaging projection control section 41)
The imaging light projection control unit 41 outputs the pattern light formation information, the trigger signal, and the resynchronization trigger signal to the lighting device 3 at a predetermined timing, and outputs the information regarding the imaging condition, the trigger signal, and the resynchronization trigger signal. , to the imaging device 2 at a predetermined timing. The trigger signal and resynchronization trigger signal output to the illumination device 3 are synchronized with the trigger signal and resynchronization trigger signal output to the imaging device 2 . The formation information of the pattern light and the information on the imaging conditions can be stored, for example, in the imaging projection control section 41 or another storage section (not shown). When the user performs a predetermined operation (height measurement preparation operation, inspection preparation operation), the pattern light formation information is output to the illumination device 3 and temporarily stored in the light projection control unit 39 of the illumination device 3, Further, the information regarding the imaging conditions is output to the imaging device 2 and temporarily stored in the exposure control section 23 . In this example, the lighting device 3 can be called a smart type lighting device because the lighting device 3 is configured to control the LED and the LCD by the light projection control unit 39 built in the lighting device 3. In addition, since the imaging device 2 is configured to control the imaging element 22 by the exposure control unit 23 built in the imaging device 2, it can be called a smart type imaging device.

In this way, when the imaging device 2 and the lighting device 3 perform separate control, as the number of times of imaging increases, the imaging timing and the lighting timing (projection timing of the pattern light) deviate. There is a problem that the obtained image becomes dark. In particular, as described above, a total of 12 images, 8 images of the phase shift pattern image set and 4 of the gray code pattern image set, constitute the first pattern image set, and similarly constitute the second pattern image set. Furthermore, if HDR imaging is also performed, the number of times of imaging increases, and the difference between the imaging timing and the illumination timing becomes noticeable.

In this example, the resynchronization trigger signal is synchronously output to the illumination device 3 and the imaging device 2, so that the illumination device 3 and the imaging device 2 can be synchronized during imaging. I have to. Therefore, even if the number of imaging times increases, the difference between the imaging timing and the illumination timing is extremely small to the extent that it does not become a problem, and it is possible to suppress the darkening of the image during the irradiation of the phase shift pattern or the gray code pattern. , the possibility of erroneous phase distortion and code determination can be reduced. The resynchronization trigger signal can also be output multiple times.

The imaging light projection control section 41 includes an irradiation mode switching section 41a. A first irradiation mode in which the first pattern light for measurement and the second pattern light for measurement are emitted by the first light projection unit 31 and the second light projection unit 32, respectively, and the first light projection unit 31 and the second light projection unit 32 After irradiating the first pattern light for measurement and the second pattern light for measurement, respectively, the third light projection unit 33 and the fourth light projection unit 34 irradiate the third light pattern for measurement and the fourth pattern light for measurement, respectively. and a third irradiation mode in which the third pattern light for measurement and the fourth pattern light for measurement are emitted by the third light projecting section 33 and the fourth light projecting section 34, respectively. , can be switched to any one illumination mode. The user can switch the irradiation mode by operating the console section 6 or the mouse 7 while viewing the display section 5 . Alternatively, the controller unit 4 can be configured to automatically switch the irradiation mode.

(Configuration of height measuring unit 42)
The height measurement unit 42 measures the irradiation angle information of each pixel of the first angle image and the irradiation angle information of each pixel of the second angle image generated by the phase calculation unit 26, and the first height in the illumination housing 30 of the illumination device 3. The height of the measuring object W in the central axis A direction of the lighting device 3 can be measured according to the relative position information of the light projecting section 31 and the second light projecting section 32 .

A specific method for measuring the height by the height measuring unit 42 will be described below. As described above, the angle from the illumination for each pixel is determined by generating the angle image by unwrapping the phase. The first angle image is an image showing the angle of light emitted from the first LED 31b to the object W to be measured, and the second angle image is an image showing the angle of light emitted from the second LED 32b to the object W to be measured. be. The first LED 31b and the second LED 32b are integrally supported by the illumination housing 30, and the distance between the first LED 31b and the second LED 32b is l (shown in FIG. 17) as described above.

FIG. 17 describes the case of obtaining the height of an arbitrary point H on the object W to be measured. The direction directly below the first LED 31b is 0°, and the direction 45° from the first LED 31b is 1. Moreover, the right direction in FIG. 17 is positive, and the left direction is negative. The angle of light emitted from the first LED 31b to the point H can be obtained from the pixel corresponding to the point H in the first angle image, and the inclination of the straight line connecting the point H and the first LED 31b is 1/a1. Further, the angle of the light irradiated from the second LED 32b to the point H can be obtained from the pixel corresponding to the point H in the second angle image, and the inclination of the straight line connecting the point H and the second LED 32b is -1/a2. do. a1 and a2 are phases.

Z=1/a1*X+0... Formula 1
Z=−1/a2*(X−l) … Formula 2
Solving Z for Equations 1 and 2 gives the height.

a1Z=X
a2Z=-X+l
Z=l/(a1+a2)
X=a1*l/(a1+a2)
In this manner, the height of each point on the object W to be measured can be obtained. Since each of the above equations has no variable relating to the position of the imaging device 2, it can be seen that the position of the imaging device 2 is irrelevant when obtaining the height of each point on the object W to be measured. However, since there is no angle information for pixels that are invalid pixels in the angle image, the height of that point cannot be obtained. That is, the calculated Z-coordinate indicates not the distance between the imaging device 2 and the object W to be measured, but the distance to the object W when viewed from the illumination device 3 . The Z coordinate obtained by the installation position of the illumination device 3 is determined regardless of the installation position of the imaging device 2 .

Also, although not shown, the angle of the light emitted from the third LED 33b to the point H can be similarly obtained from the pixel corresponding to the point H in the third angle image. Since the angle of the projected light can be obtained from the pixel corresponding to the point H in the fourth angle image, the height of each pixel can be obtained based on the third angle image and the fourth angle image.

For example, in FIG. 15, the height measurement unit 42 obtains irradiation angle information of each pixel of the first angle image, irradiation angle information of each pixel of the second angle image, and the first light projection unit 31 and A first height image representing the height of the measurement object W is generated according to the relative position information of the second light projecting unit 32, and the irradiation angle information of each pixel of the third angle image and each of the fourth angle images are generated. When the second height image representing the height of the measurement target W is generated according to the irradiation angle information of the pixels and the relative position information of the third light projecting section 33 and the fourth light projecting section 34 in the illumination housing 30 is shown.

Since the first height image can grasp the height of each pixel, it can be used as an image to be inspected when performing various inspections. Also, since the second height image can also grasp the height of each pixel, it can be used as an inspection object image used when performing various inspections. Therefore, the height measuring unit 42 can also be called an inspection target image generation unit that generates an inspection target image based on a plurality of intermediate images.

In the case shown in FIG. 15 , first, a first angle image is generated by a first pattern image set obtained by projecting pattern light by the first light projecting unit 31 , and a first angle image is generated by projecting the pattern light by the second light projecting unit 32 . A first angle image is generated from the obtained second pattern image set. In the first angle image, since the first light projecting unit 31 irradiates light from the left side of the measurement object W, a shadow is formed on the right side of the measurement object W, and that portion is an invalid pixel. On the other hand, in the second angle image, since the second light projecting unit 32 irradiates light from the right side of the measurement object W, a shadow is formed on the left side of the measurement object W, and that portion becomes an invalid pixel. there is Since the first height image is generated from the first angle image and the second angle image, pixels that are invalid pixels in one of the angle images are also invalid pixels in the first height image.

Similarly, a third angle image is generated from a third pattern image set obtained by projecting the pattern light by the third light projecting unit 33, and a fourth angle image is generated by projecting the pattern light by the fourth light projecting unit 34. Generate a fourth angle image with the pattern image set. In the third angle image, the third light projecting unit 33 irradiates light from the upper side of the measurement object W (upper side in the figure), so that the lower side of the measurement object W (lower side in the figure) side) is cast, and that portion is an invalid pixel. On the other hand, in the fourth angle image, since the fourth light projecting unit 34 irradiates light from the lower side of the measurement object W, a shadow is formed above the measurement object W, and the shadowed portion becomes an invalid pixel. ing. Since the second height image is generated from the third angle image and the fourth angle image, pixels that are invalid pixels in one of the angle images are also invalid pixels in the second height image. In order to reduce invalid pixels as much as possible, in this embodiment, an image synthesizing section 43 is provided in the controller section 4 as shown in FIG.

In this embodiment, the case where the height measuring section 42 is provided in the controller section 4 has been described, but the present invention is not limited to this, and the height measuring section may be provided in the imaging device 2 although not shown.

(Configuration of Image Synthesizer 43)
The image synthesizing unit 43 is configured to synthesize the first height image and the second height image to generate a post-synthesis height image. As a result, the portions of the first height image that are invalid pixels and the portions that are not invalid pixels of the second height image are represented by valid pixels in the combined height image. Conversely, portions that are invalid pixels in the second height image but not invalid pixels in the first height image are represented by valid pixels in the height image after synthesis. Therefore, the number of invalid pixels can be reduced in the combined height image. Conversely, if you want to obtain a height with high reliability, only when both the first height image and the second height image are valid and the difference between them is smaller than a predetermined value, the average height may be enabled.

In other words, by irradiating the measurement object W with pattern light from four different directions, it is possible to increase the number of effective pixels in the height image, reduce blind spots, and increase the reliability of the measurement results. can improve sexuality. In the case of the measurement object W in which the number of invalid pixels is sufficiently reduced by irradiating the pattern light from two directions, only one height image should be generated. In this case, it is also possible for the user to select whether to generate the first height image or the second height image. Generating only one height image has the advantage of shortening the measurement time.

Since the height image after synthesis can also grasp the height of each pixel, it can be used as an image to be inspected when performing various inspections. Therefore, the image synthesizing unit 43 can also be called an inspection target image generation unit that generates an inspection target image.

In this embodiment, the case where the image synthesizing unit 43 is provided in the controller unit 4 has been described.

(Configuration of Inspection Unit 45)
The inspection unit 45 is a part that performs inspection processing based on an arbitrary image among the first height image, the second height image, and the combined height image. The inspection unit 45 includes a presence/absence inspection unit 45a, an appearance inspection unit 45b, and a dimension inspection unit 45c, but this is an example, and all these inspection units are not essential. You may provide an inspection part other than. The presence/absence inspection unit 45a is configured to determine the presence/absence of the object W to be measured, the presence/absence of components attached to the object W to be measured, and the like by image processing. The appearance inspection unit 45b is configured to be able to determine by image processing whether or not the external shape of the object W to be measured has a predetermined shape. The dimension inspection unit 45c is configured to determine whether or not the dimensions of each part of the measurement object W are predetermined dimensions, or to determine the dimensions of each part by image processing. Since these determination methods are conventionally well-known methods, detailed description thereof will be omitted.

(Configuration of display control unit 46)
The display control unit 46 displays the first height image, the second height image, the combined height image, etc. on the display unit 5, and controls the operation user interface for operating the image processing device 1, the image processing device, and the like. A setting user interface for setting 1, a height measurement result display user interface for displaying the height measurement result of the measurement object, and an inspection result display user for displaying various inspection results of the measurement object It is configured such that an interface or the like can be generated and displayed on the display unit 5 .

(Configuration of history storage unit 47)
The history storage unit 47 can be configured by a storage device such as a RAM, for example. The history storage unit 47 can store the first height image, the second height image, the combined height image, and the like output from the imaging device 2 to the controller unit 4 . Images stored in the history storage unit 47 can be read out and displayed on the display unit 5 by operating the console unit 6 or the mouse 7 .

(Correction processing)
As described above, in the present embodiment, the measurement object W on which the pattern light is projected from the first light projecting unit 31 is imaged to generate the first pattern image, and the pattern light is projected from the second light projecting unit 32 . is projected onto the measurement object W to generate a second pattern image; based on the first pattern image and the second pattern image, an angle image having irradiation angle information is generated; The height of the measuring object W can be measured regardless of the relative positional relationship between the imaging device 2 and the lighting device 3 based on the relative positions of the first LED 31b and the second LED 32b and the irradiation angle information.

In this case, the relative positions of the first LED 31b and the second LED 32b affect the height measurement result, so it is necessary to strictly define the relative positions of the first LED 31b and the second LED 32b. However, it is difficult to strictly define the relative positions of the first LED 31b and the second LED 32b because it is inevitable that there will be variations in the parts and in the assembly positions during manufacturing. Therefore, the illumination device 3 of this example is configured to be able to perform positional correction of the first LED 31b and the second LED 32b.

FIG. 18 shows the overall flow of correction processing. A specific example of deviation will be described with reference to FIG. 19 . In FIG. 19(A), the "0" point should be located directly under the first LED 31b, but the mounting angle of the first LED 31b is shifted by θ0, and the "0" point is minus a predetermined amount from directly under the first LED 31b. It shows the case where it is positioned in the direction (to the left in the drawing). FIG. 19B shows a case where the first LED 31b is displaced in the Z direction, and in this example the first LED 31b is displaced downward from the normal mounting position.

By correcting these deviations, as shown in FIG. 19C, the "0" point is located directly below the first LED 31b and on the reference plane, and the irradiation angle is set to 0 to 45°. Specifically, it can be performed based on the flow shown in FIG. Further, after estimating the position of the first LED 31b, a correction coefficient is derived. The correction can be linear correction. For example, the formula Φ′=a*Φ+b can be used, where Φ′ is the corrected absolute phase and Φ is the uncorrected absolute phase. a = tan θ1 + tan θ0 and b = -tan θ0.

Also, as shown in FIG. 20, when there is a tilt angle, the larger the absolute phase, the wider the interval between them. Comparing the length of the line segment 230 with the tilt angle and the length of the line segment 231 without the tilt angle, the line segment 230 with the tilt angle is longer. error. That is, when there is a tilt angle, the value of the absolute phase Φ shifts from that in the ideal state, and becomes a smaller value than the value when there is no tilt angle.

When correcting the tilt angle, first, the tilt is directly measured and a conversion formula is applied. If the swing angle cannot be directly measured, the absolute phase should be corrected secondarily. The conversion formula is Φ'=Φcos θ/(1−Φsin θ), where Φ is the observed absolute phase and Φ' is the absolute phase after conversion (absolute phase when there is no tilt angle).

FIG. 21 schematically shows a case where the first LED 31b and the second LED 32b, which are a pair, are shifted in height. In this example, the second LED 32b is located above the first LED 31b by the dimension d. As a precondition for correcting the height of the first LED 31b and the second LED 32b, it is necessary that the irradiation angle range of the first LED 31b and the second LED 32b has already been normalized (adjusted) to 0° to 45°. A correction formula for height correction is h = (l - d * φ1) / (φ0 + φ1).

That is, the distance l between the first LED 31b and the second LED 32b is l=h*tanθ0+(h+d)*tanθ1. Solve this relationship for h.

h * (tanθ0 + tanθ1) = l - d * tanθ1
h = (l - d * tanθ1) / (tanθ0 + tanθ1)
Here, since tan θ0 and tan θ1 are the same as the absolute phase after normalization, h = (l - d * φ1)/(φ0 + φ1).

FIG. 22 is a diagram for explaining the flow of estimating a portion requiring correction and adjustment of each unit for correction. First, lighting is installed in step SB1. For example, a first LED 31b and a second LED 32b are attached to the illumination housing 30. As shown in FIG. After that, in step SB2, θxy, θxz, and residual errors are estimated in the first estimation, and then each θxy and each θxz are adjusted in step SB3. After the adjustment, the process advances to step SB4 to estimate θ0, θ1, illumination coordinates X, Y, Z, θxy, θyz, θxz, and residual errors in the second estimation. In step SB5, it is determined whether or not there is a residual error, and if it is determined that there is a residual error, the process advances to step SB3 to perform each adjustment and perform the second estimation again. The adjustment and the second estimation are repeated until the residual error disappears, and when the residual error disappears, the process advances to step SB6 to acquire and store the estimation result. The first estimate and the second estimate can be the same. The above steps can be performed by the error estimator 49a shown in FIG.

FIG. 23 is a schematic diagram for explaining a specific method of estimating deviation. Let the center coordinates of the first LED 32b be (X, 0, Z). The phase P of coordinate x is as follows.

t0 = tanθ0
t1= tanθ1
Base length l = (Zz) * (t0 + t1)
Distance from θ0 side end to x lx＝(xX) + (Zz) * t0
P = lx/l
P = {(xX) + (Zz) * t0} / {(Zz) * (t0 + t1)}

If the first LED 32b is tilted, it can be assumed that the illumination center position X has Y dependence.

X(y) = X0 + axy * y
P = {(x-(X0 + a * y)) + (Zz) * t0} / {(Zz) * (t0 + t1)}
= {(x-(X0 + a * y))} / {(Zz) * (t0 + t1)} + t0 / {t0 + t1}

The unknown coefficients are summarized as follows.

t0 ... Irradiation width 0 of the first LED 32b
t1 ... Irradiation width 1 of the first LED 32b
X0 … Lighting center X coordinate
a ... Inclination xy of the first LED 32b (a = tan θxy)
Z … Height of the first LED 32b
P = {(x-(X0 + a * y)) * Ti} / {Zz} + t0 * Ti
Here, the objective function of the least squares method is J = (p - P)^2. p is the measured phase.

Since P is not a linear combination of unknown coefficients, an iterative method such as the gradient method must be used, and the Levenberg-Marquardt method can be used.

After obtaining each coefficient, perform t0, t1, and θxy corrections.

t0' = t0 / cosθxy
t1' = t1/cosθxy

(camera calibration)
FIG. 24 is a diagram showing the concept of camera calibration. First, coefficients (camera parameters) for associating world coordinates with camera coordinates are estimated. A “camera” is the imaging device 2 . In this embodiment, world coordinates=illumination coordinates. That is, the two pairs of the first light projecting unit 31 and the second light projecting unit 32 and the third light projecting unit 33 and the fourth light projecting unit 34 can project the measurement pattern light onto the measurement object W from mutually different directions. Therefore, XYZ can be obtained without using a calibration board.

The world coordinates of the measurement object W and the corresponding camera coordinates are known. The height of the object W to be measured is changed, images are taken multiple times, and highly reliable sampling points (specific points) are used to estimate camera parameters by the well-known method of Tsai (1987). Tsai's paper assumes a flat plate, but in this embodiment, the heights of the first LED 31b and the second LED 32b are used to determine the X, Y, and Z coordinates. Any shape is acceptable. However, since the phase may be out of order, RANSAC, which is a robust estimation method, is also used to remove the error factor.

FIG. 25(A) is a diagram illustrating a case where the X, Y, and Z coordinates of the sampling point SP are obtained when the measurement object W is at the initial position (first height). As described above, the X, Y, and Z coordinates of the sampling point SP are the distance information of the first pattern image set, the second pattern image set, the third pattern image set, the fourth pattern image set, and the first LED 31b and the second LED 32b. and the distance information of the third LED 33b and the fourth LED 34b. FIG. 25B is a diagram illustrating a case where the X, Y, and Z coordinates of the sampling point SP are obtained when the object W to be measured is at a position (second height) higher than the initial position. . The X, Y, and Z coordinates of the sampling point SP can be obtained in the same manner as in FIG. 25(A).

FIG. 26 shows formulas of the camera parameter matrix and the distortion model, which are used to find the parameters for associating the world coordinates with the camera coordinates. x, y are the camera coordinates after correcting the lens distortion and are known. X, Y, Z are world coordinates, which are also known. Use x,y,X,Y,Z to estimate other parameters. s and a are skew and aspect ratio, respectively, and both are usually 1. tx, ty are the center coordinates of the image sensor used as the imaging element 22 . f is the vertical and horizontal focal length. R is the rotation matrix and T is the translation vector. Also, k1, k2, p1, and p2 are distortion parameters.

FIG. 27 is a flow chart showing a procedure for estimating each parameter. At step SC1, tx and ty are estimated, and at step SC2, R is estimated. Step SC1 and step SC2 are processes similar to 2D calibration, and tx, ty, and R can be found relatively stably. However, if there are pixels with incorrect heights at the edges of the field of view, etc., the effect is large, so robust estimation (RANZAC method) is performed.

At step SC3, f and tz are roughly estimated, and at step SC4, f and tz are estimated precisely and k is estimated. Steps SC3 and SC4 are processes for 3D calibration and lens distortion estimation, and the RANZAC method is applied because the variation is large and the least squares method may not be stable. Step SC5 is a step for fine-tuning all estimated values, although it is not an essential step.

Since the f and tz values are affected by the angle of view of the outer pixels (pixels away from the center of the image sensor), the pixels on the outer side of the image sensor are used during calibration. may be used. In addition, it is possible to omit the point of not being affected by the angle of view.

In addition, in order to increase the reliability of the sampling point SP, the X-coordinate, Y-coordinate and Z-coordinate information of the points around the sampling point SP are taken into consideration. may For example, the average value (eg, 3×3, Median, etc.) of the X, Y, and Z coordinates of the sampling point SP and its surrounding points can be used as the X, Y, and Z coordinates of the sampling point SP. . This makes it possible to generate a highly reliable calibration target.

If the first LED 31b, the second LED 32b, the third LED 33b, and the fourth LED 34b are fixed and the phase P is known, derive the world coordinates (wx, wy, wz) from the camera coordinates (xf, yf) and the phase P. can be done.

The estimation of the camera parameters described above can be performed by the controller unit 4 shown in FIG. 1 and the like. This camera parameter is a calibration target, and the generation of the camera parameter is performed by the calibration target generator 48a shown in FIG. The calibration target generator 48a can generate the calibration target using the X, Y, and Z coordinates of the sampling points SP on the surface of the measurement object W when generating the calibration target. That is, the first pattern light generation unit 31d and the second pattern light generation unit 32d form a pair, and the third pattern light generation unit 33d and the fourth pattern light generation unit 34d form a pair. Each pattern light can be sequentially projected onto the object W from a plurality of different directions. As a result, the dead angle of the pattern image is reduced, so that the X, Y and Z coordinates of the sampling point SP can be measured based on each pattern image. This measurement result becomes a calibration target.

The calibration execution unit 48b shown in FIG. 2 executes camera calibration using the calibration target generated by the calibration target generation unit 48a. As a result, the world coordinates and the coordinates of the imaging device 2 are associated with each other. Therefore, a calibration board is not required for associating the world coordinates with the coordinates of the imaging device 2 .

(Adjustment mechanism of light emitting part)
28 and 29 show structural examples of the first light projecting section 31 having an adjusting mechanism. The illumination housing 30 of the illumination device 3 includes a base member 60, an LCD holder (first pattern light generation unit holder member) 61 to which the first LCD 31d is fixed, and an LED holder (first light source holder member) to which the first LED 31b is fixed. 62. The LCD holder 61 is attached to the base member 60 so that its position can be adjusted, and the LED holder 62 is also attached to the base member 60 so that its position can be adjusted. The width direction in FIGS. 28 and 29 is the direction in which the first LEDs 31b are arranged.

That is, as shown in FIG. 28, a boss portion 60a is provided on the upper surface of the base member 60 so as to protrude upward. provided to protrude. On the upper surface of the base member 60, a first shim 63a (indicated by oblique lines slanting downward to the left in FIG. 28) is placed at a portion spaced apart from the boss portion 60a in the width direction. The LCD holder 61 is mounted on the upper end surface of the boss portion 60a and the upper surface of the first shim 63a. As shown in FIG. 29 , the LCD holder 61 is fastened and fixed to the base member 60 with a first screw 64 , a second screw 65 and two third screws 66 . The first screw 64 and the second screw 65 are screwed into the boss portion 60 a of the base member 60 . Two third screws 66 pass through the first shim 63 a and are screwed into the base member 60 . Therefore, by changing the thickness of the first shim 63a, the tilt of the LCD holder 61, that is, the tilt of the LED holder 62 can be changed. That is, θzx of the LED holder 62 can be adjusted, and θzx can be adjusted between the first light projecting section 31 and the second light projecting section 32 .

The LCD holder 61 is formed with an arcuate slit 61c through which the second screw 65 is inserted. The arcuate slit 61c extends to draw an arc around the center line of the first screw 64. As shown in FIG. Further, the LCD holder 61 is formed with a slit 61d through which the third screw 66 is inserted. By forming the arc-shaped slits 61c and 61d, the LCD holder 61 can be rotated around the center line of the first screw 64 and can be fixed at an arbitrary rotation position. Between the backing plate portion 60b of the base member 60 and the side surface of the LCD holder 61, a second shim 63b (indicated by hatching downward to the right in FIG. 28) is arranged. Therefore, by changing the thickness of the second shim 63b, the orientation of the LCD holder 61, that is, the orientation of the LED holder 62 can be changed. That is, θxy of the LED holder 62 can be adjusted, and θxy can be adjusted between the first light projecting section 31 and the second light projecting section 32 .

As shown in FIG. 28, a boss portion 61a is provided on the upper surface of the LCD holder 61 so as to protrude upward. is provided as follows. A third shim 63c is placed on the upper surface of the LCD holder 61 at a portion spaced apart from the boss portion 61a in the width direction. The LED holder 62 is mounted on the upper end surface of the boss portion 61a and the upper surface of the third shim 63c. The LED holder 62 is fastened and fixed to the LCD holder 61 in the same manner as the structure for fixing the LCD holder 61 to the base member 60 . Therefore, θzx of the LED holder 62 can be adjusted by changing the thickness of the third shim 63c. In this case, θzx can be adjusted within the first light projecting section 31 .

A fourth shim 63 d is arranged between the backing plate portion 61 b of the LCD holder 61 and the side surface of the LED holder 62 . Therefore, θxy of the LED holder 62 can be adjusted by changing the thickness of the fourth shim 63d. In this case, θxy can be adjusted within the first light projecting section 31 . Make each adjustment so that the error in the height measurement is minimized.

In addition, in the structural example shown in FIG. 30, one end in the width direction of the substrate 31c on which the first LED 31b is mounted is fixed to the casing 31a by a screw 70. As shown in FIG. A plurality of long holes 71 elongated in the width direction of the substrate 31c are provided at intervals on the other widthwise end side of the substrate 31c. Screws 72 are screwed into portions of the casing 31a corresponding to the long holes 71. As shown in FIG. The angle of the substrate 31c, that is, the angle of the first LED 31b can be adjusted by selecting the position of the substrate 31c to be fastened with the screws 72. FIG.

In the case of the structural example shown in FIG. 30, the first LED 31b and the first LCD 31d are integrated through the casing 31a, which makes it easy to improve the mutual positional accuracy, but it is inevitable that manufacturing errors may occur. If an error occurs, the substrate 31c should be rotated so as to minimize the error during height measurement.

Further, the adjusting mechanism can be provided in the second light projecting section 32, the third light projecting section 33, and the fourth light projecting section 34 as well.

(Use segment change)
Each error can be estimated by the error estimator 49a shown in FIG. 2, and at this time, the projection error of the pattern light projected by the first LCD 31d can also be estimated. (A) of FIG. 31 shows a state in which the relative positions of the first LED 31b and the first LCD 31d are displaced, and the light emitted from the first LED 31b reaches the outside of the first LCD 31d. In this case, the size of the first LCD 31d is larger than the irradiation range (0° to 45°) of the first LED 31b to ensure effective segments and surplus segments.

As shown in FIG. 31B, the first LED 31b is aligned with the designed field of view so that a projection pattern can be generated from the segment positioned directly below the first LED 31b. Based on the estimation result of the error estimator 49a, the segment to be used for the first LCD 31d is determined so as to correct the projection error of the pattern light projected by the first LCD 31d. This is performed by the used segment determination unit 49b shown in FIG.

Effective segments in FIG. 31B are segments that form a pattern, and surplus segments are segments that do not contribute to pattern formation. The start position and end position of the effective segment can be set arbitrarily, and can be determined so as to minimize the projection error of the pattern light. Since the change of the used segment can be performed on the software, it is easier than the adjustment by the physical adjustment mechanism. In the case of the phase shift method, it is only necessary to shift the phase while irradiating the entire area, so the start position and end position of the effective segment need not be the ends of the projection. In the case of a gray code, the outside of the irradiation range is always an all-black pattern, so generally the boundary between the all-black pattern and another spatial code can be the start position and end position.

The use segment can be changed in the third light projecting section 33 and the fourth light projecting section 34 as well.

(Relationship between illumination structure error and correction method)
Most of the errors described above are caused by the illumination structure. For example, θxy and θzx between the first light projecting section 31 and the second light projecting section 32 and θxy and θzx between the light projecting sections 31 to 34 can be corrected by the adjustment mechanism. Also, the field of view (θ0, θ1) can be corrected by changing the segment used. Also, the field of view (θ0, θ1) and the tilt angle θyz can be corrected by absolute phase correction. Further, the distance (l) between the LEDs of the paired light projecting units and the height difference (d) of the LEDs of the paired light projecting units can be corrected by the parameters used when measuring the height.

(correction during operation)
If the first LED 31b and the second LED 32b are misaligned in the height direction or the third LED 33b and the fourth LED 34b are misaligned in the height direction due to the influence of the ambient temperature, etc., the misalignment is corrected during operation of the image processing apparatus 1. be able to. For example, the height of the first LED 31b is taken as true, and the least squares method is used for matching.

(During operation of the image processing apparatus 1)
Next, operation of the image processing apparatus 1 will be described. 32 to 36 show the case where the object W to be measured is a rectangular parallelepiped box, and pattern light is projected from the first light projecting section 31 and the second light projecting section 32 for measurement.

First, when the user places the object W to be measured on the placement surface 100 and performs a measurement start operation or an inspection start operation, eight phase shift methods are emitted from the first light projecting unit 31 and the second light projecting unit 32, respectively. are sequentially generated and projected onto the object W to be measured. The imaging device 2 takes an image at the timing when each pattern light is projected. The phase shift pattern image set shown in FIG. 32 is an image of the pattern light projected onto the measurement object W from the first light projecting section 31 . Generating a phase image based on the phase shift pattern image set shown in FIG. 32 results in a phase image as shown on the left side of FIG. Generating an intermediate image from the phase image results in an image as shown on the right side of FIG. The illustration of the gray code pattern image is omitted.

On the other hand, the phase shift pattern image set shown in FIG. 34 is an image of the pattern light projected onto the measurement object W from the second light projecting section 32 . Generating a phase image based on the phase shift pattern image set shown in FIG. 34 results in a phase image as shown on the left side of FIG. Generating an intermediate image from the phase image results in an image as shown on the right side of FIG.

By synthesizing the intermediate image shown on the right side of FIG. 33 and the intermediate image shown on the right side of FIG. 35, a height image shown on the left side of FIG. 36 is generated. The vertical cross-sectional shape of the height image can be displayed on the display unit 5 by a user interface as shown on the right side of FIG.

Further, when a measurement start operation or an inspection start operation is performed, eight pattern lights for the phase shift method are sequentially generated from the third light projecting section 33 and the fourth light projecting section 34, respectively, and projected onto the measurement object W. . The imaging device 2 takes an image at the timing when each pattern light is projected.

(Action and effect of the embodiment)
As described above, according to this embodiment, the first light projecting unit 31 that projects the first pattern light for measurement, the second light projecting unit 32 that projects the second pattern light for measurement, and the imaging device 2 Thus, a plurality of first pattern images and second pattern images can be generated. Based on the first pattern image and the second pattern image, a first angle image in which each pixel has irradiation angle information of the first pattern light for measurement to the measurement object W, and 2, a second angle image having irradiation angle information of the pattern light for measurement can be generated.

Since the relative positions of the first light projecting section 31 and the second light projecting section 32 in the lighting housing 30 of the lighting device 3 are known, this relative position information, the irradiation angle information of each pixel of the first angle image, and the The height of the object W to be measured in the direction of the central axis A of the illumination device 3 is measured using the irradiation angle information of each pixel of the two-angle image, regardless of the relative positional relationship between the imaging device 2 and the illumination device 3. be able to.

In other words, when the lighting device 3 and the imaging device 2 are made separately so that they can be installed separately and the degree of freedom at the time of installation is increased, the position of the imaging device 2 with respect to the lighting device 3 does not need to be strictly adjusted. , the absolute shape of the object W to be measured can be measured, so the burden on the user during installation is not increased.

In addition, since the liquid crystal panel is used in the illumination device 3 as a means for generating pattern light, a reflection optical system as in the case of using a DMD is not required, and a mask having a pattern is not required to be moved. Therefore, the configuration of the illumination device 3 is simplified and the illumination device 3 is made compact. In particular, in the case of a reflective optical system, the optical system including the lens is expensive and requires strict accuracy, and the pattern may be distorted due to the distortion of the lens, which may deteriorate the accuracy. Such a situation can be avoided.

In addition, the first LED 31b and the second LED 32b are provided apart from each other in the circumferential direction of the opening 30a of the illumination housing 30, and the first LCD 31d and the second LCD 32d into which the diffused light emitted from the first LED 31b and the second LED 32b are respectively incident are arranged in the illumination housing 30. is provided in the same plane perpendicular to the central axis A of the opening 30a of the measurement object W, while suppressing the occurrence of luminance unevenness according to the position of the pattern light due to the angular characteristics of the respective LCDs 31d and 32d. On the other hand, the illumination device 3 capable of projecting pattern light from a plurality of directions can be downsized, and the installation flexibility of the illumination device 3 can be increased.

In addition, since the relative positions of the first LCD 31d and the first LED 31b are set so that the diffused light emitted from the first LED 31b is incident on the first LCD 31d within the effective angle range of the first LCD 31d, the angular characteristics of the first LCD 31d It is possible to reduce the size of the illumination device 3 while suppressing the occurrence of luminance unevenness according to the position of the pattern light caused by .

Further, when the dimension of the light emitting surface of the first LED 31b is equal to or smaller than the dimension of the half period of the wave of the pattern light, the plurality of first LEDs 31b are arranged so as to be shifted from each other in the direction in which the illuminance of the pattern light changes. can be longer than the dimension of the half period of the wave of the pattern light. As a result, wave pattern light can be generated as intended, and high-frequency undulations can be prevented from occurring in the height image.

Further, by providing the adjustment mechanism in the first light projecting section 31 and by enabling the change of the segment to be used, it is possible to correct the positional deviation of the first LED 31b. This makes it possible to obtain accurate measurement results when measuring the height of the measurement object W based on the irradiation angle information of the first LED 31b and the second LED 32b and the relative positions of the two LEDs 31b and 32b.

Further, since the calibration target can be generated based on the pattern image obtained by receiving the pattern light for measurement, the world coordinates and the coordinates of the imaging device 3 can be associated without using a calibration board. It can be performed.

The above-described embodiments are merely examples in all respects and should not be construed in a restrictive manner. Furthermore, all modifications and changes within the equivalent range of claims are within the scope of the present invention.

For example, it is possible to provide a program configured to allow a computer to implement each of the processes and functions described above, thereby allowing a user's computer to implement each of the processes and functions described above.

The form of providing the program is not particularly limited, but there are, for example, a method of providing using a network line such as the Internet, a method of providing a recording medium storing the program, and the like. In any of the providing methods, the processes and functions described above can be implemented by installing a program on the user's computer.

Devices that implement the processes and functions described above include general-purpose or dedicated devices that implement the programs in the form of software, firmware, or the like in an executable state. In addition, a form in which a part of each processing and function described above is mixed with hardware such as a predetermined gate array (FPGA, ASCI), or a partial hardware module that realizes some elements of program software and hardware can be realized with

Also, the above-described processes and functions can be realized by combining steps (processes), in which case the user will execute the image processing method.

Further, in the above embodiments, the case where the pattern light generation unit is a liquid crystal panel has been described, but the present invention is not limited to this. The mask may be moved. Also, the light source is not limited to light emitting diodes.

INDUSTRIAL APPLICABILITY As described above, the image processing apparatus according to the present invention can be used, for example, when measuring the height of an object to be measured or when inspecting an object to be measured.

1 image processing device 2 imaging device 3 lighting device 22 imaging element 26 phase calculation unit 30 lighting housing 31 first light projection unit 31b first LED (first light source)
31d 1st LCD (first pattern light generator)
31e first LED drive circuit (light source drive circuit)
31f First LCD drive circuit (liquid crystal panel drive circuit)
32 Second light projecting portion 32b Second LED (second light source)
32d Second LCD (second pattern light generator)
32e Second LED drive circuit (light source drive circuit)
32f Second LCD drive circuit (liquid crystal panel drive circuit)
33 Third light projecting portion 33b Third LED (third light source)
33d Third LCD (third pattern light generator)
33e Third LED drive circuit (light source drive circuit)
33f Third LCD drive circuit (liquid crystal panel drive circuit)
34 fourth light projecting portion 34b fourth LED (fourth light source)
34d fourth LCD (fourth pattern light generator)
34e fourth LED drive circuit (light source drive circuit)
34f fourth LCD drive circuit (liquid crystal panel drive circuit)
35 light projection control unit 41a illumination mode switching unit 42 height measurement unit (inspection target image generation unit)
43 image synthesis unit 45 inspection unit 48a calibration target generation unit 48b calibration execution unit 49a error estimation unit 49b use segment determination unit A opening

Claims (5)

In an image processing device that measures the height of an object to be measured,
an illumination housing having an opening formed in the center thereof; first and second light sources provided apart from each other in the circumferential direction of the opening in the illumination housing and emitting diffused light; and emitted from the first light source. a first liquid crystal panel disposed so as to correspond to the first light source so that the emitted light is incident thereon, and provided so that an exit surface is positioned within a plane perpendicular to the central axis of the opening; a second liquid crystal panel disposed so as to correspond to the second light source so that light emitted from the second light source is incident thereon, and provided so that an emission surface is positioned within the plane; The first light source, the second light source, and the a lighting device comprising a first liquid crystal panel and a light projection control section for controlling the second liquid crystal panel;
A first pattern image set obtained by receiving reflected light of a plurality of pattern lights emitted by the first liquid crystal panel through the opening of the illumination housing and a plurality of pattern lights emitted by the second liquid crystal panel an imaging device that generates a second pattern image set obtained by receiving the reflected light of the pattern light of
An inspection target for generating an inspection target image including height information of the measurement target in the central axis direction of the illumination device based on the first pattern image set and the second pattern image set generated by the imaging device. an image generator;
an inspection unit that executes inspection processing based on the inspection target image generated by the inspection target image generation unit ;
The illumination housing includes a plurality of observation light-emitting diodes annularly arranged so as to surround the first light source, the second light source, the first liquid crystal panel and the second liquid crystal panel in plan view, and the plurality of observation light-emitting diodes. a cover member formed to cover the observation light-emitting diodes and diffusing the observation light emitted from the plurality of observation light-emitting diodes;
The imaging device receives reflected light from the measurement object of the light diffused by the cover member to generate a luminance image,
The image processing apparatus, wherein the inspection unit is configured to be capable of executing inspection processing based on the image to be inspected and the luminance image . The image processing device according to claim 1,
An image processing apparatus, wherein the drive system of the first liquid crystal panel and the second liquid crystal panel is a TN system. The image processing device according to claim 1 or 2,
The lighting device is provided between the first light source and the second light source in the lighting housing, and is symmetrical with a third light source that emits diffused light with respect to the central axis of the opening in the lighting housing. A fourth light source that is provided point-symmetrically with the third light source as the center of the fourth light source that emits diffused light; a third liquid crystal panel provided such that an emission surface is positioned within a plane perpendicular to the central axis of the opening; a fourth liquid crystal panel arranged correspondingly and provided with an exit surface positioned within the plane;
The light projection control unit is configured to control the third light source, the fourth light source, the third liquid crystal panel, and the fourth liquid crystal panel so as to project the pattern light onto the measurement object a plurality of times. is,
The imaging device receives pattern light projected from each of the third liquid crystal panel and the fourth liquid crystal panel through the opening of the illumination housing and reflected light from a measurement object to obtain a third pattern. An image processing apparatus configured to generate an image set and a fourth pattern image set. In the image processing device according to any one of claims 1 to 3,
the pattern light has a periodic illuminance distribution that changes in a one-dimensional direction;
The image processing apparatus, wherein the first light source is composed of a plurality of light emitting diodes arranged in a direction in which the illuminance of the pattern light does not change. In the image processing device according to any one of claims 1 to 4,
The cover member is provided inside the illumination housing radially outward of the opening with respect to the emission surfaces of the first liquid crystal panel and the second liquid crystal panel. 2. An image processing apparatus, wherein the image processing apparatus is provided so as to protrude in a direction approaching an object to be measured from the emission surface of each of the two liquid crystal panels .

Images