JP7164476B2 - Image processing device - Google Patents

Description

The present invention relates to an image processing apparatus having an illumination unit that illuminates an object to be measured and an imaging unit that receives light reflected from the object to be measured, and more particularly to a technical field of a structure capable of acquiring a three-dimensional shape of the object to be measured. belongs to

Conventionally, as this type of image processing apparatus, a pattern light having a light intensity distribution that varies depending on the position is projected onto an object to be measured, the light reflected from the object to be measured is received, and the height obtained based on the amount of received light is measured. A so-called pattern projection method is known that obtains the three-dimensional shape of an object to be measured using information. As a pattern projection method, there is a phase shift method in which pattern light whose illuminance distribution is varied, for example, sinusoidally is projected a plurality of times with different phases and an image is captured each time.

Patent Document 1 discloses an illumination device that generates and projects pattern light, including a light source, a condenser lens that collects light emitted from the light source, and a liquid crystal panel that receives the light converged by the condenser lens. and configured to project a pattern formed on a liquid crystal panel onto an object to be measured.

Japanese Unexamined Patent Application Publication No. 2015-21760

By the way, in Japanese Unexamined Patent Application Publication No. 2002-100001, pattern light is generated by an illumination device, and after the projection of the pattern light is completed, an image of an object to be measured is captured by an imaging device, then another pattern light is generated, and the pattern light is Since the image capturing process is repeated for one measurement object after the projection is completed, the brightness of the disturbance light may fluctuate until the measurement of one measurement object is completed.

If the brightness of the ambient light fluctuates during the measurement, the intensity of the reflected light from the object to be measured fluctuates due to a factor completely different from that of the pattern light, resulting in errors in the measurement results. There was a risk that I would lose it.

The present invention has been made in view of such a point, and an object of the present invention is to reduce measurement errors caused by fluctuations in the brightness of ambient light.

To achieve the above object, a first invention provides an image processing apparatus for measuring a three-dimensional shape of a measurement object, comprising: a light source; A pattern generation unit that irradiates pattern light, and the pattern generation unit is controlled to sequentially shift the phase of the pattern, and when the pattern generation unit is controlled, patterns that are 180 degrees out of phase with each other are continuously generated. an illumination control unit that controls the measurement object, an imaging unit that receives the light reflected from the measurement object each time the pattern light is irradiated and generates a plurality of pattern images, and a measurement object based on the plurality of pattern images and a measuring unit for measuring a three-dimensional shape.

According to this configuration, when the illumination control section controls the pattern generation section, the pattern generation section irradiates the measurement object with the pattern light whose phase is sequentially shifted. The imaging unit receives the light reflected from the object to be measured and generates a plurality of pattern images each time the object to be measured is irradiated with the pattern light. A measurement unit measures the three-dimensional shape of the measurement object based on the plurality of pattern images generated.

When the illumination control unit controls the pattern generation unit, patterns having phases different from each other by 180 degrees are continuously generated. The phases of the even-numbered pattern light beams are opposite to each other. As a result, it is possible to narrow the irradiation interval of the pattern lights having opposite phases.

Here, since the phase calculation is performed by the following formula, the time interval between the projections of the patterns of opposite phases is important.

Phase φ=tan ⁻¹ {(I _270° −I _90° )/(I _0° −I _180° )}
In other words, the narrower the irradiation interval of the pattern lights of opposite phases, the more the fluctuation in the brightness of the disturbance light can be minimized. Even if the brightness of ambient light fluctuates until the measurement of one measurement object is completed, the influence of the fluctuation is suppressed.

In a second aspect of the present invention, the light source includes a first light source and a second light source provided point-symmetrically with respect to each other about the center of the opening in an illumination housing having an opening formed in the center, The pattern generation section includes a first pattern generation section provided corresponding to the first light source and a second pattern generation section provided corresponding to the second light source, and the illumination control section includes the first pattern generation section provided corresponding to the second light source. While one of the pattern generation unit and the second pattern generation unit is switching patterns, the other pattern generation unit that has completed pattern generation emits pattern light. It is characterized by

According to this configuration, since the first pattern generation section and the second pattern generation section are spaced apart from each other in the circumferential direction of the opening of the illumination housing, the pattern light is projected onto the object to be measured from at least two different directions. It is possible to irradiate and reduce the range of blind spots. In this case, for example, when pattern generation is completed by the second pattern generation unit while the first pattern generation unit is switching patterns, pattern light is emitted from the second pattern generation unit to the object to be measured. Can be irradiated. In other words, while one of the pattern generation units is switching the pattern, another pattern generation unit can irradiate the measurement object with pattern light, thereby speeding up the measurement of the three-dimensional shape. can.

In a third aspect of the invention, the light sources are spaced apart from the first light source and the second light source in the circumferential direction of the opening in the lighting housing and are provided point-symmetrically with respect to the center of the opening. a third light source and a fourth light source, wherein the pattern generation unit includes a third pattern generation unit provided corresponding to the third light source and a fourth pattern generation unit provided corresponding to the fourth light source and the illumination control unit is configured such that an arbitrary pattern generation unit among the first pattern generation unit, the second pattern generation unit, the third pattern generation unit, and the fourth pattern generation unit performs pattern switching. It is characterized in that the pattern light is emitted from the pattern generation unit that has completed the pattern generation while the light is on.

According to this configuration, since the first pattern generation section, the second pattern generation section, the third pattern generation section, and the fourth pattern generation section are provided apart from each other in the circumferential direction of the opening of the illumination housing, the measurement target The object can be irradiated with the patterned light from at least four different directions, and the dead angle range can be further reduced. In this case, while one of the pattern generation units is switching the pattern, another pattern generation unit can irradiate the object to be measured with pattern light, thereby speeding up the measurement of the three-dimensional shape. be able to.

A fourth aspect of the invention is characterized in that the pattern generating section is a liquid crystal panel.

In general, when patterns are generated on a liquid crystal panel, it may take a long time to complete the generation of another pattern after one pattern is generated. This is a factor that prolongs the time required to measure one measurement object, so there is a risk that the adverse effects of brightness fluctuations of ambient light will become more pronounced. In the present invention, since the pattern is generated by the liquid crystal panel, there is a possibility that the adverse effect of the fluctuation of the brightness of the disturbance light may become significant. can do. That is, it is possible to suppress the adverse effect of brightness fluctuations of ambient light while using a liquid crystal panel that is advantageous in terms of cost.

A fifth aspect of the invention is characterized in that the imaging section is composed of a color camera.

By using a color camera as the imaging unit, for example, a two-dimensional image of an object to be inspected can be provided to the user as a color image.

In addition, as a method of suppressing fluctuations in the brightness of ambient light, for example, there is a method of applying a bandpass filter. cannot be applied. According to the present invention, when a color camera is used, variations in the brightness of ambient light can be suppressed by continuously generating patterns that are 180 degrees out of phase with each other without applying a band-pass filter. .

In a sixth aspect of the invention, the illumination control unit causes the pattern generation unit to generate phase shift pattern light for generating a phase shift image and spatial code pattern light for generating a space code image, When the phase-shifting pattern light is generated, patterns having phases different from each other by 180 degrees are continuously generated, and the imaging unit captures the light reflected from the measurement object when the phase-shifting pattern light is irradiated. It is configured to receive light to generate a phase shift image, and to generate a space code image by receiving light reflected from an object to be measured when the space code pattern light is irradiated. It is characterized in that the three-dimensional shape of the object to be measured is measured by applying a phase shift method and a space encoding method based on the phase shift image and the space code image.

According to this configuration, the phase shift image and the space code image are generated respectively, and the three-dimensional shape of the measurement object can be measured by applying the phase shift method and the space encoding method, so that the measurement range can be expanded. be able to. In this case, the phase shift method is more adversely affected by brightness fluctuations of ambient light than the spatial encoding method. Since it is made to generate|occur|produce uniformly, the fluctuation|variation of the brightness of disturbance light can be suppressed.

According to the present invention, when the pattern generator is controlled, the patterns whose phases are different from each other by 180 degrees are continuously generated. The phases of the pattern light applied to the even-numbered pattern light and the pattern light applied to the even-numbered order have an opposite phase relationship. As a result, even if the brightness of ambient light fluctuates until the measurement of one measurement object is completed, the influence of the fluctuation can be suppressed, and errors in the measurement results can be reduced.

1 is a diagram showing a system configuration example of an image processing apparatus according to an embodiment of the present invention; FIG. 3 is a block diagram of a controller unit; FIG. 1 is a block diagram of an imaging device; FIG. It is a top view of an illuminating device. FIG. 10 is a bottom view of the lighting device according to Embodiment 2; FIG. 6 is a sectional view taken along the line VI-VI in FIG. 5; It is a figure explaining the height measurement method by a height measurement part. It is a figure explaining the generation point of pattern light. It is a block diagram of an illuminating device. FIG. 10 is a diagram for explaining the order of forming patterns on the first to fourth LCDs and projecting light; It is a figure which shows the irradiation order of pattern light. It is a figure which shows the light projection interval of opposite phases. 10A and 10B are timing charts for imaging in only one direction and imaging by switching between two directions; 7 is a graph showing an example of changes in brightness of ambient light; 7 is a graph showing phase calculation errors caused by brightness fluctuations of ambient light; FIG. 10 is a graph showing the difference in phase error between the case where pattern light projection order is reverse phase continuous and the case where pattern light projection order is normal pattern light projection order; FIG. FIG. 10 is a diagram illustrating the procedure for generating an intermediate image and a reliability image from a pattern image set using example images; FIG. 10 is a diagram showing a procedure for generating intermediate images and reliability images from a pattern image set; FIG. 4 is a diagram showing a user interface that implements a three-dimensional discrimination function; FIG. 10 is a diagram showing a user interface displayed when a section is instructed; FIG. 10 is a diagram showing a user interface in which an indicated cross section is displayed; FIG. 21 is a view corresponding to FIG. 21 when the allowable range is reduced; FIG. 10 is a diagram showing a user interface displaying a state in which a new three-dimensional shape of an object to be measured is input and compared with an allowable range; FIG. 10 is an image diagram of allowable range setting when detecting chipping of a comparatively large protruding measurement target. FIG. 10 is an image diagram of allowable range setting when detecting chipping of a comparatively small projecting measurement target. FIG. 10 is a diagram for explaining a case where an expansion filter is applied to both a registered three-dimensional shape and an input three-dimensional shape; It is a data flow diagram of a present Example. FIG. 4 is a dataflow diagram according to another example; FIG. FIG. 10 is an image diagram of processing of a background plane; FIG. 10 is an image diagram of processing when an upper limit plane is set in addition to a background plane; FIG. 4 is a data flow diagram for background plane processing; FIG. It is a data flow when background plane processing is applied to each of the upper limit of the allowable range and the lower limit of the allowable range. FIG. 10 is a diagram showing a user interface that implements a height binarization function; FIG. 34 is a view equivalent to FIG. 33 showing a state in which there is no portion positioned within a predetermined horizontal plane; FIG. 33 is a view equivalent to FIG. 33 when enlarged display is performed by a zoom function; FIG. 10 is a diagram showing a user interface that implements a shape comparison function in height binarized images; FIG. 10 illustrates a user interface for selecting contours to inspect; FIG. 37 is a view equivalent to FIG. 37 showing a state in which contour detection results are displayed; FIG. 38 is a view corresponding to FIG. 38 when the allowable range is expanded; FIG. 39 is a view equivalent to FIG. 39 showing a display example when three-dimensional shapes of different types are input in the operation mode;

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described in detail based on the drawings. It should be noted that the following description of preferred embodiments is essentially merely illustrative, and is not intended to limit the invention, its applications, or its uses.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described in detail based on the drawings. It should be noted that the following description of preferred embodiments is essentially merely illustrative, and is not intended to limit the invention, its applications, or its uses.

FIG. 1 is a diagram showing a system configuration example of an image processing apparatus 1 according to an embodiment of the present invention. The image processing apparatus 1 includes an imaging device 2, an illumination device 3, a controller section 4, a display section 5, a console section 6, and a mouse 7, and measures the three-dimensional shape of the object W to be measured. Thus, a height image is obtained, and various inspections can be performed on the object W to be measured based on this height image.

The object W to be measured is placed on a mounting surface 100 of a conveying device such as a belt conveyor, for example, and the height of the object W placed on the mounting surface 100 is measured and various inspections are performed. I do. The object W to be measured is preferably kept stationary during the height measurement.

The image processing device 1 can be wired to a programmable logic controller (PLC) 101 via a signal line 101a. The image processing device 1 and the PLC 101 may be wirelessly connected. The PLC 101 is a control device for sequence-controlling the conveying device and the image processing device 1, and can use a general-purpose PLC. The image processing device 1 can also be used without being connected to the PLC 101 .

The display unit 5 is a display device such as a liquid crystal display panel, and constitutes display means. The display unit 5 includes, for example, an operation user interface for operating the image processing apparatus 1, a setting user interface for setting the image processing apparatus 1, and a height display for displaying the height measurement result of the object to be measured. A user interface for displaying measurement results, a user interface for displaying inspection results for displaying various inspection results of the object to be measured, and the like can be displayed. A user of the image processing apparatus 1 can operate and set the image processing apparatus 1 by viewing the display unit 5, and can grasp the measurement results, inspection results, and the like of the measurement object W. Furthermore, the operating state of the image processing apparatus 1 can also be grasped.

As shown in FIG. 2, the display unit 5 is connected to a display control unit 46 included in the controller unit 4, and can be controlled by the display control unit 46 to display the above-described user interface, height image, and the like. is configured as

The console section 6 is input means for the user to operate the image processing apparatus 1 and input various information, and is connected to the controller section 4 . The mouse 7 is also input means for the user to operate the image processing apparatus 1 and input various information, and is connected to the controller section 4 . The console unit 6 and the mouse 7 are examples of input means, and the input means may be, for example, a touch panel screen provided in the display unit 5 or a voice input device. Can also be configured. In the case of a touch panel screen, display means and input means can be realized by one device.

A general-purpose personal computer PC for generating and storing a control program for the controller section 4 can also be connected to the controller section 4 . Also, an image processing program for performing various settings related to image processing can be installed in the personal computer PC, and various settings for image processing performed by the controller unit 4 can be performed. Alternatively, a processing order program that defines the processing order of image processing can be generated by software operating on this personal computer PC. In the controller section 4, each image processing is sequentially executed according to the processing order. The personal computer PC and the controller unit 4 are connected via a communication network. transferred to Conversely, the processing order program, layout information, etc. can be read from the controller unit 4 and edited on the personal computer PC. Note that the above program may be generated not only by the personal computer PC but also by the controller section 4 .

Also, the controller unit 4 can be constructed with dedicated hardware, but the present invention is not limited to this configuration. For example, a general-purpose personal computer PC, a work station, or the like installed with a dedicated image processing program, an inspection processing program, a height measurement program, or the like may function as the controller unit. In this case, the imaging device 2, the lighting device 3, the display section 5, the console section 6, and the mouse 7 may be connected to a personal computer PC, workstation, or the like.

Further, although the functions of the image processing apparatus 1 will be described later, all the functions of the image processing apparatus 1 may be realized by the controller section 4 or may be realized by a general-purpose personal computer PC. Alternatively, part of the functions of the image processing apparatus 1 may be realized by the controller unit 4, and the remaining functions may be realized by a general-purpose personal computer PC. The functions of the image processing apparatus 1 can be realized by software, or by a combination of hardware.

The interface for connecting the imaging device 2, lighting device 3, display unit 5, console unit 6, and mouse 7 to the controller unit 4 may be a dedicated interface, or an existing communication standard such as Ethernet ( product name), USB, RS-232C, etc. can also be used.

The height image representing the height of the measurement object W is an image representing the height of the measurement object W in the direction of the central axis A (shown in FIG. 1) of the opening 30a of the illumination device 3 shown in FIG. It can also be called a distance image. The height image can be displayed as a height with reference to the placement surface (also referred to as a reference surface) 100 of the object W to be measured, or as a relative distance to the illumination device 3 in the direction of the central axis A. It is an image in which the grayscale value of each pixel changes according to the height. In other words, the height image can be said to be an image whose gradation value is determined based on the height of the object W to be measured with respect to the placement surface 100. It can also be said that it is an image whose grayscale value is determined based on the relative distance. Further, the height image can be said to be a multivalued image having gradation values corresponding to the height of the measurement object W with respect to the mounting surface 100, and the relative distance from the illumination device 3 in the direction of the central axis A It can also be said to be a multivalued image having grayscale values corresponding to . Furthermore, the height image can be said to be a multivalued image obtained by converting the height of the measurement object W with respect to the placement surface 100 of the measurement object W into a gray value for each pixel of the luminance image. It can also be said to be a multivalued image obtained by converting the relative distance to the device 3 in the direction of the central axis A into a grayscale value.

A height image is an image containing height information of the object W to be measured. For example, a three-dimensional composite image obtained by synthesizing and pasting an optical luminance image as texture information to a distance image is also regarded as a height image. be able to. Height images are not limited to those displayed three-dimensionally, and those displayed two-dimensionally are also included.

When measuring the three-dimensional shape of an object to be measured, the coordinate system in the Z-axis direction is inverted depending on whether the height is to be determined or the depth is to be determined. Therefore, the calculation "finding a small height" is equivalent to the calculation "finding a large depth". In addition, when it is considered to obtain height, it can be called a height camera, but when it is considered to obtain depth, it can also be called a depth camera. Since the present embodiment can be applied in any case, an example thereof will be described below by using a height image.

Methods for obtaining height images as described above are broadly divided into two methods. One is a passive method ( passive measurement method), and the other is an active method (active measurement method) in which a distance image is generated by actively irradiating the measurement object W with light for measuring in the height direction. In this embodiment, a height image is obtained by an active method, and more specifically, a pattern projection method is used.

In the pattern projection method, a plurality of images are acquired by shifting the shape, phase, etc. of the pattern of the measurement pattern light (simply referred to as pattern light) projected onto the measurement object W, and the acquired plurality of images are analyzed. is a method of obtaining the three-dimensional shape of the object W to be measured. There are several types of pattern projection methods. A phase shift method that obtains the three-dimensional coordinates of the surface of the measurement object W using the phase obtained by the measurement, or a pattern that is projected onto the measurement object W is changed for each photographing, for example, the black-and-white duty ratio is 50%, and the fringe width is the entire width. 1/4, 1/8, 1/16, . There is a spatial code method for obtaining the absolute phase of . The sinusoidal pattern light and the striped pattern light are pattern light having a periodic illumination distribution that changes in one dimension. Note that “projecting” the measurement pattern light onto the measurement object W and “irradiating” the measurement object W with the measurement pattern light are synonymous.

In the image processing apparatus 1 according to the present embodiment, the height image is generated by combining the phase shift method and the spatial code method described above, but the height image is not limited to this, and the height image is generated only by the phase shift method. Alternatively, the height image may be generated only by the spatial coding method. Also, the height image of the object W to be measured may be generated using another known active method.

The outline of the method for measuring the height of the object W to be measured by the image processing device 1 is as follows. First, the measurement object W is irradiated with the first measurement pattern light and the second measurement pattern light respectively generated by the first light projection unit 31 and the second light projection unit 32 of the illumination device 3 from different directions, and the measurement is performed. The imaging device 2 receives the first measurement pattern light reflected from the object W to generate a first pattern image set including a plurality of first pattern images, and the second measurement pattern reflected from the measurement object W. The light is received by imaging device 2 to generate a second pattern image set comprising a plurality of second pattern images. After that, based on the plurality of first pattern image sets, each pixel generates a first angle image having irradiation angle information of the first measurement pattern light on the measurement object W, and based on the plurality of second pattern images to generate a second angle image in which each pixel has irradiation angle information of the second pattern light for measurement onto the object to be measured. Next, according to the irradiation angle information of each pixel of the first angle image, the irradiation angle information of each pixel of the second angle image, and the relative position information of the first light projection unit 31 and the second light projection unit 32, A height image representing the height of the object W is generated, and the height of the object W to be measured is obtained from this height image.

Although not essential, in the image processing apparatus 1, as shown in FIG. A fourth light projecting section 34 is provided. Therefore, it is also possible to irradiate the measurement target W with the third measurement pattern light and the fourth measurement pattern light respectively generated by the third light projecting section 33 and the fourth light projecting section 34 of the illumination device 3 from different directions. can. In this case, the imaging device 2 receives the third measurement pattern light reflected from the measurement object W to generate a third pattern image set including a plurality of third pattern images, and the third pattern light reflected from the measurement object W is generated. 4 The imaging device 2 receives the pattern light for measurement and generates a fourth pattern image set composed of a plurality of fourth pattern images. After that, based on the plurality of third pattern images, each pixel generates a third angle image having irradiation angle information of the third measurement pattern light onto the measurement object W, and based on the plurality of fourth pattern images, Each pixel generates a fourth angle image having irradiation angle information of the fourth measurement pattern light onto the measurement object. Next, according to the irradiation angle information of each pixel of the third angle image, the irradiation angle information of each pixel of the fourth angle image, and the relative position information of the third light projecting unit 33 and the fourth light projecting unit 34, A height image representing the height of the object W is generated, and the height of the object W to be measured is obtained from this height image.

(Phase shift method)
Here, the phase shift method will be explained. In the phase shift method, when pattern light having patterns with sinusoidally varied illuminance distributions is sequentially projected onto the object to be measured, three or more patterns of pattern light with different phases of the sinusoidal waves are projected. Each lightness value of the height measurement point is obtained from an image captured for each pattern from an angle different from the projection direction of the pattern light, and the phase value of the pattern light is calculated from each lightness value. The phase of the pattern light projected onto the measurement point changes according to the height of the measurement point, and light with a phase different from the phase observed by the pattern light reflected at the reference position is observed. become. Therefore, the phase of the pattern light at the measurement point is calculated, and the height of the measurement point is measured by substituting it into the geometric relational expression using the principle of triangulation. can be asked for. According to the phase shift method, the height of the object W to be measured can be measured with high resolution by reducing the period of the pattern light, but the measurable height range is within 2π in phase shift amount. Only low height objects (objects with a small height difference) can be measured. Therefore, the spatial code method is also used.

(Spatial code method)
According to the space code method, a space irradiated with light can be divided into a large number of small spaces each having a substantially fan-shaped cross section, and a series of space code numbers can be assigned to the small spaces. Therefore, even if the height of the object W to be measured is high, that is, even if the height difference is large, the height can be calculated from the space code number as long as it is in the space irradiated with light. Therefore, it is possible to measure the shape of the entire measuring object W having a high height. Thus, according to the spatial code method, the allowable height range (dynamic range) is widened.

(Detailed configuration of lighting device 3)
As shown in FIG. 4, the lighting device 3 according to the first embodiment includes a lighting housing 30, a first light projecting section 31, a second light projecting section 32, a third light projecting section 33, a fourth light projecting section 34 and a projection control unit 39 . As shown in FIG. 1, the illumination device 3 and the controller section 4 are connected by a connection line 3a, but the illumination device 3 and the controller section 4 may be wirelessly connected.

The illumination device 3 may be a pattern light projection dedicated device only for projecting pattern light, or may be a uniform light illumination device for irradiating the measurement object W with uniform light. When the illumination device 3 is used as a pattern light projection device, a uniform light illumination device can be provided separately from the pattern light projection device or integrally with the pattern light projection device. A light-emitting diode, a semiconductor laser, a halogen light, an HID, or the like can be used as the illumination device for uniform light.

The lighting housing 30 has an opening 30a in the center in plan view, and the first side 30A, the second side 30B, the third side 30C, and the fourth side 30D are continuous in plan view. It has a shape close to a rectangle. Since the first side portion 30A, the second side portion 30B, the third side portion 30C, and the fourth side portion 30D extend linearly, the opening portion 30a also has a substantially rectangular shape in plan view.

The outer shape of the lighting housing 30 and the shape of the opening 30a are not limited to the illustrated shapes, and may be, for example, circular. A central axis A of the opening 30a shown in FIG. When the illumination device 3 is installed so that the lower surface of the illumination housing 30 is horizontal, the central axis A of the opening 30a extends vertically. It can also be installed in such a way that it is in a folded state, in which case the central axis A of the opening 30a is inclined.

The central axis A of the opening 30a may not strictly pass through the center of the opening 30a. An axis passing through the place can also be the central axis A. That is, the central axis A can be defined as an axis passing through the center of the opening 30a and its vicinity. An extension line of the central axis A intersects the mounting surface 100 .

In the following description, for convenience, as shown in FIG. The upper side of the illumination device 3 is referred to as the upper side of the illumination device 3, and the fourth side portion 30D side is referred to as the lower side of the illumination device 3. It can be in any direction.

The interiors of the first side portion 30A, the second side portion 30B, the third side portion 30C and the fourth side portion 30D of the lighting housing 30 are hollow. The first light projecting section 31 is housed inside the first side section 30A. A second light projecting section 32, a third light projecting section 33 and a fourth light projecting section 34 are housed inside the second side section 30B, the third side section 30C and the fourth side section 30D, respectively. The first light projecting section 31 and the second light projecting section 32 form a pair, and the third light projecting section 33 and the fourth light projecting section 34 form a pair. A light projection control unit 39 is also housed inside the illumination housing 30 .

Since the first side portion 30A and the second side portion 30B are arranged to face each other across the central axis A, the first light projecting portion 31 and the second light projecting portion 32 are symmetrical about the central axis A. They are arranged so as to be bilaterally symmetrical (point symmetrical) with respect to the center, and are separated from each other in the circumferential direction of the central axis A.

Further, the third side portion 30C and the fourth side portion 30D are also arranged so as to face each other with the central axis A interposed therebetween. They are arranged so as to be vertically symmetrical (point symmetrical) with respect to the center, and are separated from each other in the circumferential direction of the central axis A. In a plan view, four light projecting units are arranged in the order of a first light projecting unit 31, a third light projecting unit 33, a second light projecting unit 33, and a fourth light projecting unit 34 clockwise around the central axis A. will be

FIG. 5 is a bottom view of the illumination device 3 according to Embodiment 2 of the present invention. In this second embodiment, eight light projection units 31 to 38 are provided inside the illumination housing 30 . A fifth light projecting portion 35 is provided between the first light projecting portion 31 and the third light projecting portion 33 , and a sixth light projecting portion 36 is provided between the second light projecting portion 32 and the fourth light projecting portion 34 . is provided, a seventh light projecting portion 37 is provided between the second light projecting portion 32 and the third light projecting portion 33, and an eighth light projecting portion 37 is provided between the first light projecting portion 33 and the fourth light projecting portion 34 A light projecting section 38 is provided. The fifth light projecting part 35 and the sixth light projecting part 36 are a pair, and are arranged so as to be symmetrical with each other with the central axis A as the center of symmetry. Also, the seventh light projecting part 37 and the eighth light projecting part 38 are a pair, and are arranged so as to be symmetrical with each other with the central axis A as the center of symmetry. In a plan view, a first light projecting part 31, a fifth light projecting part 35, a third light projecting part 33, a seventh light projecting part 37, a second light projecting part 32, a The 6 light projecting section 36, the 4th light projecting section 34, and the 8th light projecting section 38 are arranged in this order.

Further, in the illumination device 3 of the second embodiment, a uniform light illumination 50 (ring illumination) for illuminating the measurement object W with uniform light is provided separately from the light projection units 31 to 38 . The uniform light illumination 50 is provided on the outer peripheral portion of the bottom of the illumination housing 30 and has a ring shape surrounding the first to eighth light projection portions 31 to 38 . As shown in FIG. 6, the uniform light illumination 50 includes a substrate 51a, a plurality of uniform light emitting diodes 51b mounted on the substrate 51a and emitting uniform light, and a cover member 52. there is The substrate 51a is arranged so as to surround the first to eighth light projecting sections 31-38. The plurality of light emitting diodes 51b for uniform light are provided at intervals in the circumferential direction so as to surround the first to eighth light projecting portions 31 to . The cover member 52 is provided so as to cover the light-emitting diode 41b for uniform light from the light emitting surface side, and is made of a material that is translucent and has the property of diffusing light. Although not shown, illumination for uniform light can also be provided in the illumination device 3 of Embodiment 1 shown in FIG.

(Configuration of first light projecting section 31)
The first to fourth light projecting sections 31 to 34 of the lighting device 1 of the first embodiment are the same as the first to fourth light projecting sections 31 to 34 of the lighting device 1 of the second embodiment.

As shown in FIG. 6, the first light projecting portion 31 includes a casing 31a, a first LED (light emitting diode) 31b serving as a first light source for emitting diffused light, and receiving diffused light emitted from the first LED 31b. A first LCD (first pattern light generator) 31d that sequentially generates a plurality of first measurement pattern lights having patterns different from each other and irradiates the object W to be measured is provided. LCD is a liquid crystal display, ie a liquid crystal display that includes a liquid crystal panel. Although the first LCD 31d is indicated as "LCD" for convenience of explanation, it may be an LCP, that is, a liquid crystal panel (first liquid crystal panel). The light source is not limited to a light-emitting diode, and may be any light-emitting body that emits diffused light.

The first LCD 31d is arranged corresponding to the first LED 31b, and the light emitting surface of the first LED 31b faces the first LCD 31d. This ensures that the light emitted from the first LED 31b enters the first LCD 31d. For example, a white light emitting diode can be used as the light emitting diode. The first LCD 31d is provided so that the display surface (emission surface) of the first LCD 31d is positioned within a plane (indicated by a virtual line 200 in FIG. 6) perpendicular to the central axis A of the opening 30a of the illumination housing 30. The plane 200 and the display surface of the first LCD 31d are located on the same plane.

The second light projecting section 32 is configured similarly to the first light projecting section 31 . Specifically, as shown in FIG. 6, the second light projecting unit 32 includes a casing 32a, a second LED 32b mounted on a substrate 32c, and a second LCD (second pattern light) arranged corresponding to the second LED 32b. generator) 32d. The first LED 31b and the second LED 32b are paired. The first LED 31b and the second LED 32b are attached to the lighting housing 30 so that their relative positions can be corrected. Details of relative position correction will be described later.

The configuration of the first light projecting section 31 will be described in detail below. A plurality of first LEDs 31 b of the first light projecting section 31 are provided at an upper portion within the lighting housing 30 . The arrangement direction of the first LEDs 31b is a direction intersecting with the light emission direction.

That is, the substrate 31c is disposed above the casing 31a inside the casing 31a. A plurality of first LEDs 31b are mounted on the downward facing surface of the substrate 31c. The plurality of first LEDs 31b can be arranged in a straight line, or can be arranged such that adjacent first LEDs 31b are shifted in the vertical direction. When generating pattern light having a periodic illuminance distribution that varies in one dimension, the first LEDs 31b are arranged in a direction in which the illuminance of the pattern light does not change. By arranging the plurality of first LEDs 31b along the longitudinal direction of the first side portion 30A of the lighting housing 30 shown in FIG. Become.

The light emission directions of the plurality of first LEDs 31b can be the same, and in this embodiment, as indicated by the diagonal lines sloping to the left in FIG. It is set to reach the second side portion 30B side (the right side of the illumination device 3) from A. The light irradiation range of the plurality of first LEDs 31b is set wider than the imaging field of view of the imaging device 2 .

A specific description will be given of the light emission range of the plurality of first LEDs 31b. As shown in FIG. 7, the separation direction between the first light projecting section 31 and the second light projecting section 32 is defined as the X direction, and the vertical direction is defined as the Z direction. The illumination device 3 is arranged so that the lower surface of the illumination device 3 is horizontal (parallel to the mounting surface 100), and is installed at a distance of "1" above the mounting surface 100 of the object W to be measured. shall be A straight line D extending from the first LED 31b to a point C at (X, Z)=(0, 0) is drawn with X=0 directly below the first LED 31b. Also, a straight line F extending from the first LED 31b to a point E of (X, Z)=(1, 0) is drawn. At this time, the direction of the first LEDs 31b is set and the light source lens of the first LEDs 31b is designed so that the light emission range of the plurality of first LEDs 31b is an area sandwiched between the straight line D and the straight line F. there is

As shown in FIG. 6, the first LCD 31d is provided in the lighting housing 30 in a state separated downward from the first LED 31b. The driving method of the first LCD 31d is a TN (Twisted Nematic) method. Therefore, when the voltage applied to the first LCD 31d is 0, the liquid crystal composition (liquid crystal molecules) is aligned parallel to the display surface and allows the light of the first LED 31b to pass through. rises perpendicularly to the display surface, and blocks the light from the first LED 31b when the voltage reaches the maximum voltage.

(Relative position of LED and LED)
A relative positional relationship between the first LED 31b and the first LCD 31d will be described. As shown in FIG. 8, the display surface of the first LCD 31d is located on the same plane as a plane 200 perpendicular to the central axis A of the opening 30a of the illumination housing 30, and the first LCD 31d is located outside the illumination housing 30 in the radial direction. The other end is defined as the outer end side, and the radially inner end of the illumination housing 30 in the first LCD 31d is defined as the inner end side. The first LED 31b is arranged above the outer end side of the first LCD 31d. Symbol SEG in FIG. 8 denotes a segment that constitutes the first LCD 31d. A black segment SEG is a segment that does not transmit light, and a white segment SEG is a segment that transmits light. By alternately forming the black segment SEG and the white segment SEG on the display surface of the first LCD 31d in this way, the light transmitted through the first LCD 31d has a periodic intensity as shown below. It becomes a sinusoidal pattern light that changes to . This is the principle of pattern light generation. The number, interval, and formation position of the black segment SEG and the white segment SEG can be arbitrarily changed, and multiple types of pattern light with different wave periods and phases can be generated.

The number of black segments SEG and the number of white segments SEG can be the same. Therefore, when the pattern light is generated, the first LCD 31d can alternately form the light-transmitting region and the light-not-transmitting region with the same width. This is performed by a light projection control section 39, which will be described later.

Here, let us explain the general characteristics of liquid crystal panels. Light transmittance is highest when light is incident parallel to the normal to the display surface of the liquid crystal panel. The greater the angle formed by the two, the lower the light transmittance. Liquid crystal panels have angular characteristics in which light transmittance varies depending on the direction of light incidence. Therefore, when pattern light is generated using a liquid crystal panel, luminance unevenness occurs according to the position of the pattern light. There is concern that

In contrast, in the present embodiment, the relative positions of the first LCD 31d and the first LED 31b are set so that the light emitted from the first LED 31b is incident on the first LCD 31d within the effective angle range of the first LCD 31d. It is As a result, the diffused light emitted from the first LED 31b is incident on the first LCD 31d within the effective angle range of the first LCD 31d, so that uneven brightness according to the position of the pattern light due to the angular characteristics of the first LCD 31d is less likely to occur. Become.

The effective angular range of the first LCD 31d can be an angular range in which a predetermined or more contrast of pattern light can be secured. The fact that the contrast of the pattern light is equal to or higher than a predetermined level means that when the imaging device 2 receives the pattern light reflected from the object W to be measured, the imaging device 2 can obtain a pattern image capable of generating an image to be inspected. be. That is, due to the angular characteristics of the first LCD 31d, the greater the angle formed between the normal to the display surface of the first LCD 31d and the light incident direction, the lower the light transmittance. Although it is conceivable that the contrast of the pattern light is lowered, in this embodiment, the contrast is at least the level at which the height information of the measurement object W can be obtained based on the pattern image captured by the imaging device 2. The relative positions of the first LCD 31d and the first LED 31b are set such that

Also, the effective angle range of the first LCD 31d can be defined as follows. For example, when the light emitted from the first LED 31b passes through the first LCD 31d in which the liquid crystal molecules are aligned most easily, the angle range in which the light is attenuated by 10% or less is defined as the effective angle range of the first LCD 31d. Angle range. By setting the relative positions of the first LCD 31d and the first LED 31b so that the light attenuation rate is 10% or less, the height information of the measurement object W is obtained based on the pattern image captured by the imaging device 2. It is possible to secure the contrast to the extent possible.

The liquid crystal molecule alignment state that allows light to pass through most is the state in which the liquid crystal molecules are aligned in the direction that does not block the light path the least (when the voltage is 0 as described above). The effective angle range of the first LCD 31d can also be set so that the attenuation rate is 5% or less.

Further, as shown in FIG. 8, a normal line 201 drawn from the center of the first LED 31b (the center of the light emitting surface) toward the display surface of the first LCD 31d, and illumination in the effective angle range of the first LCD 31d from the center of the first LED 31b The relative positions of the first LED 31b and the first LCD 31d are set so that the angle formed by the inner virtual line 202 drawn toward the boundary on the inner end side located radially inward of the housing 30 is 50° or less. be able to. As a result, the angle range in which the light emitted from the first LED 31b is incident on the first LCD 31d becomes the optimal range for the TN liquid crystal panel, and as a result, the contrast of the pattern light can be ensured above a predetermined level. The angle α between the normal 201 and the inner virtual line 202 can be 45° or less, and can also be 40° or less.

(LED drive circuit and LCD drive circuit)
As shown in FIG. 9, the first light projection unit 31 includes a first LED drive circuit (light source drive circuit) 31e that drives the first LED 31b, and a first LCD drive circuit (liquid crystal panel drive circuit) 31f that drives the first LCD 31d. is provided. The first LED drive circuit 31 e is a circuit for changing the supply current value to the first LED 31 b and is controlled by the light projection control section 39 . Therefore, the first LED 31b is controlled by the light projection control section 39 via the first LED drive circuit 31e. Current value control by the first LED drive circuit 31e is DAC control.

The first LCD driving circuit 31f is a circuit for changing the orientation of the liquid crystal composition of each segment SEG by changing the voltage applied to each segment SEG (shown in FIG. 8) constituting the first LCD 31d. In this embodiment, as an example, there are 64 segments SEG that constitute the first LCD 31d, and the voltage applied to each of the 64 segments SEG can be individually changed. It is possible to switch between a state in which the light emitted from the 1LED 31b is allowed to pass and a state in which the light is not allowed to pass. The first LCD drive circuit 31f is controlled by a light projection control section 39 shared with the first LED drive circuit 31e. Therefore, the first LCD 31d is controlled by the light projection control section 39 via the first LCD drive circuit 31f. Further, since the first LED driving circuit 31e and the first LCD driving circuit 31f are controlled by the common light projection control section 39, the first LED driving circuit 31e and the first LCD driving circuit 31f can be driven precisely.

The first LCD driving circuit 31f controlled by the light projection control unit 39 drives the first LCD 31d to receive diffused light emitted from the first LED 31b and sequentially generate a plurality of first measurement pattern lights having different patterns. Then, the object W to be measured can be irradiated with the light. The plurality of first measurement pattern lights include pattern light for a spatial code (gray code) used in the spatial code method and pattern light having a periodic illuminance distribution used in the phase shift method.

When generating the pattern light for the spatial code, a stripe pattern is generated in which the width of the stripes is reduced to half, 1/4, . By controlling the first LCD 31d in this manner, it is possible to sequentially generate the pattern light for the spatial code.

In the case of generating pattern light for the phase shift method, a plurality of sinusoidal fringe patterns are generated with the phases shifted. In the case of this example, the LCD display is binary controlled to generate a rectangular wave pattern. However, as shown in FIG. A sinusoidal pattern can be obtained. More specifically, a pattern close to a sine wave can be obtained by combining the rectangular wave pattern formed on the liquid crystal panel and the light emitting pattern of the light emitting diodes having an area. If we had an ideal point or line light source instead of an LED, we would get a binary pattern instead of a sinusoidal pattern. Therefore, in order to obtain a sinusoidal pattern, the balance between the LED light source size and the LCD aperture size is important. Also, in this example, eight pattern lights are generated. By controlling the first LCD 31d in this manner, it is possible to sequentially generate pattern light for the phase shift method.

That is, the light projection control section 39 controls the first LED 31b and the first LCD 31d so as to sequentially generate a plurality of pattern lights according to the phase shift method and/or the spatial code method. When the projection of one pattern light among the plurality of pattern lights is completed, the next pattern light is projected. By repeating this process, all the pattern lights are projected. The pattern forming process by the first LCD 31d will be described later.

The number of pattern lights for the spatial code and the number of pattern lights for the phase shift method are not limited to the numbers shown.

(Configuration of the second light projecting section 32)
As shown in FIG. 1, the light emission range of the second LED 32b of the second light projecting portion 32 is at least on the first side portion 30A side of the central axis A of the opening portion 30a of the illumination housing 30 from directly below the second LED 32b (illumination left side of device 3). That is, the light emission range of the second LED 32b of the second light projecting section 32 is the light emission range of the first LED 31b of the first light projecting section 31 with the central axis A of the opening 30a of the illumination housing 30 as the center of symmetry. It is designed to be bilaterally symmetrical. The light emission range of the second LED 32b is indicated by diagonal lines sloping downward to the right in FIG.

As shown in FIG. 9, the second light projection unit 32 includes a second LED drive circuit (light source drive circuit) 32e that drives the second LED 32b, and a second LCD drive circuit (liquid crystal panel drive circuit) 32f that drives the second LCD 32d. are provided, and these are controlled by the light projection control section 39 . Since the second LCD 32d is driven in the same manner as the first LCD 31d, the second LCD 32d receives the diffused light emitted from the second LED 32b, sequentially generates a plurality of second measurement pattern lights having different patterns, and irradiates the object W to be measured. be able to. The plurality of second pattern lights for measurement include pattern light for the spatial code and pattern light for the phase shift method.

As shown in FIG. 1, the pattern light emitted from the first light projection section 31 and the pattern light emitted from the second light projection section 32 intersect on the central axis A of the opening 30a of the illumination housing 30. , the first light projecting part 31 and the second light projecting part 32 are integrally supported on the lighting housing 30 in a state in which they are separated from each other in the circumferential direction of the central axis A so as to have substantially the same spread angle. It is “Integratedly supported” means that the first light projecting unit 31 and the second light projecting unit 32 are supported so that the relative positional relationship between the first light projecting unit 31 and the second light projecting unit 32 does not change during installation or use. That is, the second light projecting part 32 is fixed with respect to the lighting housing 30 . Therefore, during operation, the relative positions of the first light projecting part 31 and the second light projecting part 32 in the illumination housing 30 do not change, so that the center of the first LED 31b and the center of the second LED 32b are arranged as shown in FIG. If l is set as the distance between the center of the first LED 31b and the center of the second LED 32b during operation, the distance between the center of the first LED 31b and the center of the second LED 32b is fixed to l. The distance between the central portion of the first LED 31b and the central portion of the second LED 32b is the relative position information of the first light projecting portion 31 and the second light projecting portion 32 in the lighting housing 30. 2 can be stored. It should be noted that the separation distance between the central portion of the first LED 31b and the central portion of the second LED 32b can be changed when not in operation.

Further, the relative positional information of the first light projecting portion 31 and the second light projecting portion 32 in the lighting housing 30 may be a straight line distance between the center of the first LED 31b and the center of the second LED 32b. When the light emitted from the LED is turned back by a mirror or the like to irradiate the object W to be measured, the distance can be set in consideration of the path length of the light.

Since the first LCD 31 d is arranged on the left side of the illumination device 3 , it projects pattern light from the left side onto the measurement object W placed on the placement surface 100 . Further, since the second LCD 32d is arranged on the right side of the lighting device 3, it projects pattern light onto the measurement object W placed on the placement surface 100 from the right side. The first LCD 31d and the second LCD 32d are liquid crystal panels that project pattern light onto the object W to be measured from different directions.

(Configuration of the third light projecting section 33 and the fourth light projecting section 34)
The third light projecting section 33 and the fourth light projecting section 34 are configured in the same manner as the first light projecting section 31. As shown in FIG. 9, the third light projecting section 33 includes a third LED (third light source ) 33b, and a third LCD (third pattern light generator) 33d arranged corresponding to the third LED 33b. and a fourth LCD (fourth pattern light generating section) 34d arranged in a line. The third LED 33b and the fourth LED 34b are paired. The third LED 33b and the fourth LED 34b are attached to the lighting housing 30 so that their relative positions can be corrected. Details of relative position correction will be described later.

The display surfaces of the third LCD 33d and the fourth LCD 34d are positioned on the same plane as the plane perpendicular to the central axis A of the opening 30a of the illumination housing 30 (the plane denoted by reference numeral 200 in FIG. 6).

The light emission range of the third LED 33b of the third light projecting section 33 and the light emission range of the fourth LED 34b of the fourth light projecting section 34 are defined by the light emission range of the first LED 31b of the first light projecting section 31 and the light emission range of the fourth LED 34b. It is set to be the same as the relationship with the light emission range of the second LED 32 b of the second light projecting section 32 . Specifically, the light emission range of the third LED 33b of the third light projecting portion 33 is set so as to reach at least the fourth side portion 30D side of the central axis A of the opening 30a of the illumination housing 30 from directly below the third LED 33b. is set. The light emission range of the fourth LED 34b of the fourth light projecting portion 34 is set to reach at least the third side portion 30C side of the central axis A of the opening 30a of the lighting housing 30 from directly below the fourth LED 34b. Therefore, when the central axis A of the opening 30a of the illumination housing 30 is taken as the center of symmetry, the light emission range of the third LED 33b of the third light projecting part 33 and the light emission range of the fourth light projecting part 34 are vertically symmetrical. and the light emission range of the fourth LED 34b are set.

As shown in FIG. 9, the third light projection unit 33 includes a third LED drive circuit (light source drive circuit) 33e that drives the third LED 33b, and a third LCD drive circuit (liquid crystal panel drive circuit) 33f that drives the third LCD 33d. are provided, and these are controlled by the light projection control section 39 . Since the third LCD 33d is driven in the same manner as the first LCD 31d, the third LCD 33d receives the diffused light emitted from the third LED 33b, sequentially generates a plurality of third measurement pattern lights having mutually different patterns, and irradiates the object W to be measured. be able to. The plurality of third measurement pattern lights include pattern light for spatial code and pattern light for phase shift method.

The fourth light projecting section 34 is provided with a fourth LED driving circuit (light source driving circuit) 34e for driving the fourth LED 34b and a fourth LCD driving circuit (liquid crystal panel driving circuit) 34f for driving the fourth LCD 34d. , are controlled by the projection control unit 39 . Since the fourth LCD 34d is driven in the same manner as the first LCD 31d, the fourth LCD 34d receives diffused light emitted from the fourth LED 34b, sequentially generates a plurality of fourth measurement pattern lights having different patterns, and irradiates the object W to be measured. be able to. The plurality of fourth measurement pattern lights includes pattern light for spatial code and pattern light for phase shift method.

The pattern light emitted from the third light projecting part 33 and the pattern light emitted from the fourth light projecting part 34 are substantially the same so that they intersect on the central axis A of the opening 30 a of the illumination housing 30 . The third light projecting portion 33 and the fourth light projecting portion 34 are integrally supported by the lighting housing 30 while being separated from each other in the circumferential direction of the central axis A so as to have a spread angle. Therefore, since the relative positions of the third light projecting portion 33 and the fourth light projecting portion 34 in the lighting housing 30 do not change, the distance between the center of the third LED 33b and the center of the fourth LED 34b is set to a predetermined value in advance. If set, the separation distance between the center of the third LED 33b and the center of the fourth LED 34b is fixed at a predetermined value during operation. The distance between the central portion of the third LED 33b and the central portion of the fourth LED 34b is the relative position information of the third light projecting portion 33 and the fourth light projecting portion 34 in the lighting housing 30. 2 can be stored.

Since the third LCD 33d is arranged above the illumination device 3, it projects pattern light onto the measurement object W placed on the placement surface 100 from that direction. Further, since the fourth LCD 34d is arranged below the illumination device 3, it projects pattern light onto the measurement object W placed on the placement surface 100 from that direction. The third LCD 33d and the fourth LCD 34d are liquid crystal panels that project pattern light onto the object W to be measured from different directions.

The fifth to eighth light projecting sections 35 to 38 according to the second embodiment shown in FIG. 5 are configured similarly to the first to fourth light projecting sections 31 to 34. As shown in FIG.

(Control by light projection control unit 39)
As shown in FIG. 9, in this embodiment, a first LED driving circuit 31e, a second LED driving circuit 32e, a third LED driving circuit 33e, a fourth LED driving circuit 34e, a first LCD driving circuit 31f, a second LCD driving circuit 32f, and a third LCD Since the drive circuit 33f and the fourth LCD drive circuit 34f are controlled by the common light projection control section 39, these drive circuits can be precisely synchronized. Similarly, the fifth to eighth light projecting sections 35 to 38 of Embodiment 2 also have an LED drive circuit and an LCD drive circuit, and can be precisely synchronized.

As shown in FIG. 10, the light projection control unit 39 completes projection of one of the plurality of pattern lights from any one of the first LCD 31d, the second LCD 32d, the third LCD 33d, and the fourth LCD 34d. By the time the pattern light is projected next, at least on another liquid crystal panel to which the pattern light is to be projected next, the formation processing of the pattern to be projected next is completed, and after the projection of the pattern light by the one liquid crystal panel is completed, the other liquid crystal panel The first LCD 31d, the second LCD 32d, the third LCD 33d, and the fourth LCD 34d are controlled so as to repeatedly execute the process of projecting the next pattern light onto the liquid crystal panel.

Specifically, the light projection control unit 39 of the illumination device 3 receives from the controller unit 4 a trigger signal for starting projection of the pattern light and a replay signal for synchronizing with the imaging device 2 during projection of the pattern light. A synchronous trigger signal is input. A trigger signal may be input from the PLC 101 . For example, a trigger signal can be input to the light projection control unit 39 based on the detection result of a photoelectric sensor or the like connected to the PLC 101 . The device that generates the trigger signal may not be the PLC 101, but may be a photoelectric sensor or the like. In this case, a photoelectric sensor or the like can be directly connected to the light projection control section 39 or connected via the controller section 4 .

The pattern formation and light projection order of the first LCD 31d, the second LCD 32d, the third LCD 33d and the fourth LCD 34d will be described in detail below with reference to FIG. When the trigger signal is input, the light projection control unit 39 switches the pattern (phase shift 1) currently formed on the first LCD 31d to a pattern (phase shift 2) different from the current display mode. , controls the first LCD 31d via the first LCD drive circuit 31f. Here, in order to switch the pattern on the first LCD 31d from the phase shift 1 to the phase shift 2, the voltage applied to the liquid crystal composition of each segment constituting the first LCD 31d is changed by the first LED driving circuit 31e by a known method. Let The time from when the voltage applied to the liquid crystal composition is changed until the orientation of the liquid crystal composition is changed is longer than the imaging interval of the imaging device 2, which will be described later. That is, in order to switch the pattern (phase shift 1) currently formed on the first LCD 31d to a different pattern (phase shift 2), a predetermined pattern switching time longer than the imaging interval of the imaging device 2 is required. Similarly, the second LCD 32d, the third LCD 33d and the fourth LCD 34d also require time to switch the pattern from phase shift 1 to phase shift 2 in order.

While the pattern (phase shift 1) is completely formed on the first LCD 31d, light is emitted from the first LED 31b in synchronization with the formation of the pattern (phase shift 1), and light is emitted from the second LED 32b, the third LED 33b and the fourth LED 34b. controls so as not to emit light. As a result, only the pattern (phase shift 1) formed on the first LCD 31d is projected onto the measurement target W as pattern light. There is no projection onto the object W.

The time during which the pattern (phase shift 1) is formed on the first LCD 31d is part of the pattern switching time for forming the pattern on the second LCD 32d. The time for forming the pattern on the second LCD 32d is longer than the time for forming the pattern (phase shift 1) on the first LCD 31d. begins before the formation of the is complete.

When the imaging of the pattern light of the pattern (phase shift 1) projected onto the measurement object W is completed, while the pattern (phase shift 1) is completely formed on the second LCD 32d, the pattern (phase shift 1) is emitted from the second LED 32b in synchronization with the formation of . As a result, only the pattern (phase shift 1) formed on the second LCD 32d is projected onto the measuring object W as pattern light.

The time during which the pattern (phase shift 1) is formed on the second LCD 32d is part of the switching time for forming the pattern on the third LCD 33d. The time for forming the pattern (phase shift 1) on the third LCD 33d is longer than the time for forming the pattern (phase shift 1) on the second LCD 32d. It begins before the formation of (phase shift 1) is completed.

When the imaging of the pattern light of the pattern (phase shift 1) projected onto the measurement object W is completed, while the pattern (phase shift 1) is completely formed on the third LCD 33d, the pattern (phase shift 1) is emitted from the third LED 33b in synchronization with the formation of . As a result, only the pattern (phase shift 1) formed on the third LCD 33d is projected onto the measuring object W as pattern light.

The time during which the pattern (phase shift 1) is formed on the third LCD 33d is part of the switching time for forming the pattern (phase shift 1) on the fourth LCD 34d. The time for forming the pattern (phase shift 1) on the fourth LCD 34d is longer than the time for forming the pattern (phase shift 1) on the third LCD 33d. It begins before the formation of (phase shift 1) is complete.

When the imaging of the pattern light of the pattern (phase shift 1) projected onto the measurement object W is completed, while the pattern (phase shift 1) is completely formed on the fourth LCD 34d, the pattern (phase shift 1) , light is emitted from the fourth LED 34b, and light is not emitted from the first LED 31b, the second LED 32b, and the third LED 33b. As a result, only the pattern (phase shift 1) formed on the fourth LCD 34d is projected onto the measuring object W as pattern light. Part of the time during which this pattern is formed is part of the switching time for forming the next pattern (phase shift 2) on the first LCD 31d.

That is, in this embodiment, one of the first LCD 31d, the second LCD 32d, the third LCD 33d, and the fourth LCD 34d does not successively project a plurality of pattern lights by one liquid crystal panel. When the projection of the pattern light is completed, the first pattern light is projected by another liquid crystal panel, and when the projection of the first pattern light by the another liquid crystal panel is completed, the first pattern is projected by another liquid crystal panel. When the light is projected and the projection of the first pattern light is completed in all the liquid crystal panels, the second pattern light is projected by the one liquid crystal panel, and the second pattern light is projected by the one liquid crystal panel. When the projection of the second pattern light is completed, the second pattern light is projected by another liquid crystal panel, and when the projection of the second pattern light by the another liquid crystal panel is completed, the second pattern light is projected by another liquid crystal panel. The first LCD 31d, the second LCD 32d, the third LCD 33d, and the fourth LCD 34d are controlled so as to project the pattern light. Therefore, the liquid crystal panel on which pattern light is not projected can be prepared for forming a pattern to be projected next, so that the slow response speed of the liquid crystal panel can be compensated for.

In the above example, the case where the pattern light is projected by all of the first LCD 31d, the second LCD 32d, the third LCD 33d, and the fourth LCD 34d has been described. The pattern light can also be projected only by the 3LCD 33d and the 4th LCD 34d. When the pattern light is projected only by the first LCD 31d and the second LCD 32d, the pattern light may be projected alternately. For example, while the first pattern light is being projected by the first LCD 31d, A first pattern formation process is performed, and then a second pattern formation process is performed on the first LCD 31d while the first pattern light is being projected on the second LCD 32d, and this is repeated. The same is true when pattern light is projected only by the third LCD 33d and the fourth LCD 34d.

In addition to the trigger signal and the resynchronization trigger signal, pattern light formation information is also transmitted from the controller 4 to the light projection control section 39 , and the transmitted pattern light formation information is temporarily sent to the light projection control section 39 . The first LED 31b, the second LED 32b, the third LED 33b and the fourth LED 34b, and the first LCD 31d, the second LCD 32d, the third LCD 33d and the fourth LCD 34d are controlled based on the stored pattern light formation information.

The pattern light formation information includes, for example, the irradiation mode, the presence or absence of irradiation of the pattern light for the spatial code, the specific pattern and number of the pattern light for the spatial code, the presence or absence of the irradiation of the pattern light for the phase shift method, The specific pattern and number of pattern lights for the phase shift method, the irradiation order of the pattern lights, and the like are included. The irradiation modes include a first irradiation mode in which only the first LCD 31d and the second LCD 32d irradiate the pattern light and project it onto the measurement object W, and a first irradiation mode in which the pattern light is projected on the measurement object W, and all of the first LCD 31d, the second LCD 32d, the third LCD 33d, and the fourth LCD 34d project the pattern light. A second irradiation mode and a third irradiation mode in which pattern light is irradiated only by the third LCD 33d and the fourth LCD 34d and projected onto the measurement object W are included.

(Irradiation order of pattern light)
FIG. 11 shows an example of the irradiation order of the pattern light, and FIG. 12 shows the light projection interval between the opposite phases of the pattern light. Before describing the irradiation order of the pattern light, the influence of ambient light will be described with reference to FIGS. 14 and 15. FIG. FIG. 14 is a graph showing changes over time in brightness of disturbance light on the surface of the measurement object W placed on the placement surface 100 . The surface of the object W to be measured may receive not only the light emitted from the illumination device 3 but also the light emitted from another illumination device installed in the surroundings or sunlight. The light irradiated onto the surface of the object W to be measured from other than the illumination device 3 is ambient light. When an object temporarily passes between the light source of the ambient light and the object W to be measured, or when the intensity of the light emitted from the light source of the ambient light changes, the surface of the object W is irradiated. This means that the brightness of the disturbance light that is generated changes. As shown in FIG. 14, the variation in the brightness of the disturbance light, if the variation width is b1 at time t1, greatly expands to b2 at time t2. That is, the longer the time, the greater the variation in the brightness of the ambient light with which the surface of the object W to be measured is irradiated.

In this embodiment, as described above, one measurement object W is irradiated with a large number of phase-shifted pattern lights, and a pattern image is generated each time. become longer. Therefore, fluctuations in the brightness of ambient light tend to increase.

As shown in FIG. 15, when the fluctuation of the brightness of the ambient light increases, the phase calculation error caused by the brightness of the ambient light becomes a problem. As shown in this figure, the error has an undulating shape close to a sine wave, and the magnitude of this error varies depending on the degree of fluctuation in the brightness of the ambient light. occurs, it appears as a regular waveform, which is conspicuous to the user and may cause erroneous determination.

In particular, in the case of a large imaging device 2, the amount of disturbance light tends to increase relative to the light amount ratio of the pattern light, resulting in a large phase calculation error. There is a growing need to reduce the errors caused by small fluctuations.

FIG. 11 shows a case where there are 8 phase-shifted light projections 0 to 7 as imaging sequence numbers for a certain direction. The first projection mode has eight patterns of 0°, 45°, 90°, 135°, 180°, 225°, 270°, and 315°, which correspond to phase shift projections 0-7. there is Therefore, after projecting pattern light with a phase of 0°, pattern light with a phase of 45° is projected, and then pattern light with a phase of 90° is projected. , 270° and 315°. The number of phase shifts may be other than eight, and may be four, six, or the like.

FIG. 12 shows the time interval from the projection of the pattern to the projection of the opposite phase pattern for each of the first projection mode and the second projection mode shown in FIG.

In the case of the first light projection mode, as shown in FIG. 12, when the illumination direction is not switched, the imaging interval until the opposite phase pattern is imaged is 4 images. is phase-shifted, 8 images are taken, every 4 directions is 16 images, and every 8 directions is 32 images. Therefore, the error caused by the brightness variation of the disturbance light tends to become conspicuous.

The reason why the time interval between projections of patterns with opposite phases is important is that phase calculation is performed by the following equation. The following formula is a formula when imaging is performed 4 times, but cases other than 4 such as 6 times and 8 times can also be described by mathematical formulas that produce differences between opposite phases.

Phase φ=tan ⁻¹ {(I _270° −I _90° )/(I _0° −I _180° )}
As shown by this equation, the difference between opposite phases is important in phase calculation. In the present embodiment, in order to bring the timing of imaging at which the phase of the pattern light is opposite to each other, that is, to minimize the brightness fluctuation of the disturbance light during two imagings at which the phase of the pattern light is opposite to each other. , can execute light projection in the second light projection mode shown in FIG.

The second projection mode has eight patterns of 0°, 180°, 45°, 225°, 90°, 270°, 135°, and 315°, which correspond to phase shift projections 0-7. there is Therefore, after projecting pattern light with a phase of 0°, pattern light with a phase of 180°, that is, pattern light with a phase opposite to 0° is projected, and then pattern light with a phase of 45° is projected. , pattern light with a phase of 225°, i.e., pattern light with a phase opposite to 45°, is projected, then pattern light with a phase of 90° is projected, and then pattern light with a phase of 270°, i.e., pattern light with a phase opposite to 90°, is projected. A pattern light with a phase is projected, then a pattern light with a phase of 135° is projected, and then a pattern light with a phase of 315°, that is, a pattern light with a phase opposite to 135° is projected. The pattern light of 0° and the pattern light of 180° form an opposite phase pair, the pattern light of 45° and the pattern light of 225° form an opposite phase pair, and the pattern light of 90° and the pattern light of 270° form an opposite phase pair. A pair of opposite phases is formed, and the pattern light of 135° and the pattern light of 315° form a pair of opposite phases. In other words, the phase of the pattern light projected at the odd-numbered order and the phase of the pattern light projected at the even-numbered order are in an opposite phase relationship.

The order of pattern light projection is not limited to the order described above, and may be changed as long as the pair of opposite phases is maintained. For example, the order of projection of the pattern light may be 0°, 180°, 45°, 225°, 270°, 90°, 135°, 315°, or 0°, 180° , 225°, 45°, 90°, 270°, 315°, 135°. In these cases as well, the pattern light can be irradiated continuously in opposite phases.

In the case of the second projection mode, as shown in FIG. 12, in the case where the illumination direction is not switched but the phase is shifted, the imaging interval until the opposite phase pattern is imaged is one image. 2 imaging, 4 imaging for every 4 directions, and 8 imaging for every 8 directions, shorten the imaging interval at which the pattern light has the opposite phase. Specifically, the imaging interval at which the phase is reversed in the second light projection phase is 1/4 of that in the case of the first light projection phase. Therefore, the error caused by the brightness variation of the disturbance light described above is reduced.

FIG. 13 is a timing chart for imaging in only one direction and for imaging by switching the phase shift every two directions. The upper side of FIG. 13 is an example of switching the phase shift every two directions, and the lower side of FIG. 13 is an example of imaging in only one direction. When switching the phase shift for each of the two directions, when projecting light from a certain direction (assuming an upward direction), light is projected with "phase shift 0", and after completing imaging, the LCD in the upward direction is switched. . While switching the LCD in the upward direction, the light projection direction is switched to another direction (temporarily assumed to be the downward direction), and the light is projected with a "phase shift of 0" for imaging. When the switching of the upward LCD is completed, the upward "phase shift 1" is projected and an image is captured. This is repeated. Thus, in the case of two directions, since the phase shift is also performed while switching the projection direction, pattern light can be projected by the other LCD while the other LCD is being switched.

When capturing an image in only one direction, the LCD is switched after the light is projected with "phase shift 0" and the image capturing is completed. After the switching of the LCD is completed, "phase shift 1" is projected and an image is captured.

A reduction in phase error will be described based on the graph shown in FIG. The horizontal axis of the upper graph represents the fluctuation period of the brightness of the ambient light, which is indicated by the light projection time interval ratio of the pattern light. The disturbance light is assumed to be impulse-like fluctuating disturbance light, here a signal of a sine half wave is assumed as the disturbance light. The fluctuation of ambient light becomes slower toward the right side of the horizontal axis. Due to the high speed of imaging, generally possible fluctuating ambient light is close to the far right part of the graph. The vertical axis of the graph is the phase error. In the case of the second projection phase (continuous opposite phase), when the fluctuation period of the brightness of the disturbance light exceeds 3 in the projection time interval ratio, the phase The error is greatly reduced, and the phase error can be reduced even in the area on the right side of the graph. The horizontal axis of the lower graph in FIG. 16 is the same as the horizontal axis of the upper graph, but the vertical axis represents the phase error as the ratio between the first projection phase and the second projection phase. In the case of the second light projection phase, the phase error decreases to 30% or less compared to the first light projection phase when the fluctuation period of the brightness of the disturbance light exceeds 4 in terms of the light projection time interval ratio. As a result, the phase error can be minimized without taking countermeasures against light shielding or applying a band-pass filter to the imaging device 2, and as a result, the inspection accuracy can be improved. , can increase the stability of the inspection.

(Configuration of imaging device 2)
As shown in FIG. 1 and the like, the imaging device 2 is provided separately from the illumination device 3 . As shown in FIG. 1, the imaging device 2 and the controller section 4 are connected by a connection line 2a, but the imaging device 2 and the controller section 4 may be wirelessly connected.

Since the imaging device 2 constitutes a part of the image processing device 1, it can also be called an imaging unit. Since the imaging device 2 is provided separately from the lighting device 3, the imaging device 2 and the lighting device 3 can be installed separately. Therefore, the installation location of the imaging device 2 and the installation location of the lighting device 3 can be changed, and the installation location of the imaging device 2 and the installation location of the lighting device 3 can be separated. As a result, the degree of freedom in installing the imaging device 2 and the lighting device 3 can be greatly improved, and the image processing device 1 can be introduced to any production site or the like.

In addition, if the installation location of the imaging device 2 and the installation location of the lighting device 3 can be the same at the site, the imaging device 2 and the lighting device 3 can be attached to the same member, and the installation state can be determined by the user. It can be arbitrarily changed according to the site. Also, the imaging device 2 and the lighting device 3 can be attached to the same member and used as an integrated unit.

The imaging device 2 is arranged above the lighting housing 30 of the lighting device 3, that is, on the side opposite to the direction in which the pattern light is emitted, so as to look into the opening 30a of the lighting housing 30. As shown in FIG. Therefore, the imaging device 2 receives the first measurement pattern light reflected from the measurement object W through the opening 30a of the illumination housing 30 of the illumination device 3 to generate a plurality of first pattern images, A plurality of second pattern images can be generated by receiving the second measurement pattern light reflected from the measurement object W. FIG. Further, when the lighting device 3 has the third light projecting section 33 and the fourth light projecting section 34 , the image capturing device 2 projects the measurement target through the opening 30 a of the lighting housing 30 of the lighting device 3 . A plurality of third pattern images are generated by receiving the third pattern light for measurement reflected from the object W, and a plurality of fourth pattern images are generated by receiving the fourth pattern light for measurement reflected from the object W to be measured. can be generated. Similarly, when the fifth to eighth light projection units 35 to 38 are provided, the fifth to eighth pattern images can be generated.

As shown in FIG. 3, the imaging device 2 includes a lens 21 that forms an optical system, and an imaging device 22 that is a light-receiving device that receives light incident from the lens 21 . , a so-called camera is configured. The lens 21 is a member for forming an image of at least a height measurement area or an inspection target area on the measurement object W on the imaging device 22 . The optical axis of the lens 21 may be aligned with the central axis A of the opening 30a of the illumination housing 30 of the lighting device 3, but they do not have to be aligned. In addition, the distance between the imaging device 2 and the lighting device 3 in the direction of the central axis A can be set arbitrarily within a range in which the lighting device 3 does not interfere with the imaging by the imaging device 2, and is designed with a high degree of freedom in installation. there is

Also, as the imaging element 22, a CCD, a CMOS sensor, or the like can be used. The imaging device 22 receives the reflected light from the object W to be measured, acquires an image, and outputs the acquired image data to the data processing unit 24 . In this example, a high resolution CMOS sensor is used as the imaging device 22 . It is also possible to use an imaging device capable of capturing images in color. The imaging device 22 can also capture a normal luminance image in addition to the pattern projection image. When capturing a normal luminance image, all the LEDs 31b, 32b, 33b, and 34b of the illumination device 3 are turned on, and all the LCDs 31d, 32d, 33d, and 34d are controlled so as not to form pattern light. good. If there is a uniform light illumination 50 as shown in FIGS. 5 and 6, it can be used to capture a normal luminance image with the imaging device 2. FIG.

The imaging device 2 further includes an exposure control unit 23, a data processing unit 24, a phase calculation unit 26, an image processing unit 27, an image storage unit 28, and an output control unit 29, in addition to the color camera. there is The data processing unit 24, the phase calculation unit 26, the image processing unit 27, and the image storage unit 28 are connected to a common bus line 25 built in the imaging device 2, and are capable of transmitting and receiving data to each other. . The exposure control section 23, the data processing section 24, the phase calculation section 26, the image processing section 27, the image storage section 28 and the output control section 29 can be configured by hardware or by software.

(Configuration of Exposure Control Unit 23)
A trigger signal for starting imaging and a resynchronization trigger signal for synchronizing with the illumination device 3 during imaging are input to the exposure control section 23 from the controller section 4 . Input timings of the trigger signal and the resynchronization trigger signal input to the exposure control unit 23 are set so as to be the same timings as the trigger signal and the resynchronization trigger signal input to the illumination device 3 .

The exposure control unit 23 is a part that directly controls the imaging device 22, and controls the imaging timing and exposure time of the imaging device 22 by the trigger signal and the resynchronization trigger signal input to the exposure control unit 23. The exposure control unit 23 is configured to receive and store information about imaging conditions from the controller unit 4 . The information about the imaging conditions includes, for example, the number of imaging times, the imaging interval (the time after imaging until the next imaging), the exposure time (shutter speed) at the time of imaging, and the like.

The exposure control section 23 causes the imaging device 22 to start imaging in response to the input of the trigger signal sent from the controller section 4 . In this embodiment, since it is necessary to generate a plurality of pattern images by inputting a single trigger signal, a resynchronization trigger signal is input from the controller unit 4 during imaging. Synchronization with the device 3 is possible.

Specifically, while the pattern completely formed on the first LCD 31 d is projected onto the measurement object W as pattern light, the exposure control unit 23 takes an image so that the image sensor 22 takes an image (exposure). Control the element 22 . The exposure time and the time during which the pattern is projected onto the object W to be measured as pattern light can be the same, but the timing at which the exposure is started can be slightly later than the timing at which the pattern light is started to be projected. good too.

After that, while the pattern formed on the second LCD 32d is projected onto the object W to be measured as pattern light, the exposure control unit 23 controls the imaging device 22 so that the imaging device 22 takes an image. When this imaging is completed, the exposure control unit 23 controls the imaging device 22 so that the imaging device 22 takes an image while the pattern formed on the third LCD 33d is projected onto the measurement object W as pattern light. do. After that, while the pattern formed on the fourth LCD 34d is projected onto the object W to be measured as pattern light, the exposure control unit 23 controls the imaging device 22 so that the imaging device 22 takes an image. By repeating this, a plurality of first pattern images, a plurality of second pattern images, a plurality of third pattern images, and a plurality of fourth pattern images are generated.

The imaging device 22 transfers the image data to the data processing unit 24 each time it completes imaging. The image data can be stored in the image storage section 28 shown in FIG. That is, since the image pickup timing by the image pickup device 22 and the image request timing of the controller unit 4 do not match, the image storage unit 28 functions as a buffer that absorbs this discrepancy.

The image data is transferred to the data processing unit 24 shown in FIG. 3 between the imaging and the next imaging. . When the image of the measurement object W irradiated with a certain pattern of light is completed, the image data of the previous pattern is stored while the image of the measurement object W irradiated with the next pattern of light is being imaged. Transfer to the processing unit 24 . In this manner, the image data captured last time can be transferred to the data processing unit 24 at the time of the next image capturing.

Further, it is also possible to image the measurement object W irradiated with pattern light of a certain pattern a plurality of times. In this case, the first LED 31b can be lit only when the imaging element 22 takes an image. The exposure time of the imaging element 22 can be set so that the first imaging is longer than the second imaging, and the second imaging is set to be longer than the first imaging. You can also When capturing an image of the measurement object W irradiated with pattern light of other patterns, the image can be captured multiple times. Thereby, while one pattern light among the plurality of pattern lights is being projected onto the measurement object W, a plurality of images with different exposure times can be generated. A plurality of images with different exposure times are used when performing high dynamic range processing, which will be described later. It should be noted that the first LED 31b may be kept on while the measurement object W irradiated with the pattern light of a certain pattern is imaged a plurality of times.

(Configuration of data processing unit 24)
The data processing unit 24 shown in FIG. 3 generates a plurality of pattern image sets based on the image data output from the imaging element 22. FIG. When the imaging device 22 generates a plurality of first pattern images, the data processing section 24 generates a first pattern image set consisting of a plurality of first pattern images. Similarly, generating a second pattern image set consisting of a plurality of second pattern images, generating a third pattern image set consisting of a plurality of third pattern images, and generating a fourth pattern image set consisting of a plurality of fourth pattern images to generate Therefore, the imaging device 2 can receive the reflected light from the measurement object W of the plurality of pattern lights projected from each liquid crystal panel, and generate a plurality of pattern image sets corresponding to each liquid crystal panel. .

When pattern light is projected only by the first LCD 31d and the second LCD 32d, a first pattern image set and a second pattern image set are generated. When pattern light is projected only by the third LCD 33d and the fourth LCD 34d, a third pattern image set and a fourth pattern image set are generated.

The data processing unit 24 can generate a phase shift pattern image set by projecting pattern light according to the phase shift method, and can generate a gray code pattern image set by projecting pattern light according to the spatial code method.

The pattern light according to the phase shift method is pattern light whose illuminance distribution is varied, for example, in a sinusoidal shape, but other pattern light may be used. In this embodiment, the number of pattern lights according to the phase shift method is eight, but the number is not limited to this. On the other hand, the pattern light according to the spatial code method is a striped pattern in which the striped width is reduced to half, quarter, . In this embodiment, the number of pattern lights according to the spatial code method is four, but the number is not limited to this. The pattern explained in this example is a case where the gray code is used as a spatial code, and it is not the purpose of the gray code to form pattern light by dividing the stripe width in half, but as a result It just happens. Also, the Gray code is a kind of code system in which noise resistance is taken into account by setting the Hamming distance to 1 between adjacent codes.

As shown in FIG. 17, when the first light projecting unit 31 of the illumination device 3 irradiates the measurement object W with four pattern lights according to the spatial code method, the data processing unit 24 consists of four different images. Generate a set of gray code pattern images. Further, when the first light projecting unit 31 of the illumination device 3 irradiates the measuring object W with eight pattern lights according to the phase shift method, the data processing unit 24 generates a phase shift pattern image set consisting of eight different images. to generate Both the gray code pattern image set and the phase shift pattern image set obtained by irradiation of the pattern light by the first light projecting section 31 are the first pattern image set.

Similarly, when the second light projecting unit 32 of the illumination device 3 irradiates the object W to be measured with pattern light according to the spatial code method, a gray code pattern image set is generated and a pattern according to the phase shift method is generated. When light is applied to the measurement object W, a phase shift pattern image set is generated. Both the gray code pattern image set and the phase shift pattern image set obtained by irradiation of the pattern light by the second light projecting section 32 are the second pattern image set.

Similarly, when the third light projecting unit 33 of the illumination device 3 irradiates the object W to be measured with pattern light according to the spatial code method, a gray code pattern image set is generated and a pattern according to the phase shift method is generated. When light is applied to the measurement object W, a phase shift pattern image set is generated. Both the gray code pattern image set and the phase shift pattern image set obtained by irradiation of the pattern light by the third light projecting section 33 are the third pattern image set.

Similarly, when the fourth light projecting unit 34 of the illumination device 3 irradiates the object W to be measured with pattern light according to the spatial code method, a gray code pattern image set is generated and a pattern according to the phase shift method is generated. When light is applied to the measurement object W, a phase shift pattern image set is generated. Both the gray code pattern image set and the phase shift pattern image set obtained by irradiation of the pattern light by the fourth light projecting section 34 are the fourth pattern image set.

Each pattern image set can be stored in the image storage unit 28 shown in FIG.

As shown in FIG. 3, the data processing section 24 has an HDR processing section 24a. HDR processing is high dynamic range imaging synthesis processing, and the HDR processing unit 24a synthesizes a plurality of images with different exposure times. That is, as described above, when the measurement object W irradiated with pattern light of a certain pattern is imaged a plurality of times with different exposure times, a plurality of luminance images with different exposure times are obtained. By synthesizing these luminance images, an image having a dynamic range wider than that of each luminance image can be generated. A conventionally well-known method can be used for the HDR synthesis method. Instead of changing the exposure time, a plurality of luminance images with different brightness may be obtained by changing the intensity of the radiated light, and these luminance images may be synthesized.

(Configuration of Phase Calculation Unit 26)
The phase calculator 26 shown in FIG. 3 is a part that calculates an absolute phase image that is the original data of the height image. As shown in FIG. 18, in step SA1, each image data of the phase shift pattern image set is obtained, and relative phase calculation processing is performed by using the phase shift method. This can be expressed as a relative phase (pre-unwrapping phase), and a phase image is obtained by the relative phase calculation processing in step SA1.

On the other hand, at step SA3 in FIG. 18, each image data of the gray code pattern image set is acquired, and the spatial code calculation process is performed by using the spatial code method to obtain the stripe number image. The fringe number image is an image that is made identifiable by assigning a series of space code numbers to the small spaces when the space irradiated with light is divided into a large number of small spaces.

At step SA4 in FIG. 18, absolute phase phasing processing is performed. In the absolute phase phasing process, the phase image obtained in step SA1 and the fringe number image obtained in step SA3 are synthesized (unwrapped) to generate an absolute phase image (intermediate image). In other words, the spatial code number obtained by the spatial code method can be used to correct the phase jump (phase unwrap) by the phase shift method, so it is possible to obtain high-resolution measurement results while ensuring a wide dynamic range of height. .

Height measurement may be performed only by the phase shift method. In this case, the height measurement dynamic range is narrowed, so that the height cannot be measured correctly in the case of an object W having a large difference in height such that the phase shifts by one period or more. Conversely, in the case of the object W to be measured that has a small change in height, the striped image is not picked up or synthesized by the spatial code method, so there is the advantage of speeding up the processing accordingly. For example, when measuring an object W that has little difference in the height direction, it is not necessary to take a large dynamic range. can be shortened. Also, since the absolute height is known, the height may be measured only by the spatial code method. In this case, the accuracy can be improved by increasing the code.

Further, at step SA2 in FIG. 18, each image data of the phase shift pattern image set is obtained, and reliability image calculation processing is performed. In the reliability image calculation process, a reliability image indicating phase reliability is calculated. This is an image that can be used to determine invalid pixels.

The phase image, fringe number image and reliability image can be stored in the image storage unit 28 shown in FIG.

The absolute phase image generated by the phase calculator 26 can also be said to be an angle image in which each pixel has irradiation angle information of the pattern light for measurement onto the object W to be measured. That is, since the first pattern image set (phase shift pattern image set) includes eight first pattern images obtained by shifting the phase of the sinusoidal fringe pattern, the phase shift method can be used. Thus, each pixel has irradiation angle information of the pattern light for measurement to the object W to be measured. In other words, the phase calculator 26 is a part that generates a first angle image in which each pixel has irradiation angle information of the first measurement pattern light onto the measurement object W based on a plurality of first pattern images. , can also be called an angle image generator. The first angle image is an image of the angle of the light emitted from the first LED 31b to the object W to be measured.

Similarly, the phase calculator 26 calculates a second angle image in which each pixel has irradiation angle information of the second measurement pattern light on the measurement object W based on the plurality of second pattern images, and a plurality of third pattern images. A third angle image in which each pixel has irradiation angle information of the third measurement pattern light to the measurement object W based on the image, A fourth angle image having irradiation angle information of the four measurement pattern lights can be generated. The second angle image is an image of the angle of the light emitted from the second LED 32b to the object W to be measured. The third angle image is an image obtained by imaging the angle of the light emitted from the third LED 33b to the object W to be measured. The fourth angle image is an image of the angle of the light emitted from the fourth LED 34b to the object W to be measured. The uppermost image of the intermediate images in FIG. 17 is the first angle image, the second image from the top is the second angle image, the third image from the top is the third angle image, and the bottom image is the third angle image. The image is the fourth angle image. The parts of each angle image that appear to be blacked out are the parts that are shaded by the lighting (each of the LEDs), and are invalid pixels without angle information.

(Configuration of image processing unit 27)
The image processing unit 27 is a part that performs image processing such as gamma correction, white balance adjustment, and gain correction on each of the pattern images, phase images, stripe number images, and reliability images. Each pattern image, phase image, fringe number image, and reliability image after image processing can be stored in the image storage unit 28 . Image processing is not limited to the processing described above.

(Configuration of output control unit 29)
Upon receiving the image output request signal output from the controller unit 4, the output control unit 29 selects the image instructed by the image output request signal among the images stored in the image storage unit 28 according to the image output request signal. is a part that outputs to the controller unit 4 via the image processing unit 27 . In this example, each pattern image, phase image, fringe number image and reliability image before image processing are stored in the image storage unit 28, and an image requested by an image output request signal from the controller unit 4 is processed. Only the image processing section 27 performs image processing and outputs the data to the controller section 4 . The image output request signal can be output when the user performs various measurement operations or inspection operations.

In this embodiment, the data processing section 24 , the phase calculation section 26 and the image processing section 27 are provided in the imaging device 2 , but they may also be provided in the controller section 4 . In this case, the image data output from the imaging device 22 is output to the controller section 4 and processed.

(Configuration of controller section 4)
As shown in FIG. 2 , the controller section 4 includes an imaging projection control section 41 , a height measurement section 42 , an image composition section 43 , an inspection section 45 and a display control section 46 . The controller unit 4 is provided separately from the imaging device 2 and the lighting device 3 .

(Structure of imaging projection control section 41)
The imaging light projection control unit 41 outputs the pattern light formation information, the trigger signal, and the resynchronization trigger signal to the lighting device 3 at a predetermined timing, and outputs the information regarding the imaging condition, the trigger signal, and the resynchronization trigger signal. , to the imaging device 2 at a predetermined timing. The trigger signal and resynchronization trigger signal output to the illumination device 3 are synchronized with the trigger signal and resynchronization trigger signal output to the imaging device 2 . The formation information of the pattern light and the information on the imaging conditions can be stored, for example, in the imaging projection control section 41 or another storage section (not shown). When the user performs a predetermined operation (height measurement preparation operation, inspection preparation operation), the pattern light formation information is output to the illumination device 3 and temporarily stored in the light projection control unit 39 of the illumination device 3, Further, the information regarding the imaging conditions is output to the imaging device 2 and temporarily stored in the exposure control section 23 . In this example, the lighting device 3 can be called a smart type lighting device because the lighting device 3 is configured to control the LED and the LCD by the light projection control unit 39 built in the lighting device 3. In addition, since the imaging device 2 is configured to control the imaging element 22 by the exposure control unit 23 built in the imaging device 2, it can be called a smart type imaging device.

In this way, when the imaging device 2 and the lighting device 3 perform separate control, as the number of times of imaging increases, the imaging timing and the lighting timing (projection timing of the pattern light) deviate. There is a problem that the obtained image becomes dark. In particular, as described above, a total of 12 images, 8 images of the phase shift pattern image set and 4 of the gray code pattern image set, constitute the first pattern image set, and similarly constitute the second pattern image set. Furthermore, if HDR imaging is also performed, the number of times of imaging increases, and the difference between the imaging timing and the illumination timing becomes noticeable.

In this example, the resynchronization trigger signal is synchronously output to the illumination device 3 and the imaging device 2, so that the illumination device 3 and the imaging device 2 can be synchronized during imaging. I have to. Therefore, even if the number of imaging times increases, the difference between the imaging timing and the illumination timing is extremely small to the extent that it does not become a problem, and it is possible to suppress the darkening of the image during the irradiation of the phase shift pattern or the gray code pattern. , the possibility of erroneous phase distortion and code determination can be reduced. The resynchronization trigger signal can also be output multiple times.

The imaging light projection control section 41 includes an irradiation mode switching section 41a. A first irradiation mode in which the first pattern light for measurement and the second pattern light for measurement are emitted by the first light projection unit 31 and the second light projection unit 32, respectively, and the first light projection unit 31 and the second light projection unit 32 After irradiating the first pattern light for measurement and the second pattern light for measurement, respectively, the third light projection unit 33 and the fourth light projection unit 34 irradiate the third light pattern for measurement and the fourth pattern light for measurement, respectively. and a third irradiation mode in which the third pattern light for measurement and the fourth pattern light for measurement are emitted by the third light projecting section 33 and the fourth light projecting section 34, respectively. , can be switched to any one illumination mode. The user can switch the irradiation mode by operating the console section 6 or the mouse 7 while viewing the display section 5 . Alternatively, the controller unit 4 can be configured to automatically switch the irradiation mode.

(Configuration of height measuring unit 42)
The height measurement unit 42 measures the irradiation angle information of each pixel of the first angle image and the irradiation angle information of each pixel of the second angle image generated by the phase calculation unit 26, and the first height in the illumination housing 30 of the illumination device 3. The height of the measuring object W in the central axis A direction of the lighting device 3 can be measured according to the relative position information of the light projecting section 31 and the second light projecting section 32 .

A specific method for measuring the height by the height measuring unit 42 will be described below. As described above, the angle from the illumination for each pixel is determined by generating the angle image by unwrapping the phase. The first angle image is an image showing the angle of light emitted from the first LED 31b to the object W to be measured, and the second angle image is an image showing the angle of light emitted from the second LED 32b to the object W to be measured. be. The first LED 31b and the second LED 32b are integrally supported by the illumination housing 30, and the distance between the first LED 31b and the second LED 32b is l (shown in FIG. 7) as described above.

FIG. 7 illustrates the case of obtaining the height of an arbitrary point H on the object W to be measured. The direction directly below the first LED 31b is 0°, and the direction 45° from the first LED 31b is 1. Moreover, the right direction in FIG. 7 is positive, and the left direction is negative. The angle of light emitted from the first LED 31b to the point H can be obtained from the pixel corresponding to the point H in the first angle image, and the inclination of the straight line connecting the point H and the first LED 31b is 1/a1. Further, the angle of the light irradiated from the second LED 32b to the point H can be obtained from the pixel corresponding to the point H in the second angle image, and the inclination of the straight line connecting the point H and the second LED 32b is -1/a2. do. a1 and a2 are phases.

Z=1/a1*X+0... Formula 1
Z=−1/a2*(X−l) … Formula 2

Solving Z for Equations 1 and 2 gives the height.
a1Z=X
a2Z=-X+l
Z=l/(a1+a2)
X=a1*l/(a1+a2)

In this manner, the height of each point on the object W to be measured can be obtained. Since each of the above equations has no variable relating to the position of the imaging device 2, it can be seen that the position of the imaging device 2 is irrelevant when obtaining the height of each point on the object W to be measured. However, since there is no angle information for pixels that are invalid pixels in the angle image, the height of that point cannot be obtained. That is, the calculated Z-coordinate indicates not the distance between the imaging device 2 and the object W to be measured, but the distance to the object W when viewed from the illumination device 3 . The Z coordinate obtained by the installation position of the illumination device 3 is determined regardless of the installation position of the imaging device 2 .

Also, although not shown, the angle of the light emitted from the third LED 33b to the point H can be similarly obtained from the pixel corresponding to the point H in the third angle image. Since the angle of the projected light can be obtained from the pixel corresponding to the point H in the fourth angle image, the height of each pixel can be obtained based on the third angle image and the fourth angle image.

For example, in FIG. 17, the height measurement unit 42 obtains irradiation angle information of each pixel of the first angle image, irradiation angle information of each pixel of the second angle image, and the first light projection unit 31 and A first height image representing the height of the measurement object W is generated according to the relative position information of the second light projecting unit 32, and the irradiation angle information of each pixel of the third angle image and each of the fourth angle images are generated. When the second height image representing the height of the measurement target W is generated according to the irradiation angle information of the pixels and the relative position information of the third light projecting section 33 and the fourth light projecting section 34 in the illumination housing 30 is shown.

Since the first height image can grasp the height of each pixel, it can be used as an image to be inspected when performing various inspections. Also, since the second height image can also grasp the height of each pixel, it can be used as an inspection object image used when performing various inspections. Therefore, the height measuring unit 42 can also be called an inspection target image generation unit that generates an inspection target image based on a plurality of intermediate images.

In the case shown in FIG. 17 , first, a first angle image is generated from a first pattern image set obtained by projecting pattern light by the first light projecting unit 31 , and the pattern light is projected by the second light projecting unit 32 to generate a first angle image. A first angle image is generated from the obtained second pattern image set. In the first angle image, since the first light projecting unit 31 irradiates light from the left side of the measurement object W, a shadow is formed on the right side of the measurement object W, and that portion is an invalid pixel. On the other hand, in the second angle image, since the second light projecting unit 32 irradiates light from the right side of the measurement object W, a shadow is formed on the left side of the measurement object W, and that portion becomes an invalid pixel. there is Since the first height image is generated from the first angle image and the second angle image, pixels that are invalid pixels in one of the angle images are also invalid pixels in the first height image.

Similarly, a third angle image is generated from a third pattern image set obtained by projecting the pattern light by the third light projecting unit 33, and a fourth angle image is generated by projecting the pattern light by the fourth light projecting unit 34. Generate a fourth angle image with the pattern image set. In the third angle image, the third light projecting unit 33 irradiates light from the upper side of the measurement object W (upper side in the figure), so that the lower side of the measurement object W (lower side in the figure) side) is cast, and that portion is an invalid pixel. On the other hand, in the fourth angle image, since the fourth light projecting unit 34 irradiates light from the lower side of the measurement object W, a shadow is formed above the measurement object W, and the shadowed portion becomes an invalid pixel. ing. Since the second height image is generated from the third angle image and the fourth angle image, pixels that are invalid pixels in one of the angle images are also invalid pixels in the second height image. In order to reduce invalid pixels as much as possible, in this embodiment, an image synthesizing section 43 is provided in the controller section 4 as shown in FIG.

In this embodiment, the case where the height measuring section 42 is provided in the controller section 4 has been described, but the present invention is not limited to this, and the height measuring section may be provided in the imaging device 2 although not shown.

(Configuration of Image Synthesizer 43)
The image synthesizing unit 43 is configured to synthesize the first height image and the second height image to generate a post-synthesis height image. As a result, the portions of the first height image that are invalid pixels and the portions that are not invalid pixels of the second height image are represented by valid pixels in the combined height image. Conversely, portions that are invalid pixels in the second height image but not invalid pixels in the first height image are represented by valid pixels in the height image after synthesis. Therefore, the number of invalid pixels can be reduced in the combined height image. Conversely, if you want to obtain a height with high reliability, only when both the first height image and the second height image are valid and the difference between them is smaller than a predetermined value, the average height may be enabled.

In other words, by irradiating the measurement object W with pattern light from four different directions, it is possible to increase the number of effective pixels in the height image, reduce blind spots, and increase the reliability of the measurement results. can improve sexuality. In the case of the measurement object W in which the number of invalid pixels is sufficiently reduced by irradiating the pattern light from two directions, only one height image should be generated. In this case, it is also possible for the user to select whether to generate the first height image or the second height image. Generating only one height image has the advantage of shortening the measurement time.

Since the height image after synthesis can also grasp the height of each pixel, it can be used as an image to be inspected when performing various inspections. Therefore, the image synthesizing unit 43 can also be called an inspection target image generation unit that generates an inspection target image.

In this embodiment, the case where the image synthesizing unit 43 is provided in the controller unit 4 has been described.

(Configuration of Inspection Unit 45)
The inspection unit 45 is a part that performs inspection processing based on an arbitrary image among the first height image, the second height image, and the combined height image. The inspection unit 45 includes a presence/absence inspection unit 45a, an appearance inspection unit 45b, and a dimension inspection unit 45c, but this is an example, and all these inspection units are not essential. You may provide an inspection part other than. The presence/absence inspection unit 45a is configured to determine the presence/absence of the object W to be measured, the presence/absence of components attached to the object W to be measured, and the like by image processing. The appearance inspection unit 45b is configured to be able to determine by image processing whether or not the external shape of the object W to be measured has a predetermined shape. The dimension inspection unit 45c is configured to determine whether or not the dimensions of each part of the measurement object W are predetermined dimensions, or to determine the dimensions of each part by image processing. Since these determination methods are conventionally well-known methods, detailed description thereof will be omitted.

(Configuration of display control unit 46)
The display control unit 46 displays the first height image, the second height image, the combined height image, etc. on the display unit 5, and controls the operation user interface for operating the image processing device 1, the image processing device, and the like. A setting user interface for setting 1, a height measurement result display user interface for displaying the height measurement result of the measurement object, and an inspection result display user for displaying various inspection results of the measurement object It is configured such that an interface or the like can be generated and displayed on the display unit 5 .

(Example of user interface)
FIG. 19 is a diagram showing an example of a three-dimensional discrimination user interface 300 that implements the three-dimensional discrimination function of the image processing apparatus 1. As shown in FIG. The 3D discrimination function is a function that compares the 3D shape of the measurement object registered in advance by the user with the 3D shape of the measurement object obtained by measuring as described above, and extracts the difference. is. The user can acquire the three-dimensional shape of the measurement object by taking an image of the measurement object, store it in the controller unit 4, and register it. The user can set the tolerance (permissible range) for detection in each of the XYZ directions based on the registered three-dimensional shape. In addition, the three-dimensional search function of the image processing apparatus 1 can three-dimensionally correct the three-dimensional shape of the object to be measured based on the registered three-dimensional shape. After correcting the position of the 3D shape of the measured object obtained by the measurement using the 3D search function, it is superimposed on the 3D shape of the registered measured object, the same points are compared, and the difference is calculated based on the tolerance. . The three-dimensional discrimination user interface 300 can cause the display unit 5 to display the location where the difference exceeding the tolerance is calculated.

The three-dimensional discrimination user interface 300 shown in FIG. 19 switches and alternately displays the registered three-dimensional shape 301A of the measurement object and the three-dimensional shape 301B of the measurement object obtained by measurement at regular time intervals. is configured to allow The three-dimensional shape 301A of the registered measurement object is shown in the three-dimensional discrimination user interface 300 on the upper side of FIG. A three-dimensional shape 301B of the object to be measured obtained by measurement is shown in the three-dimensional discrimination user interface 300 on the lower side of FIG. That is, it is an image in which the three-dimensional discrimination user interface 300 on the upper side of FIG. 19 and the three-dimensional discrimination user interface 300 on the lower side are alternately displayed on the display unit 5 . The three-dimensional search function allows the registered three-dimensional shape 301A of the object to be measured and the three-dimensional shape 301B of the object to be measured to be displayed at the same position on the three-dimensional discrimination user interface 300. can easily grasp the difference portion just by continuing to look at the three-dimensional discrimination user interface 300 . Incidentally, the certain period of time is not particularly limited, but can be set to, for example, about one second. In addition, it is also possible to colorize and display the difference portion. Reference numeral 302 indicates the colored portion.

FIG. 20 shows a user interface 310 displayed when a cross-section is specified to display the allowable range of the 3D discrimination function. This user interface 310 is provided with a shape display area 310a in which a three-dimensional shape 301B of the object to be measured is displayed. In the shape display area 310a, it is possible to display cross-section position setting portions 310b and 310b for arbitrarily setting the position for acquiring the cross section of the three-dimensional shape 301B. The cross-section position setting portions 310b, 310b can be moved on the shape display area 310a by mouse operation or the like.

After the positions of the cross-section position setting portions 310b and 310b are determined, a cross-section between the cross-section position setting portions 310b and 310b is obtained, and a cross-section of the three-dimensional shape 301B is displayed in the shape display area 310a as shown in FIG. A shape line 312 indicating the outer shape of the three-dimensional shape 301B and an allowable range display portion 313 indicating the allowable range of the outer shape of the three-dimensional shape 301B are displayed in the shape display area 310a. The allowable range display section 313 can have, for example, a display form in which the allowable range is colored in a strip. If the shape line 312 falls within the range displayed by the allowable range display section 313, it can be determined that the inspection object has no defects. If it does not fit, it can be determined that the inspection object has a defect or the like.

The allowable range can be changed by manipulation on user interface 310 . FIG. 22 shows a case in which the allowable range is reduced, and the allowable range display section 313 is narrower than in FIG. The tolerance can also be expanded. If the allowable range is narrowed, part of the three-dimensional shape 301B will be out of the allowable range display portion 313, and this portion will be displayed as a colored portion 313 in the shape display area 310a. In this way, since the allowable range can be adjusted while comparing the three-dimensional shape 301B and the allowable range, a more appropriate allowable range can be set, and as a result, the inspection accuracy and stability are improved. do.

FIG. 23 shows a case where the three-dimensional shape 301B of the measurement object obtained by new measurement is input after setting the allowable range as described above. A portion of the three-dimensional shape 301B outside the allowable range is displayed as a colored portion 313. FIG.

As described above, the three-dimensional shape registered in advance and the three-dimensional shape of the object to be measured newly obtained by measurement are compared, and if the difference is within a predetermined allowable range, the object to be measured is judged to be a non-defective product. Judge as. Although the allowable range in the XY direction (horizontal direction) and the allowable range in the Z direction (vertical direction) can be set, the width of each allowable range may be the same or different. The allowable range in the Z direction can be set by adding and subtracting heights to and from the registered three-dimensional shape, and the allowable range in the XY directions can be set by dilation/contraction filters for the registered three-dimensional shape. The upper limit of the allowable range is calculated by adding the expansion processing for the XY allowable range and the Z allowable range to the registered three-dimensional shape. The lower limit of the allowable range is calculated from the registered three-dimensional shape by contraction processing for the XY allowable range and subtraction of the Z allowable range.

FIG. 24 is an image diagram of allowable range setting when detecting chipping of a projecting measurement object, and shows a case where the projection is relatively large. Chipping can be detected if there is a height below the lower dotted line (lower limit of allowable range) of the registered three-dimensional shape, but the size at which chipping can be detected is smaller than the actual size of the projection. FIG. 25 is an image diagram for detecting chipping of a smaller protrusion. If the same allowable range in the XY directions as in FIG. 24 is set, even if there is a chipping in the protrusion, detection becomes impossible.

Therefore, in this embodiment, both the registered three-dimensional shape and the input three-dimensional shape are subjected to dilation processing with the same size as the XY allowable range. As shown in FIG. 26, by applying a dilation filter having the same size as the XY tolerance set by the user to both the registered three-dimensional shape and the input three-dimensional shape, the apparent size of the projection can be increased. Since the lower limit of the allowable range set for this enlarged protrusion is equal to the actual size of the protrusion, the size at which chipping of the protrusion can be detected is equal to the actual size of the protrusion.

When the projection emphasis process is turned on, the dilation process is automatically applied in the same size as the size of the allowable range in the XY directions set by the user. When the user sets the allowable range in terms of actual size values (millimeters, inches, etc.), the actual size values are automatically converted to the number of pixels using a conversion coefficient that represents the actual size value per pixel, and the converted number of pixels is Expansion/contraction processing can be applied to increase strength. As a result, the expansion/contraction processing with the optimum strength can be automatically applied simply by setting the tolerance range that the user wants to allow.

In the present embodiment, an example of performing dilation processing with the same size as the XY tolerance has been described, but the dilation processing need not be exactly the same as the XY tolerance. Almost the same effect can be obtained as long as the size is the same. Also, by performing contraction processing instead of expansion processing, it is possible to inspect whether or not recesses are filled instead of missing protrusions.

FIG. 27 is a data flow diagram of this embodiment. With respect to the three-dimensional shape input by new measurement, the three-dimensional posture is estimated using the three-dimensional search function described above using the pre-registered three-dimensional shape, and correction is performed. Next, the same dilation process is applied to both the registered 3D shape and the input 3D shape based on the set values of the allowable range in the XY directions (horizontal direction) set by the user to generate difference data. . Whether the difference data is within a predetermined range or not determines whether the object to be measured is good or bad.

FIG. 28 shows another data flow diagram. The difference from FIG. 27 is that the upper limit and lower limit of the allowable range are calculated from the registered three-dimensional shape to which the expansion process is applied, based on the set values of the allowable range in the XY directions. Applying expansion processing in the XY directions to the registered three-dimensional shape, and applying the allowable range in the XY directions to this expanded three-dimensional shape, the width in the XY directions of the measurement object at the lower limit of the allowable range is It is substantially equal to the width of the actual object to be measured in the XY directions. As a result, it is possible to detect the lack of projections (in the case of expansion processing) or the filling of holes (in the case of contraction processing) that are substantially equal in size to the widths in the XY directions of the actual object to be measured.

Generally, in three-dimensional measurement, measurement results cannot be obtained for positions where the pattern is not irradiated or positions that are blind spots from the camera. Pixels for which no measurement result was obtained are called "invalid pixels." It is difficult to completely eliminate these invalid pixels, especially in the pattern projection method using the principle of triangulation.

When performing inspection using the difference between the registered 3D shape and the input 3D shape obtained by measuring the object to be measured, the invalid pixel portions existing in the input 3D shape will be different from the registered 3D shape as they are. Unable to define difference. In addition, the difference when there are invalid pixels not only on the input image side but also on the registered image side cannot be defined as it is. Therefore, conventionally, filter processing is performed to fill ineffective pixels with an average value of surrounding heights. However, since the process of filling ineffective pixels by filtering determines the height based on the heights of pixels at different positions, the height is not stable compared to effective pixels. Also, the height cannot be determined if there are no effective pixels nearby.

Therefore, in this embodiment, the user designates a plane that serves as a background (hereinafter referred to as a background plane), and invalid pixels are set to the height of the background plane. It allows us to define the difference. FIG. 29 is an image diagram of processing of the background plane. A position where the cross-sectional profile of the object to be measured is interrupted indicates an invalid pixel for which no measurement result was obtained. The height of the invalid pixels is set to the height of the background plane, and pixels with heights lower than the background plane are clipped at the height of the background plane. As a result, even if there are invalid pixels, it is possible to calculate the difference between the registered three-dimensional shape and the input three-dimensional shape, and calculate the measured value such as the volume of the difference. The difference volume is calculated by integrating the height difference in the XY direction. By giving , we can compute these measurements. In addition, compared to the case where invalid pixels are filled by filtering such as the above-described averaging filter, it is possible to define the height even if there are no valid pixels nearby, and the influence of the surrounding height is reduced. Since it is not received, the determination result is stable.

In addition to the background plane, an upper limit plane (hereinafter referred to as upper limit plane) can also be set. FIG. 30 is a diagram showing a processing image when the upper limit plane is set in addition to the background plane. Invalid pixels are set to the height of the background section, and the height of the portion exceeding the upper limit plane is set to the height of the upper limit plane.

FIG. 31 is a data flow diagram for background plane processing. After correcting the three-dimensional posture of the input three-dimensional shape based on the three-dimensional search function described above, background plane processing is applied. The same background plane processing is applied to the registered 3D shape. The registered 3D shape after the background plane processing is applied is compared with the input 3D shape, and difference data is extracted.

FIG. 32 shows a data flow when background plane processing is applied to each of the upper limit and lower limit of the allowable range set by the user for the registered three-dimensional shape. As described above, the upper limit and lower limit of the allowable range are set for the registered 3D shape, so if invalid pixels exist in the registered 3D shape, invalid pixels will also occur in the upper and lower limits of the allowable range. Resulting in. Therefore, by applying the same background plane processing as for the input 3D shape to the upper and lower limits of the allowable range, the invalid pixels at the upper and lower limits of the allowable range can be replaced with the height of the background plane. can. Invalid pixels at the upper limit of the allowable range and invalid pixels at the lower limit of the allowable range are unified to the same background plane height set by the user. Therefore, if the corresponding pixel of the input 3D shape is not an invalid pixel but a valid pixel and a height greater than the height of the background plane is obtained, it is detected as a protrusion exceeding the allowable range, and a height smaller than the height of the background plane is detected. is obtained, it is detected as chipping below the allowable range. In this way, in an application that detects protrusions and missing parts of an input 3D shape using the tolerance (tolerance) set by the user for the registered 3D shape, the upper and lower limits of the registered 3D shape By applying the background plane processing to , it is possible to perform stable inspection even when there are invalid pixels in the registered three-dimensional shape.

As described above, the use of background plane processing makes it extremely easy to handle invalid pixels. On the user interface, when the user limits the measurement area in the height direction, the height value given to the invalid pixels is automatically limited to the lower limit of the measurement area. Therefore, the user does not need to be conscious of how to handle invalid pixels.

Note that the projection enhancement processing and the background plane processing are executed by the inspection section 45 provided in the controller section according to the user's operation.

FIG. 33 shows a user interface 320 that implements the height binarization function that the image processing apparatus 1 has. The height binarization function is a function of generating a binarized image depending on whether or not a certain height of a three-dimensional shape is crossed. By measuring the binarized area using the height binarization function, it is possible to determine, for example, the presence or absence of a part.

The user interface 320 for the height binarization function includes a three-dimensional shape display area 321 in which the three-dimensional shape 301B is obliquely displayed, and a binarized height image display in which a binarized height image is displayed. A region 322 is provided. In the three-dimensional shape display area 321, a horizontal plane 321a is superimposed on the three-dimensional shape 301B. The horizontal plane 321 a can be arbitrarily moved in the height direction by operating means such as a slider bar 321 b provided on the side of the three-dimensional shape display area 321 . In the binarized height image display area 322, the portion of the three-dimensional shape 301B located within the horizontal plane 321a and the other portions are displayed in different colors. Reference numeral 323 denotes a portion located within the horizontal plane 321a. FIG. 34 shows a state in which there is no portion located within the horizontal plane 321a.

The height binarization function is equipped with a zoom function. FIG. 35 shows an example in which the three-dimensional shape 301B displayed in the three-dimensional shape display area 321 is enlarged and displayed. A conventionally used method can be used for the enlargement method. By enlarging the three-dimensional shape 301B, the moving width of the horizontal plane 321a when the slider bar 321b is operated is narrowed, and finer adjustment becomes possible.

FIG. 36 shows a user interface 330 that implements a shape comparison function in height binarized images. This function is effective in inspecting differences in shape that cannot be determined only by measuring the binarized area using the height binarized image. A user interface 330 that implements the shape comparison function includes a three-dimensional shape display area 331 that obliquely displays the three-dimensional shape 301B, and a binarized height image display area 332 that displays a binarized height image. and are provided. In the three-dimensional shape display area 331, a horizontal plane 331a similar to the user interface 320 for the height binarization function is superimposed on the three-dimensional shape 301B. The horizontal plane 331 a can be arbitrarily moved in the height direction by operating means such as a slider bar 331 b provided on the side of the three-dimensional shape display area 331 . In the binarized height image display area 332, a portion 333 of the three-dimensional shape 301B located within the horizontal plane 331a is displayed.

FIG. 37 is a user interface 340 for selecting contours to inspect. The user interface 340 is provided with a contour display area 341 in which contours are displayed. A part of the contour displayed in the contour display area 341 can also be deleted.

FIG. 38 shows a state in which a result display area 335 for displaying contour detection results is displayed in the user interface 330 shown in FIG. The detected contour 336 is also displayed in the result display area 335 along with the allowable range 335a.

FIG. 39 shows a case where the allowable range is expanded. The allowable range can be expanded or contracted.

The above is performed in the setting mode of the image processing apparatus 1 . FIG. 40 shows a display example when a three-dimensional shape of a different type is input in the operation mode of the image processing apparatus 1. FIG. If a three-dimensional shape of a different type is input, a portion exceeding the allowable range is generated, so that it can be detected that a three-dimensional shape of a different type is input.

(Action and effect of the embodiment)
As described above, according to the image processing apparatus 1 according to this embodiment, the measurement target W is irradiated with the pattern light whose phase is sequentially shifted, and each time the measurement target W is irradiated with the pattern light, the measurement Light reflected from the object W can be received to generate a plurality of pattern images. The three-dimensional shape of the measurement object W can be measured based on the plurality of pattern images generated.

When irradiating the pattern light, the patterns having phases different from each other by 180 degrees are continuously irradiated. The phase with the pattern light to be applied has an opposite phase relationship. As a result, it is possible to minimize the variation in the brightness of the disturbance light by narrowing the irradiation intervals of the pattern lights of opposite phases, so that the brightness of the disturbance light is reduced until the measurement of one measurement object W is completed. Even if there is a change in , the influence of the change can be suppressed, and as a result, the measurement error can be reduced.

The above-described embodiments are merely examples in all respects and should not be construed in a restrictive manner. Furthermore, all modifications and changes within the equivalent scope of claims are within the scope of the present invention.

INDUSTRIAL APPLICABILITY As described above, the image processing apparatus according to the present invention can be used, for example, when measuring the height of an object to be measured or when inspecting an object to be measured.

1 image processing device 2 imaging device 3 lighting device 22 imaging element 26 phase calculation unit 30 lighting housing 31 first light projection unit 31b first LED (first light source)
31d 1st LCD (first pattern light generator)
31e first LED drive circuit (light source drive circuit)
31f First LCD drive circuit (liquid crystal panel drive circuit)
32 Second light projecting portion 32b Second LED (second light source)
32d Second LCD (second pattern light generator)
32e Second LED drive circuit (light source drive circuit)
32f Second LCD drive circuit (liquid crystal panel drive circuit)
33 Third light projecting portion 33b Third LED (third light source)
33d Third LCD (third pattern light generator)
33e Third LED drive circuit (light source drive circuit)
33f Third LCD drive circuit (liquid crystal panel drive circuit)
34 fourth light projecting portion 34b fourth LED (fourth light source)
34d 4th LCD (4th pattern light generator)
34e fourth LED drive circuit (light source drive circuit)
34f fourth LCD drive circuit (liquid crystal panel drive circuit)
35 light projection control unit 42 height measurement unit 43 image synthesizing unit 45 inspection unit A opening

Claims (6)

In an image processing device that measures the three-dimensional shape of an object to be measured,
a light source;
a pattern generation unit that receives the light emitted from the light source and irradiates the object to be measured with periodic pattern light;
an illumination control unit that controls the pattern generation unit so as to sequentially shift the phase of the pattern, and continuously generates patterns with phases different from each other by 180 degrees when controlling the pattern generation unit;
an imaging unit that receives the light reflected from the object to be measured and generates a plurality of pattern images each time the pattern light is irradiated;
and a measuring unit that measures the three-dimensional shape of the object based on the plurality of pattern images. The image processing device according to claim 1,
The light source includes a first light source and a second light source provided point-symmetrically with respect to each other about the center of the opening in an illumination housing having an opening formed in the center,
The pattern generator includes a first pattern generator provided corresponding to the first light source and a second pattern generator provided corresponding to the second light source,
The illumination control unit controls pattern light from the pattern generation unit that has completed pattern generation while an arbitrary pattern generation unit among the first pattern generation unit and the second pattern generation unit is switching patterns. An image processing apparatus, characterized in that it is configured to irradiate. In the image processing device according to claim 2,
The light sources are spaced from the first light source and the second light source in the circumferential direction of the opening in the lighting housing and provided point-symmetrically with respect to the center of the opening. including 4 light sources,
The pattern generator includes a third pattern generator provided corresponding to the third light source and a fourth pattern generator provided corresponding to the fourth light source,
While the illumination control unit is switching the patterns of any one of the first pattern generation unit, the second pattern generation unit, the third pattern generation unit, and the fourth pattern generation unit, 2. An image processing apparatus characterized in that the pattern light is emitted from a pattern generation unit that has completed pattern generation. In the image processing device according to any one of claims 1 to 3,
The image processing apparatus, wherein the pattern generation unit is a liquid crystal panel. In the image processing device according to any one of claims 1 to 4,
The image processing apparatus, wherein the imaging unit is composed of a color camera. In the image processing device according to any one of claims 1 to 5,
The illumination control unit causes the pattern generation unit to generate a phase shift pattern light for generating a phase shift image and a spatial code pattern light for generating a space code image, and the phase shift pattern light is generated, patterns whose phases are different from each other by 180 degrees are continuously generated,
The imaging unit receives light reflected from a measurement object when the phase-shifting pattern light is irradiated, generates a phase-shifted image, and receives the measurement object when the spatial-code pattern light is irradiated. configured to receive light reflected from and generate a spatially coded image;
The measurement unit is configured to measure the three-dimensional shape of the measurement target by applying a phase shift method and a space encoding method based on the phase shift image and the space code image. image processing device.

Images