JP2017003469A - Three-dimensional measurement device, three-dimensional measurement device control method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To highly accurately measure a three-dimensional shape of a work with high speed.SOLUTION: A three-dimensional measurement device measuring a three-dimensional shape of works on the basis of an image obtained by imaging the work having patterns projected comprises: a synthesis unit that synthesizes first and second images of the work acquired respectively under first and second shooting conditions so that a magnitude relation between an amount of exposure per unit time by a light source for projecting the pattern and a luminance value corresponding to the amount of exposure is maintained between the first and second images; and a measurement unit that measures the three-dimensional shape of the work on the basis of the image synthesized by the synthesis unit.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a three-dimensional measuring apparatus, a control method for the three-dimensional measuring apparatus, and a program.

  A spatially encoded pattern projection method is known as a method for measuring the shape of a test object from image information using the principle of triangulation. A plurality of grid patterns are projected onto a test object by a projection device such as a projector, the plurality of grid patterns are imaged by an imaging device, and the test is performed based on the light / dark position of the grid pattern and the geometric relationship between the imaging device and the projection device. Measure the shape of an object. In the spatially encoded pattern projection method, the code associated with the bright and dark positions of the projection pattern is analyzed as the height from the reference plane at each point of the test object, and the three-dimensional shape of the test object is measured. . However, in the three-dimensional shape measurement using such a spatially encoded pattern projection method, there is a problem when a region having various reflection characteristics such as a low reflection region and a high reflection region is widely distributed on the surface of the test object. May occur. For example, even if the light quantity of the light source for projecting the pattern is adjusted or the exposure of the imaging device is adjusted, a wide range of reflection distribution may not be covered.

  On the other hand, Patent Document 1 discloses a method using an image capturing unit that captures a light cutting line generated by a light cutting method using slit light to obtain grayscale grayscale image data. Create a composite image by replacing the brightness of saturated pixels with the brightest brightness value below saturation brightness from a line light image captured under multiple imaging conditions, and measure the 3D shape of the test object from that image By doing so, it is possible to measure the shape of an object having a wide range of reflection characteristics.

  Also, in Patent Document 2, spatial codes are associated from pattern intensity images captured under a plurality of imaging conditions using the spatial encoding pattern projection method, and the spatial codes under the highest average luminance of the images are integrated. By doing so, the shape of the test object is measured.

JP-A-62-21011 JP 2007-271530 A

  However, in the case of the technique described in Patent Document 1, the luminance of the image is simply replaced with the highest luminance value that is not saturated. Therefore, in the spatially encoded pattern projection method, there is a problem in that it is difficult to perform high-accuracy three-dimensional shape measurement because there is a case where association is erroneously performed in association with codes according to the light / dark positions of patterns. .

  Further, in the technique described in Patent Document 2, since it is necessary to perform integration by associating spatial codes from pattern intensity images photographed under a plurality of photographing conditions, the amount of calculation increases and a large amount of computer memory is consumed. Resulting in. Therefore, there is a problem that it is difficult to perform three-dimensional shape measurement at high speed.

  The present invention has been made in view of the above problems, and an object thereof is to provide a technique for measuring a three-dimensional shape of a test object at high speed and with high accuracy.

In order to achieve the above object, a three-dimensional measurement apparatus according to an aspect of the present invention has the following arrangement. That is,
A three-dimensional measurement apparatus that measures a three-dimensional shape of the test object based on an image obtained by photographing the test object on which a pattern is projected,
The first image and the second image of the test object acquired under the first imaging condition and the second imaging condition, respectively, the exposure amount per unit time and the exposure by the light source for projecting the pattern Combining means for combining so that a magnitude relationship of luminance values corresponding to a quantity is maintained between the first image and the second image;
Measuring means for measuring the three-dimensional shape of the test object based on the image synthesized by the synthesizing means;
It is characterized by providing.

  According to the present invention, the three-dimensional shape of a test object can be measured at high speed and with high accuracy.

It is the figure which showed the three-dimensional measuring apparatus which concerns on one Embodiment of this invention. It is a flowchart which shows the procedure of the process which the three-dimensional measuring apparatus which concerns on Embodiment 1 of this invention implements. It is the figure which showed an example of the projection pattern which concerns on one Embodiment of this invention. It is a figure which shows an example of the brightness | luminance of the pattern intensity image in the several imaging condition which concerns on Embodiment 1 of this invention. It is a figure which shows the relationship between the exposure amount before and behind image composition which concerns on Embodiment 1 of this invention, and a luminance value. It is a figure which shows an example of the image brightness | luminance which added the offset which concerns on Embodiment 1 of this invention. It is the figure which showed an example of the image brightness | luminance combined by adding the offset which concerns on Embodiment 1 of this invention. It is a figure which shows an example of the space code which concerns on one Embodiment of this invention. It is a figure for demonstrating the predominance of this invention. It is a figure which shows the relationship between the exposure amount before and behind image composition in an image composition, and a luminance value. It is a figure which shows the relationship between the exposure amount and luminance value in the range compression of an image. It is a flowchart which shows the procedure of the process which the three-dimensional measuring apparatus which concerns on Embodiment 2 of this invention implements. It is a figure which shows the relationship between the exposure amount before and behind image composition which concerns on Embodiment 2 of this invention, and a luminance value. It is a figure which shows the relationship between the exposure amount before and behind image composition, and a luminance value in case the some imaging | photography conditions which concern on Embodiment 2 of this invention are three. It is a figure which shows an example of the brightness | luminance of the pattern intensity image in the several imaging condition which concerns on Embodiment 3 of this invention. It is a figure which shows the relationship between the exposure amount before and behind image composition which concerns on Embodiment 3 of this invention, and a luminance value. It is a figure which shows an example of the image brightness | luminance which added the offset which concerns on Embodiment 3. FIG. It is a figure which shows an example of the image brightness | luminance combined by adding the offset which concerns on Embodiment 3 of this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

(Embodiment 1)
FIG. 1 is a diagram showing a three-dimensional measuring apparatus according to an embodiment of the present invention. The three-dimensional measuring apparatus includes a light source 1, a pattern generation unit 2, projection lenses 3 a and 3 b, condenser lenses 5 a and 5 b, an image sensor 6, and a control / analysis unit 7. For example, the pattern generation unit 2 and the projection lenses 3 a and 3 b may constitute a projection unit, and the condenser lenses 5 a and 5 b and the imaging element 6 may constitute an imaging unit.

  The light source 1 emits a light beam. The pattern generation unit 2 generates a lattice pattern to be projected onto the test object 4 and modulates the pattern. The projection lenses 3a and 3b irradiate the object 4 with a pattern. The condensing lenses 5a and 5b condense the pattern reflected by the test object 4. The image sensor 6 acquires the intensity of the light beam diffused and reflected from the test object 4. The control / analysis unit 7 controls the light emission of the light source 1, the projection pattern of the pattern generation unit 2, the exposure time and gain of the image sensor 6, and the intensity information of the light beam obtained from the image sensor 6. Calculate the shape.

  The control / analysis unit 7 includes a light source control unit 7a, a projection pattern control unit 7b, an image sensor control unit 7c, a storage unit 7d, an image composition unit 7e, and a distance calculation unit 7f. The light source control unit 7 a controls the light emission intensity, the light emission time, and the like of the light source 1. The projection pattern control unit 7b controls the change of the projection pattern generated by the pattern generation unit 2. The image sensor control unit 7c changes the exposure time of the image sensor 6 and takes an image. The storage unit 7d stores captured images and results during distance calculation. The image composition unit 7e composes the stored images. The distance calculation unit 7f calculates the three-dimensional shape of the test object 4 from the synthesized image.

  An LED or the like is used for the light source 1. The light source 1 emits a light beam and illuminates a pattern generation unit 2 that generates a lattice pattern in which bright portions and dark portions are periodically arranged. For example, as the pattern generation unit 2, a mask pattern in which light shielding portions and non-light shielding portions are regularly arranged may be used. Further, by using a liquid crystal element, a digital mirror device (DMD) or the like, an arbitrary pattern such as a monochrome pattern or a sinusoidal pattern may be generated.

  A light beam having a lattice pattern transmitted through the pattern generation unit 2 is incident on the projection lenses 3a and 3b. The light beam emitted from the projection lens 3b is irradiated onto the test object 4, and the pattern generated by the pattern generation unit 2 is projected. The pattern reflected and diffused by the test object 4 enters the imaging lenses 5 a and 5 b, and an image of the pattern irradiated on the test object 4 is formed on the image sensor 6. The light intensity of the image of the formed pattern is detected by an image sensor 6 such as a CCD or CMOS sensor. Then, the data is processed by the control / analysis unit 7 including a CPU, a memory, a display, a storage device such as a hard disk, and a general-purpose computer having various interfaces for input / output.

  While changing the light emission intensity set by the light source control unit 7a and the pattern projected on the test object 4 specified by the projection pattern control unit 7b, the exposure time and gain set by the image sensor control unit 7c are used. The image sensor 6 captures the intensity of the projected pattern a plurality of times. Then, the photographing result is stored in the storage unit 7d.

  Then, it is determined by the image composition unit 7e whether or not photographing has been performed under preset photographing conditions (a combination of exposure time, light emission intensity of the light source, gain of the image sensor, etc.). When imaging is not performed under all imaging conditions, the emission intensity of the light source 1 is changed by the light source control unit 7a, the exposure time and gain of the imaging device 6 are changed by the imaging device control unit 7c, and the test object 4 again. The pattern is projected onto the pattern and the pattern intensity is photographed. After photographing under all photographing conditions is completed, the image composition unit 7e composes the photographed pattern intensity images and stores them in the storage unit 7d.

  Thereafter, the distance calculation unit 7f assigns a spatial code to each region of the test object 4 based on the intensity information from the combined pattern intensity image, and from the geometric relationship between the pattern generation unit 2 and the image sensor 6, The three-dimensional shape of the specimen 4 is measured.

  Based on the above principle, the sensitivity area is expanded by combining images from pattern intensity images shot under multiple shooting conditions so that the relationship between the pattern intensity and the luminance value in the multiple shooting conditions is continuously connected. An example in which the three-dimensional shape of the test object is measured from an enlarged image will be described.

  In particular, in the present embodiment, the relationship between the exposure amount per unit time and the luminance value is continuous for pattern intensity images shot under two different shooting conditions (exposure time, light emission intensity of the light source, gain of the image sensor, etc.). To be connected. Thus, an example will be described in which the three-dimensional shape of the test object is accurately measured without reversing the magnitude relationship between the exposure amount per unit time and the image luminance value under each imaging condition.

  FIG. 2 is a flowchart showing a procedure of processing performed by the three-dimensional measurement apparatus according to Embodiment 1 of the present invention.

  In step S <b> 101, the control / analysis unit 7 sets one of the two shooting conditions set in advance in the light source 1 and the image sensor 6. Here, as one of the preset shooting conditions (hereinafter referred to as the first shooting condition), even if there is a pixel saturated in the pattern intensity image except for the shadow area, a dark area is present. The exposure time, the light emission intensity of the light source, and the gain of the image sensor are set. Further, as another imaging condition (hereinafter, referred to as a second imaging condition), an exposure time that does not saturate saturated pixels in the pattern intensity image acquired under the first imaging condition, light emission of the light source 1 Intensity and gain of the image sensor 6 are set. At this time, the exposure time, the light emission intensity of the light source 1, and the gain of the image sensor 6 may all be set differently depending on the first imaging condition and the second imaging condition, or only one or two of them may be set. Different settings may be used.

In S102, the image sensor 6 captures a pattern intensity image under the capturing conditions set in S101. At this time, patterns as shown by 301 to 310 in FIG. 3 are projected in order. In this example, since it is a gray code pattern of the 4-bit spatial encoding pattern projection method, the outgoing light from the projector can be divided by 2 ⁴ (= 16). Increasing the number of bits increases the number of projections, but can increase the number of divisions of emitted light. For example, in the case of 10 bits, it becomes possible to divide 2 ¹⁰ (= 1024), the area of the test object 4 can be divided more finely, and a detailed shape can be measured.

  In step S <b> 103, the control / analysis unit 7 determines whether or not the pattern intensity image has been shot under all the preset shooting conditions. If shooting has not been performed under all shooting conditions, the process returns to S101, and the control / analysis unit 7 sets shooting conditions for which shooting has not been performed in the light source 1 and the image sensor 6. Then, the process proceeds to S102, and a pattern intensity image is taken again. On the other hand, if the pattern intensity image has been captured under all the preset capturing conditions, the process proceeds to S104. In S104, the control / analysis unit 7 synthesizes the pattern intensity images so that the magnitude relationship between the exposure amount per unit time and the luminance value in each photographing condition is maintained, and generates an image in which the sensitivity region is enlarged. .

  Here, with reference to FIG. 4A and FIG. 4B, the synthesis of the pattern intensity image according to the present embodiment will be described. First, FIG. 4A shows the luminance value of a pattern intensity image photographed under the first photographing condition that is set so that a dark region can be photographed brightly, that is, blackout is eliminated. Of the pixels X1-1 to X7-1, the pixels X3-1 and X7-1 are saturated. On the other hand, FIG. 4B shows the luminance value of the pattern intensity image photographed under the photographing conditions set so that the pixels are not saturated, that is, the saturated pixels are eliminated. All the pixels X1-2 to X7-2 are not saturated.

  In this embodiment, the luminance value of the saturated pixel in the pattern intensity image imaged under the first imaging condition is replaced with another luminance value. Specifically, an offset Δ set so that the exposure value per unit time and the luminance value in each imaging condition are continuously connected to the luminance value of the pixel of the pattern intensity image captured under the second imaging condition. By replacing with the added one, the pattern intensity image is synthesized.

That is, the luminance values of the pixels X3-1 and X7-1 in FIG. 4A are replaced with the luminance values of the pixels X3-2 and X7-2 in FIG. Next, the offset Δ will be described with reference to FIG. FIG. 5A shows the relationship between the exposure amount per unit time and the luminance value of the image for the pixels of the image sensor 6 under the first imaging condition. In the first imaging condition, the exposure time is longer than that in the second imaging condition, and the light emission intensity of the light source 1 and the gain of the image sensor 6 are high, so that the exposure amount ρ1 per unit time is saturated. On the other hand, as shown in FIG. 5B, the exposure time is shorter in the second imaging condition than in the first imaging condition, and the light emission intensity of the light source 1 and the gain of the imaging element 6 are low. Therefore, the luminance value can be acquired without saturation up to the exposure amount ρ2 per unit time larger than ρ1. In order for the relationship between the exposure amount per unit time and the luminance value in a plurality of imaging conditions to be continuously connected, as shown in FIG. 5C, the luminance value at ρ1 in the second imaging condition is the first value. It is necessary to obtain a saturated luminance value in the shooting conditions. Therefore, the exposure time _{T 1} in the first imaging condition, luminous intensity _{W 1} of the light source 1, the gain _{G 1} of the image pickup device 6, the exposure time _{T 2} in the second imaging condition, the emission intensity _{W 2} of the light source 1, the imaging If the gain G _{2 of the} element 6 and the gradation width of the image are α, the offset Δ can be expressed by the following equation (1).

Here, Roundup () is a function that rounds up the numerical value in parentheses to an integer. α is the gradation width of the image. When the gradation of the image is 12 bits, the range of the luminance value of the image is 2 ¹² = 4096. An offset Δ calculated from two different shooting conditions is added to the luminance value of the pattern intensity image shot under the second shooting condition. The pattern intensity image luminance with the offset Δ added is shown in FIG. The dotted line in FIG. 6 indicates the luminance value of the pattern intensity image captured under the second imaging condition, and the solid line indicates the luminance value to which the offset Δ is added. By adding the offset Δ, the relationship between the pattern intensity and the luminance value in a plurality of photographing conditions is continuously connected.

  Next, the pattern intensity image captured under the first imaging condition and the image obtained by adding the offset Δ to the pattern intensity image captured under the second imaging condition are combined. In the image composition, the luminance of the saturated pixel in the pattern intensity image photographed under the first photographing condition is obtained by adding the offset Δ to the luminance value of the non-saturated pixel photographed under the second photographing condition. replace.

  Here, FIG. 7 shows an outline of the combined pattern intensity image. The brightness values of the saturated pixels X3-1 and X7-1 in the pattern intensity image photographed under the first photographing condition shown in FIG. 4A were photographed under the second photographing condition shown in FIG. The pattern intensity image is synthesized by replacing the intensity values of the pixels X3-2 ′ and X7-2 ′ of the image obtained by adding the offset Δ to the pattern intensity image.

  Next, in S105, the control / analysis unit 7 measures the three-dimensional shape of the test object 4 based on the pattern intensity image synthesized in S104, and ends the process. In this embodiment, a spatial encoding pattern projection method is used. In the spatial encoding pattern projection method, brightness / darkness determination is performed from the luminance information of the pattern intensity image of each bit, and a spatial code is assigned to each region of the imaging range. For example, when the brightness determination is performed on the 1-bit image and the spatial code is assigned, the luminance and negative pattern of the positive pattern for the same pixel in the positive pattern image 302 in FIG. 3 and the negative pattern image 307 in FIG. The average value with the luminance is set as a threshold value. Then, a pixel brighter than the threshold is determined as 1 and a dark pixel is determined as 0, and a spatial code is assigned. If this is performed on the pattern intensity image of each bit and the process is performed up to 4 bits, a spatial code as shown in FIG. 8 is given. At this time, by adding an offset Δ to the luminance value of the pattern intensity image under the second imaging condition in S104, it is possible to correctly assign a spatial code without making a light / dark decision. Then, the three-dimensional shape of the test object 4 is calculated from the assigned space code and the geometric relationship between the pattern generation unit 2 and the image sensor 6.

  Here, FIG. 9 shows an example of each synthesis result when the offset is added and when it is not added. Reference numeral 901 in FIG. 9 indicates a case where the luminance value of the positive pattern under the first imaging condition is saturated and the luminance value of the positive pattern under the second imaging condition is smaller than the luminance value of the negative pattern under the first imaging condition. ing. In the case of 901, in the image synthesized without adding the offset Δ, the magnitude relationship between the luminance value of the positive pattern and the luminance value of the negative pattern is reversed, as indicated by 902 in FIG. For this reason, the light / dark judgment is wrong, the space code cannot be correctly assigned, and the three-dimensional shape of the test object 4 cannot be measured accurately.

  On the other hand, in the image synthesized by adding the offset Δ, the magnitude relationship between the luminance value of the positive pattern and the luminance value of the negative pattern is maintained as indicated by reference numeral 903 in FIG. It can be performed. Therefore, it is possible to correctly assign the space code, and to accurately measure the three-dimensional shape of the test object 4.

  In addition, as an image composition method in which the magnitude relationship between the luminance value of the positive pattern and the luminance value of the negative pattern is not reversed, as shown in FIGS. 10 (a) to 10 (c), per unit time in each photographing condition. There is widely known a method of performing an operation on image luminance so that the relationship between the exposure amount and the luminance coincides to synthesize an image. However, the method is implemented by multiplying the image brightness value of the second shooting condition by the ratio of the brightness value with the same exposure amount per unit time in the first shooting condition and the second shooting condition. There are many. For this reason, as the ratio between the luminance values of the first shooting condition and the second shooting condition increases, the combined saturated luminance value increases. Therefore, a large amount of memory is required for the storage unit 7d in order to store the combined image.

  On the other hand, even in the case of two images, the number of gradations of the image is less than twice the number of gradations of the image before composition even when the ratio of luminance values exceeds twice under the two shooting conditions. Don't be. However, the saturation luminance value is larger than twice, and the memory amount necessary for the storage unit 7d is more than twice, but the information amount is not more than twice, and the memory use efficiency is not high. For example, when the gradation of the image before synthesis is 12 bits, a gradation value of 13 bits or more (8192 or more) is required, but the number of gradations is not more than 8192, so the data arrangement in the memory space is sparse. The memory usage efficiency is not high.

  It is also intended for human viewing in photos, etc., and when combining images from multiple shooting conditions to expand the dynamic range, to reduce the increase in memory usage, As shown in FIG. 11B, compression may be performed using a logarithmic function to reduce an increase in memory usage. Alternatively, as shown in FIG. 11C, the increase in memory usage may be reduced by linearly compressing the gradation width (lowering the bit). However, when such compression is performed, the number of gradations does not change when the exposure amount per unit time in FIG. 11A is in the range of ρ1 to ρ2, but the gradation width is doubled. Since the data density is low, it is unlikely to be a problem. On the other hand, a large amount of gradation information may be lost in the region up to ρ1 where the data density is high.

  On the other hand, according to the image composition according to the present embodiment, even if the ratio of the luminance values becomes twice or more, the saturation luminance after the composition is less than twice. Therefore, for example, when the gradation of an image before composition is 12 bits, it can be expressed with a gradation value of 13 bits or less (8192 or less), and the memory use efficiency is high. Images can be stored.

  In addition, even if the image brightness after composition is compressed to the gradation width before composition, the data density is constant regardless of the range of exposure per unit time, so a lot of information may be lost in a specific area. Absent. For this reason, the amount of gradation information after compression tends to remain large compared to the conventional method. Compared with the method of calculating and integrating the shape of the test object from the pattern intensity image of each imaging condition as disclosed in Patent Document 2, the image synthesis method according to the present embodiment has a small amount of calculation. Thus, the three-dimensional shape of the test object can be measured at a higher speed.

  As described above, according to the present embodiment, areas having various reflection characteristics such as a low reflection area and a high reflection area are widely distributed, and it is difficult to perform sufficient shape measurement by shooting under one shooting condition. For inspections, three-dimensional shapes can be measured more quickly and accurately with a minimum increase in memory.

(Embodiment 2)
In the present embodiment, an example will be described in which an offset to be added at the time of image composition is a fixed value without being calculated from a plurality of shooting conditions. The configuration of the three-dimensional measurement apparatus necessary for realizing this embodiment is the same as that shown in FIG. The function of each part is the same as in FIG.

  FIG. 12 is a flowchart showing a procedure of processing performed by the three-dimensional measurement apparatus according to Embodiment 2 of the present invention.

  In step S <b> 201, the control / analysis unit 7 sets one of the two shooting conditions set in advance in the light source 1 and the image sensor 6 as in step S <b> 101. In S202, the image sensor 6 captures a pattern intensity image under the capturing conditions set in S201, as in S102. In S203, similarly to S103, the control / analysis unit 7 determines whether or not the pattern intensity image has been shot under all the preset shooting conditions.

  In step S <b> 204, the control / analysis unit 7 synthesizes a pattern intensity image using a fixed value offset to generate an image in which the sensitivity region is enlarged.

  Here, the synthesis of the pattern intensity image according to the present embodiment will be described. Also in this embodiment, the brightness value of the saturated pixel in the pattern intensity image shot under the first shooting condition is added to the brightness value of the pixel of the pattern intensity image shot under the second shooting condition. Is replaced with the pattern intensity image. The first shooting condition and the second shooting condition are the same as in the first embodiment.

In the first embodiment, the example in which the offset Δ is calculated so that the exposure amount per unit time and the luminance value are continuously connected in a plurality of shooting conditions has been described. On the other hand, in the present embodiment, the pattern intensity image is synthesized using a fixed value offset Δ _const regardless of a plurality of imaging conditions. The offset Δ _const is set to a value that does not reverse the magnitude relationship between the amount of exposure amount per unit time and the luminance value between the first imaging condition and the second imaging condition.

For example, the offset Δ _const is a value obtained by adding a predetermined value (for example, 1) to the saturation luminance value of the pattern intensity image before synthesis. For example, when the gradation width of the pre-combination image is 12 bits, the exposure amount per unit time of the first shooting condition and the second shooting condition is set by setting the offset Δ _const to 4096 (= 2 ¹² +1). The pattern intensity image can be synthesized without reversing the magnitude relationship between the brightness value and the brightness value.

Here, FIG. 13C shows the exposure amount per unit time and the luminance value when the fixed value offset Δ _const is a value obtained by adding 1 to the saturation luminance value of the pattern intensity image before synthesis. Show the relationship. FIG. 13A shows the relationship between the exposure amount per unit time and the luminance value of the image under the first imaging condition. FIG. 13B shows the relationship between the exposure amount per unit time and the luminance value of the image under the second imaging condition. By performing synthesis with a fixed offset larger than the saturation luminance value of the image to be synthesized, compared to the first embodiment, although the memory usage efficiency is somewhat reduced, the processing for calculating the offset is not required, so that the processing speed is higher. It is possible to synthesize a pattern intensity image.

Further, in the present embodiment, the case of two shooting conditions has been described, but an offset Δ _const is added every time a pattern intensity image is synthesized even under three or more shooting conditions, so that the exposure amount per unit time can be increased. A pattern intensity image can be synthesized so that the magnitude relationship between the luminance and the luminance value is not reversed.

FIG. 14 shows the relationship between the exposure amount per unit time and the luminance value when images of three shooting conditions are combined. In addition to the first imaging condition and the second imaging condition, when a third imaging condition that is saturated when the exposure amount per unit time is ρ3 is added, a pattern intensity image of the third imaging condition is synthesized. At this time, 2 × Δ _const may be offset and added.

Even when the number of shooting conditions is N (N is a natural number), if (N−1) × Δ _const is offset and added to the image of the Nth shooting condition, the exposure amount per unit time The pattern intensity image can be synthesized without reversing the relationship between the magnitude of the brightness and the brightness value.

  In S205, similarly to S105, the control / analysis unit 7 measures the three-dimensional shape of the test object 4 from the pattern intensity image synthesized in S204, and ends the process. In this embodiment, a spatial encoding pattern projection method is used.

  As described above, according to the present embodiment, it is possible to synthesize a pattern intensity image with less calculation processing than in the first embodiment, and it is possible to measure a three-dimensional shape more quickly and accurately. .

(Embodiment 3)
In the present embodiment, an example in which different offsets are applied as offsets added at the time of image composition when there are three shooting conditions will be described. The configuration of the three-dimensional measurement apparatus necessary for realizing this embodiment is the same as that shown in FIG. The function of each part is the same as in FIG.

  In the present embodiment, processing is performed on the same principle as in the first embodiment. Image synthesis is performed using pattern intensity images photographed under three photographing conditions so that the relationship between the exposure amount per unit time and the luminance value is continuously connected. Thereby, the sensitivity region is expanded, and the three-dimensional shape of the test object 4 is measured from the image in which the sensitivity region is expanded.

  Hereinafter, the procedure of the process according to the present embodiment will be described with reference to FIG.

  In step S <b> 101, the control / analysis unit 7 sets one of the three shooting conditions set in advance in the light source 1 and the image sensor 6.

Assuming that the average luminance of the pattern intensity image is B _{1 to} B ₃ , three shooting conditions are set so that the average luminance satisfies the following formula (2).

As a first imaging condition, in the pattern intensity image, an exposure time, a light emission intensity of a light source, and a gain of an imaging element are set so that a dark area is brightly photographed even if there is a saturated pixel except for a shadow area. Set. As the second imaging condition, to the extent that pixel is saturated in acquired pattern intensity image at a first imaging condition is not saturated, as much as possible the average luminance B ₂ of the pattern intensity image is increased, the exposure time The emission intensity of the light source and the gain of the image sensor are set.

The third imaging condition, to the extent that pixel is saturated in acquired pattern intensity image in the second imaging condition is not saturated, so that the average luminance B ₃ as possible pattern intensity image increases, exposure time, light source The emission intensity and gain of the image sensor are set.

  In addition, in the combined pattern intensity image, three imaging conditions may be set so that there is no saturated pixel and the average luminance is high. At this time, the exposure time, the light emission intensity of the light source, and the gain of the image sensor may all be set differently in the first shooting condition, the second shooting condition, and the third shooting condition. Alternatively, only two different settings may be used. Since the processing of S102 and S103 is the same as that of the first embodiment, the description thereof is omitted.

  Subsequently, in S104, the control / analysis unit 7 combines the pattern intensity images so as to maintain the magnitude relationship between the exposure amount per unit time and the luminance value in each photographing condition, and generates an image in which the sensitivity region is enlarged. To do.

  Here, with reference to FIGS. 15A to 15C, the synthesis of the pattern intensity image according to the present embodiment will be described. FIG. 15A shows the luminance value of the pattern intensity image photographed under the first photographing condition, and the pixels X3-1 and X7-1 are saturated among the pixels X1-1 to X7-1. FIG. 15B shows the luminance value of the pattern intensity image photographed under the second photographing condition, and the pixel X7-2 out of the pixels X1-2 to X7-2 is saturated. FIG. 15C shows the luminance value of the pattern intensity image photographed under the third photographing condition, and all the pixels X1-3 to X7-3 are not saturated.

  In the present embodiment, when the pattern intensity image is combined, the pixel that is saturated in the image captured under the first imaging condition is calculated based on the first imaging condition and the second imaging condition. Replace Δ1 with the luminance value of the image shot under the second shooting condition. Thus, the first stage image is synthesized. Next, in addition to the offset Δ1, pixels that are saturated in the combined image are further shot with the offset Δ2 calculated based on the second shooting condition and the third shooting condition under the third shooting condition. Replace with the brightness value of the image. As a result, the second stage of synthesis is performed, and images of all shooting conditions are synthesized.

  Specifically, first, the luminance values of the pixels X3-1 and X7-1 in FIG. 15A are obtained by adding the offset Δ1 to the luminance values of the pixels X3-2 and X7-2 in FIG. replace. At this time, the offset Δ1 is set so that the relationship between the exposure amount per unit time and the luminance value in the first shooting condition and the second shooting condition is continuously connected.

  Next, the luminance value of the pixel X7-2 in FIG. 15B is replaced with the luminance value of the pixel X7-3 in FIG. 15C plus the offset Δ1 and the offset Δ2. At this time, the offset Δ2 is set so that the relationship between the exposure amount per unit time and the luminance value in the second imaging condition and the third imaging condition is continuously connected.

  The addition of the offset Δ1 and the offset Δ2 will be described with reference to FIGS. FIG. 16A shows the relationship between the exposure amount per unit time and the luminance value of the image under the first imaging condition. In the first imaging condition, the exposure time is longer than in the second imaging condition and the third imaging condition, and the light emission intensity of the light source 1 and the gain of the image sensor 6 are high. It will be saturated at the incident light quantity ρ1 ′.

  FIG. 16B shows the relationship between the exposure amount per unit time and the luminance value of the image under the second imaging condition. In the second imaging condition, as shown in FIG. 16B, the exposure time is shorter than the first imaging condition, and the light emission intensity of the light source 1 and the gain of the image sensor 6 are low. The luminance value can be acquired without being saturated up to a large exposure amount ρ2 ′ per unit time.

  FIG. 16C shows the relationship between the exposure amount per unit time and the luminance value of the image under the third imaging condition. In the third imaging condition, as shown in FIG. 16C, the exposure time is shorter than that in the second imaging condition, and the light emission intensity of the light source 1 and the gain of the image sensor 6 are further lower. , The luminance value can be acquired without saturation up to an exposure amount ρ3 ′ per unit time larger than ρ2 ′.

  Then, in order for the relationship between the exposure amount per unit time and the luminance value in a plurality of imaging conditions to be continuously connected, as shown in FIG. 16D, the luminance value at ρ1 ′ in the second imaging condition is It is necessary to have a saturated luminance value in the first imaging condition. In addition, the luminance value at ρ2 ′ under the third imaging condition needs to be the saturation luminance value under the second imaging condition.

Here, it is assumed that the exposure time T ₁ in the first imaging condition, the light emission intensity of the light source 1 is W ₁ , and the gain of the image sensor 6 is G ₁ . The exposure time in the second imaging condition is T ₂ , the light emission intensity of the light source 1 is W ₂ , and the gain of the image sensor 6 is G ₂ . Furthermore, the exposure time under the third imaging condition is T ₃ , the light emission intensity of the light source 1 is W ₃ , the gain of the image sensor 6 is G ₃ , and the gradation width of the image is α. When these are used, the offset Δ1 and the offset Δ2 can be expressed by the following equations (3) and (4).

Here, Roundup () is a function that rounds up the numerical value in parentheses to an integer. α is the gradation width of the image. When the gradation of the image is 12 bits, the range of the luminance value of the image is 2 ¹² = 4096. The offset Δ1 is added to the luminance value of the pattern intensity image photographed under the second photographing condition. Further, the offset Δ1 and the offset Δ2 are added to the luminance value of the pattern intensity image photographed under the third photographing condition.

  FIG. 17A shows the pattern intensity image luminance to which the offset Δ1 is added. The dotted line in FIG. 17A indicates the luminance value of the pattern intensity image captured under the second imaging condition, and the solid line indicates the luminance value to which the offset Δ1 is added.

  Further, FIG. 17B shows the pattern intensity image luminance to which the offset Δ1 and the offset Δ2 are added. The dotted line in FIG. 17B indicates the luminance of the pattern intensity image captured under the third imaging condition, the alternate long and short dash line indicates the luminance value obtained by adding the offset Δ1, and the solid line indicates the luminance value obtained by adding the offset Δ1 and the offset Δ2. Show. By adding the offset Δ1 and the offset Δ2, it is possible to continuously connect the relationship between the exposure amount per unit time and the luminance value under a plurality of photographing conditions.

  Next, the pattern intensity image shot under the first shooting condition and the image obtained by adding the offsets Δ1 and Δ2 to the pattern intensity image shot under the second shooting condition and the third shooting condition are combined.

  In the image composition, the luminance of the saturated pixel in the pattern intensity image photographed under the first photographing condition is added to the luminance value of the non-saturated pixel photographed under the second photographing condition by adding the offset Δ1. replace. Next, in the synthesized pattern intensity image, the luminance of the saturated pixel is replaced with the luminance value of the non-saturated pixel imaged under the third imaging condition plus the offset Δ1 and offset Δ2.

  Reference numeral 1801 in FIG. 18 is an example of the brightness of the combined pattern intensity image. The intensity values of the pixels X3-1 and X7-1 that are saturated in the pattern intensity image photographed under the first photographing condition as shown in 1802 are the pattern intensities photographed under the second photographing condition as shown in 1803. The brightness values of the pixels X3-2 ′ and X7-2 ′ of the image obtained by adding the offset Δ1 to the image are respectively replaced. Next, in the image in which the luminance values of the pixels X3-1 and X7-1 are replaced with the luminance values of the pixels X3-2 ′ and X7-2 ′, the luminance value of the pixel X7-2 ′ is a pixel as indicated by 1804. A pattern intensity image as indicated by 1801 is synthesized by replacing with the luminance value of X7-3 ′. Thereby, it is possible to synthesize a pattern intensity image without reversing the magnitude relationship between the amount of exposure per unit time and the luminance value under each photographing condition.

  Next, in S105, similarly to the first embodiment, the control / analysis unit 7 measures the three-dimensional shape of the test object 4 based on the pattern intensity image synthesized in S104, and ends the process. In this embodiment, a spatial encoding pattern projection method is used. As described above, in this embodiment, even when the ratio of the luminance values under the condition that the exposure amount per unit time is the same for the three photographing conditions is 3 times or more, the saturation luminance after synthesis is 3 It can only be less than double. For this reason, it is possible to express with 14-bit gradation, high memory use efficiency, and it is possible to store the combined image with a minimum increase in memory.

  Further, the present invention can be applied even when the number of shooting conditions is increased from three and N shooting conditions (N is a natural number). In this case, the offset ΔN−1 is calculated based on the (N−1) th shooting condition and the Nth shooting condition. Then, the luminance value of the saturated pixel in the pattern intensity image photographed under the (N-1) th photographing condition is added to the luminance value of the pattern intensity image photographed under the Nth photographing condition from offsets [Delta] 1 to [Delta] N-1. Replace with something. As a result, it is possible to synthesize a pattern intensity image that can correctly perform contrast determination in the spatially encoded pattern projection method without reversing the magnitude relationship between the amount of exposure per unit time and the luminance value in each shooting condition. it can.

  As described above, according to the present embodiment, areas having various reflection characteristics such as a low reflection area and a high reflection area are widely distributed, and it is difficult to perform sufficient shape measurement by shooting under one shooting condition. For inspections, three-dimensional shapes can be measured more quickly and accurately with a minimum increase in memory.

(Other embodiments)
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in the computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

  1: Light source, 2: Pattern generation unit, 3a, 3b: Projection lens, 4: Test object, 5a, 5b: Condensing lens, 6: Image sensor, 7: Control / analysis unit, 7a: Light source control unit, 7b : Projection pattern control unit, 7c: image sensor control unit, 7d: storage unit, 7e: image composition unit, 7f: distance calculation unit

Claims (13)

A three-dimensional measurement apparatus that measures a three-dimensional shape of the test object based on an image obtained by photographing the test object on which a pattern is projected,
The first image and the second image of the test object acquired under the first imaging condition and the second imaging condition, respectively, the exposure amount per unit time and the exposure by the light source for projecting the pattern Combining means for combining so that a magnitude relationship of luminance values corresponding to a quantity is maintained between the first image and the second image;
Measuring means for measuring the three-dimensional shape of the test object based on the image synthesized by the synthesizing means;
A three-dimensional measuring apparatus comprising:   The synthesizing unit performs synthesis so that a relationship between an exposure amount per unit time and a luminance value corresponding to the exposure amount under the first imaging condition and the second imaging condition is continuously connected. The three-dimensional measuring apparatus according to claim 1.   The three-dimensional measurement apparatus according to claim 2, wherein the synthesizing unit calculates an offset based on the first imaging condition and the second imaging condition, and performs synthesis using the offset.   The synthesizing unit calculates the offset based on an exposure time of a light source, a light emission intensity, a gain of an image sensor, and a gradation width of an image in each of the first photographing condition and the second photographing condition. The three-dimensional measuring apparatus according to claim 3.   The three-dimensional measurement apparatus according to claim 1, wherein the synthesizing unit performs synthesis using a fixed value offset.   The three-dimensional measurement apparatus according to claim 5, wherein the offset of the fixed value is a value obtained by adding a predetermined value to the saturation luminance value of the image.   The composition is performed by replacing the luminance value of the saturated pixel in the image photographed under the first photographing condition with the luminance value of the pixel of the image photographed under the second photographing condition plus the offset. The three-dimensional measuring apparatus according to claim 4, wherein the three-dimensional measuring apparatus is performed.   8. The method according to claim 1, wherein at least one of an exposure time and light emission intensity by the light source and a gain of the image sensor is different between the first imaging condition and the second imaging condition. The three-dimensional measuring device described in 1.   The synthesizing means includes a first image, a second image, and a third image of the test object acquired under the first imaging condition, the second imaging condition, and the third imaging condition, respectively. The magnitude relationship of the luminance values corresponding to the exposure amount per unit time by the light source for projecting the pattern is held between the first image, the second image, and the third image. The three-dimensional measuring apparatus according to claim 1, wherein the three-dimensional measuring apparatus is synthesized as follows. The synthesis means includes
Calculating a first offset based on an exposure time, a light emission intensity, a gain of an image sensor, and a gradation width of an image of each light source of the first imaging condition and the second imaging condition;
Calculating a second offset based on the exposure time, the light emission intensity, the gain of the image sensor, and the gradation width of the image of each light source of the second imaging condition and the third imaging condition;
The three-dimensional measurement apparatus according to claim 9, wherein synthesis is performed using the first offset and the second offset. A projection unit that projects the pattern onto the test object;
The three-dimensional measurement apparatus according to claim 1, further comprising: an imaging unit that images the object on which the pattern is projected. A control method for a three-dimensional measurement apparatus that measures a three-dimensional shape of the test object based on an image obtained by photographing the test object on which a pattern is projected,
An exposure per unit time by a light source for projecting the first image and the second image of the test object acquired by the combining unit under the first imaging condition and the second imaging condition, respectively. A combining step of combining the luminance values corresponding to the amounts so as to be maintained between the first image and the second image;
A measuring step for measuring a three-dimensional shape of the test object based on the image synthesized by the synthesis step; and
A control method for a three-dimensional measuring apparatus, comprising:   The program for making a computer perform each process of the control method of the three-dimensional measuring apparatus of Claim 12.

Images