JPH0498108A - Three-dimensional shape measuring device - Google Patents

Abstract

PURPOSE:To obtain a device by which an object of a three-dimensional shape can be measured in a high speed, by extracting code series of slit images of first and second lattice patterns which have been projected on a measuring object. CONSTITUTION:A plurality of patterns are produced by a projection pattern producer 11, and they are projected on a measuring object O by means of a projection device 10. The picture-images of the measuring object O at the time respective patterns have been projected are input through an imaging device 20, and they are stored in the devices 21a, 21b for picture-image memories I, II. Further, while scanning, the picture-image memories 21a, 21b are read, and the positions and brightness of slit images are obtained by a slit-image detecting part 22. Then, in a cord extracting part 23, the brightness of slit images in two sheets of picture images is compared to determine its code, and the code series obtained in a decoding part 24 are decoded. By the operation like this, since each slit image can be corresponded to a projection slit, the three-dimensional positions and the shape of an object can be measured.

Description

Detailed Description of the Invention (Industrial Application Field) The present invention projects a lattice pattern onto a three-dimensional object to be measured, and detects the projected pattern image from an image obtained by imaging the object. , relates to a device that non-contactly measures the three-dimensional position and shape of the three-dimensional object to be measured, and particularly relates to a device that measures using a coded grid pattern characterized by color, brightness, etc. according to a code series. .

(Prior Art) Various techniques for non-contact measurement of the three-dimensional position and shape of a three-dimensional object to be measured have been developed, and have been put to practical use in industrial, medical, and other fields. When using this type of technology, ■ high precision and high reliability, ■ high speed,
Depending on the application, conditions such as ■A wide range of applicable targets/objects, and ■Easy processing and control are required.

Typical techniques of this type include (a) a method of projecting pre-structured illumination light onto an object and determining the position of the light image from an image of the object; b) There is a stereo method in which the object position is measured using a plurality of images of the object taken from different directions.

In the stereo method (b) of item 1, there is a problem that automatic measurement is difficult because a processing method for correlating object images in multiple images has not been established. On the other hand, the method (a) using structured illumination light is easy to process because it only needs to be able to extract a light image from an image, and has been put to practical use as an industrial measurement method. .

In the above, the simplest structured illumination light is spot light or slit light, and it is easy to detect these light images from the image of the object onto which it is projected, and processing using high-speed hardware is also possible. It is. However, with spot light, only one point on an object can be measured from one image, and with slit light, only the coordinates of a point sequence along the optical image can be measured from one image. Therefore, in order to measure the entire shape of an object, spot light requires two-dimensional scanning,
Slit light performs one-dimensional scanning, and a light image must be detected from the image each time it is scanned. For this reason, measurement takes time and it is difficult to measure moving objects.

In order to solve this problem, a method of projecting a two-dimensional structured pattern has been proposed.

FIG. 6 is a basic configuration diagram of a measuring device that projects a lattice pattern onto an object to be measured and images the object to measure its three-dimensional shape.

In the figure, 1 is a projection device for projecting the lattice pattern 3 onto an object, 2 is an imaging device for observing the object to be measured ○, 3 is a lattice pattern consisting of a row of slits, etc., 0 is the object to be measured, Ll, L2 is a projection lens of the projection device 1 and an imaging lens of the imaging device 2, respectively. The principle of three-dimensional measurement using the apparatus configured as shown in the figure will be briefly explained.

As shown in the figure, the position of an image of light passing through a slit pl in the grating pattern 3 projected onto the object to be measured O by the projection lens L1 is defined as pi.

Assume that this image is formed in the imaging device 2 at a point i/ on the imaging surface A through the imaging lens L2. At this time, by measuring the position X of point i/ on the imaging surface A, the three-dimensional coordinates of point pl on the object can be calculated. As shown in the figure, the center of the projection lens L1 is the origin, and the optical axis is Z.
After determining the coordinate system for the axes, the coordinate values of point pi (Xi,
Yi, Zi) are given by the following equation.

However, Q is the distance between the projection lens Ll and the grating pattern 3, Q2 is the distance between the imaging lens L2 and the imaging surface A, and D is the distance between the centers of the projection lens Ll and the imaging lens L2.

Further, ul is the distance between the slit p1 of interest in the grating pattern 3 and the center of the projection lens L1, and the pitch of the grating is p.

If the slit number counted from the center of the lens L1 is 1, then ui=pXi.

In order to perform three-dimensional measurement of the object to be measured O according to the above principle, as is clear from equation (1), for each slit pi of the projected slit lattice pattern 3, the slit image p1 of the projected slit lattice pattern 3 must be Coordinate X on imaging plane A
1 must be able to measure 4 times. That is, in order to uniquely measure the three-dimensional shape of the object to be measured O, it is necessary to be able to associate a pattern image obtained by imaging the object with a projected pattern.

(Problems to be Solved by the Invention) In order to construct this kind of structured projection pattern as described above, the following methods have been proposed in the past.

■A method that uses a grid pattern in which multiple slits are characterized by changes in color, brightness, slit width, slit length, etc. according to a code sequence (this is called the spatial code method).

FIG. 7 shows an example of a grid pattern encoded in three colors, R (red), G (green), and B (blue), and a grid image captured by projecting this onto an object. In this method, the color code of each slit image is extracted from the grid image as shown in FIG. It can be made to correspond to the slits of the projection grating shown in FIG. However, one of the major problems with this method is that characteristics such as the color of the slit image cannot be extracted correctly due to the influence of the color and pattern of the object to be measured.

■A method of projecting multiple structured patterns in chronological order (this is called the chronological code method). In this method, a time-series code is extracted from a plurality of images taken for each projection pattern, and by decoding the time-series code, it is possible to associate the time-series code with the projection pattern. However, with this method,
Since it is necessary to obtain multiple images in chronological order, it is difficult to measure fast-moving objects.

(Objective of the Invention) The present invention solves the drawbacks of the above-mentioned spatial code method and time-series code method, and provides a device that can measure a three-dimensional object at high speed without being affected by the color or pattern of the object to be measured. The purpose is to obtain.

(Means for Solving the Problems) In order to solve the above problems and achieve the objects, the present invention provides means for generating a grid pattern and projecting the grid pattern onto a three-dimensional object, and a first method for projecting a uniform white pattern. A second image on which a grid pattern encoded by the image and color changes is projected.
means for capturing images of each of the images, means for storing the first and second images captured by the image capturing means in a memory, means for detecting a slit image from each of the second images, and a second image.
means for extracting a code sequence as a color code of the slit image using a color that has a maximum ratio between the color component of the brightness value of the slit image in the image and the color component of the brightness value of the position of the slit image in the first image; , a decoding means for measuring the position and shape of a three-dimensional object by decoding the code sequence and associating it with the projected grid pattern.

(Function) The present invention uses a projection display or an electronically controllable pattern projection device to generate slits that are coded by changing color, brightness, slit width, etc. according to a code series, similar to the above-mentioned spatial code method. The first consisting of columns
A first image obtained by generating a lattice pattern, projecting it onto an object to be measured, and capturing the image with an imaging device such as a TV camera, and a code different from the code sequence of the first lattice pattern. A second image obtained by generating a second lattice pattern composed of a series of encoded slit rows and projecting it onto the object to be measured, or uniform illumination light generated by the pattern projection device. A first or second grating projected onto the object to be measured is calculated by comparing the first and second images with a second image obtained by imaging the object to be measured. The position and shape of a three-dimensional object are measured by extracting the code sequence of the slit image of the pattern and associating it with the projected slit array.

(Embodiment) FIG. 1 shows a configuration diagram according to an embodiment of the present invention, in which a projection pattern projection device 10 and an imaging device 20 have the same configuration as the solid projection device 1 and imaging device 2 shown in FIG. It uses That is, in the figure, 0 is the object to be measured, 10
1 is a projection display or a projection device whose projection pattern can be electronically controlled; 11 is a projection pattern generator; 20 is an imaging device such as a TV camera; 21a;
1b is an image memory I for storing image data captured by a TV camera.

2, 22 is a slit image detection unit from image data, 23 is a code extraction unit for the extracted slit image, and 24 is a decoding unit for the code sequence of the detected slit image.

In the apparatus configured as shown in the figure, a projection pattern generator 11 generates a plurality of patterns to
The projection device 10 projects the image on the image. Images of the object to be measured O when each pattern is projected are inputted by the imaging device 20, and are stored in the image memories 1 and 21a and 21 of the image memories 1 and II, respectively.
Store in b.

Furthermore, the image memories 21a and 21b are scanned and read out, and the slit image detection section 22 determines the position and brightness of the slit image. Next, the code extractor 23 compares the brightness of the slit images of the two images to determine the code, and the decoder 24 decodes the obtained code sequence. Through such an operation, each slit image can be associated with the projection slit, so that the three-dimensional position and shape of the object can be measured as explained in FIG. 6 above.

Furthermore, the operation will be explained in the case where a specific projection pattern is used. FIG. 2 shows an example of an image obtained by projecting two types of patterns onto the object to be measured O, which are stored in the image memories 21a and 22b, respectively.

In the figure, (a) is a color image when a uniform white pattern is projected, and (b) is an R image as shown in FIG. 7(a).
This is an example of a color image obtained by projecting a grid pattern encoded in three colors: (red), G (green), and B (blue). (b)
An image like this is called a coded grid image.

When the slit image detection unit 22 scans the coded lattice image in (b) in the X direction, the brightness of each slit image becomes high, so for example, by selecting the peak point of brightness, the slit image can be easily detected. can be detected. Image coordinates of this peak point (×,.

yP) and the color component of the luminance value at the peak point (■, .

I, , Ib).

At the same time, the color component Dr of the luminance value at the same coordinate value (xP+yp) of the image obtained by projecting the uniform white pattern of (a). ,■,. + Ib. ) to measure. Furthermore, the code extraction unit 23 calculates the ratio of color components of each luminance value (L/Ir., Ig/Ig., ■b/1b-), and focuses on the color with the largest value. Let it be the color code of the slit image.

According to the above means, the influence of the color of the object to be measured O is
Since the correction is performed using an image obtained by projecting a uniform white pattern, it is possible to stably extract color codes without being affected by the measurement target.

The above is an example of using a color code grid, but a similar method can also be applied to a grid pattern encoded by changes in brightness. Normally, the brightness of an image is greatly influenced by the pattern on the surface of an object, so it is extremely difficult to accurately detect changes in brightness of this type of projection pattern.

Now, as shown in the example in Figure 3, the brightness of the slit is set to 3.
Suppose we use a grid pattern encoded by varying the steps S,, S,, S,.

This lattice pattern is projected onto the object to be measured O as shown in Figure 2 (
A coded grid image similar to b) is captured, a slit image is detected as in the case of the color coded grid above, and its brightness value ■
get. At the same time, the brightness value ■ is obtained from the same position of the image in which a uniform white pattern as shown in FIG. 2(a) is projected. seek. The ratio of these brightness values (γ-I/I) is calculated, and the brightness code C is determined according to the following conditions.

If γ(T, then C=S If T, < γ(T, then C=S.

If T〈γ, then C-separate. However, T,, T, is a threshold value. As described above, by using the brightness value of an image projected with a uniform white pattern, it is possible to extract the sign due to changes in the brightness of the slit without being affected by the reflectance of the object to be measured. .

In the explanation of the embodiment described above, by using an image captured by projecting a uniform white pattern, the code of the slit image is extracted without being affected by the color or pattern of the object to be measured. there were.

However, when using a color coding grid as shown in Fig. 2, if any of the luminance color components (1,,,I,.+Ib.) of an image projected with a uniform white pattern becomes very small, , the ratio of the luminance of the slit image corresponding to the color component to the color component (■r/■r.

Ig/I. , Ib/Ib. ) becomes large, resulting in an error in determining the color code of the slit. This situation is likely to occur when the object has a nearly monochromatic color.

In order to cope with such a situation, the projection pattern generator 11 of FIG. 1 generates a plurality of encoded lattice patterns as shown in FIG. 4. That is, FIG. 4 is an example of a color encoding lattice pattern encoded using three colors of RGB, and is a lattice pattern in which specific color codes are sequentially assigned to the same code series.

In the measuring device shown in FIG. 1, the plurality of lattice patterns configured as described above are sequentially projected onto the object to be measured, and the images are sequentially captured by the imaging device 2o, and three types of images are respectively stored in the image memory. Store. As in the case of FIG. 2, each image is scanned to detect a slit image and the peak value of image brightness is determined. This is expressed as (Ir,, t,
, Ih, ) (Ir,.

■ Lt-1Ib-) (Ir3.Ig-2Ib-).

As shown in Fig. 4, when focusing on a certain slit image, its color code is C-(R
,G,B),C2-(G,B,R),c,=(B.

R, G), so we focused on the maximum value of the color component of the peak brightness value of each of the three images, and calculated the above C,...
The code can be determined by determining which pattern among C, , C, by majority vote. or,
From the peak brightness values of the three images, I, = (Ir, +I,
+I, , ), I, = (I, 10 Ib2 + I, , ), L =
Find (I,,+I,,+I,,),C,=Max(1
,,I,,I,) can also define a color code pattern.

FIG. 5 shows an example of the configuration of an apparatus that processes the three types of coded grid images described above. In the figure, the imaging device 2 of FIG.
Only the part after 0 is shown. In the figure, 2
1a, 21b, 21c are three sets of image memories r, n, m, 22a for storing the above three types of coded grid images.
, 22b and 22c are image memories 21a and 22c, respectively.
A slit image detection unit scans 21b and 21c to read image data and detects the position and peak brightness value of the slit image. The extraction circuit 232 is a circuit that compares the extracted set of three color codes with the above code pattern to determine the code. The code sequence of the encoded lattice image determined as described above is decoded by the decoding unit 24 and is associated with the projected lattice pattern.

As explained above, by sequentially changing the color codes assigned to a fixed code sequence, multiple coded grid patterns are constructed, and these are sequentially projected to determine the color of the target object using the captured image. The code of a coded grid image can be extracted without being affected by the pattern.

Furthermore, in the above description, all of the images in which a plurality of encoded lattice patterns are projected are used, but it is not always necessary to use all of the patterns. That is, if a code can be reliably extracted from a grid image of a certain coded grid pattern, there is no need to project any more specific patterns.

Therefore, for example, in FIG. 5, when the color code extraction unit 231 performs color determination of the slit image, the maximum value of the difference between the luminance values of each color.

Max(l Ir-IEl, l Ig-Ibl l
If 'b-11)/(I, +I +Ib) is equal to or greater than the threshold value, the color determination is determined to be correct. If the color judgment is correctly performed for all the strip images, the projection of any new coding patterns is discontinued. As described above, by determining the correctness of code extraction of the lattice image, measurement according to the object becomes possible.

(Effects of the Invention) As explained above, according to the present invention, a lattice pattern encoded by changing characteristics such as color and transmittance is projected onto an object, and the three-dimensional position of the object is determined from an image taken of the object. When measuring the shape, the code of the lattice image can be extracted without being affected by changes in the color, pattern, or reflectance of the object being measured, making it possible to perform highly reliable measurements. Become.

In the present invention, it is also necessary to sequentially project two or three types of patterns and capture images each time, but compared to the method of sequentially projecting time-series patterns, the number of images required is far smaller, so movement is reduced. If the speed is not too fast, it is also possible to measure moving objects.

In the above explanation, the characteristics of the three colors RGB were used when performing color encoding, but according to the present invention, the code of the color encoded grid image can be extracted without being affected by the color or pattern of the object. Therefore, more colors can be used for encoding.

Now, when encoding according to the maximum length sequence code using q features, if the code length is k, the total sequence length is (q'
−1) can be created. Therefore, if the number of required lattice slits is constant, the larger q is, the smaller the code length can be used.
The number of slits required to decode the lattice image can also be reduced.

For example, when creating a coding grid consisting of about 200 slits and coding with three colors, the code length =
When applying a code sequence of 5 and encoding with 6 colors, a code sequence of =3 is applied to the code length. It turns out. In other words, when decoding a coded lattice image, five consecutive slit images are required in the case of three colors, but six consecutive slit images are required.
In the case of color, it is sufficient to obtain three slit images, so
Therefore, even small objects can be measured.

[Brief explanation of drawings]

Figure 1 is a configuration diagram according to an embodiment of the present invention, Figure 2 is an example of an image obtained by projecting two types of patterns onto an object to be measured, and Figure 3 is a diagram of the slit brightness in three stages. FIG. 4 is a diagram showing an example of a coding grid characterized by changes. FIG. 4 is a diagram showing an example of three types of color coding grids. FIG. FIG. 6 is a basic configuration diagram of an apparatus that projects a lattice pattern onto an object to be measured and images the object to measure its three-dimensional shape. FIG. 7 is an example of a conventional color code lattice and a lattice image. l, lO... Projection device, 2, 20... Imaging device, 1i-... Projection pattern generator, 21a, 21b
. 21c...-image memory, 22.22a, 22b,
22c... Slit detection unit, 23... Code extraction unit, 24-... Decoding unit, 231a, 231b, 23
]c--color code extraction unit. Patent applicant Nippon Telegraph and Telephone Corporation (a) Figure p (b) No. (a) (b) 1st and 2nd order Jt! n 345... 1'2'3'4'N'

Claims (3)

[Claims] (1) means for generating a lattice pattern and projecting the lattice pattern onto a three-dimensional object; means for respectively capturing an image; means for storing the first and second images captured by the image capturing means in a memory; means for detecting a slit image from each of the second images; and a slit image in the second image. means for extracting a code sequence by using the color code of the slit image as a color having the maximum ratio of the color component of the brightness value of the first image to the color component of the brightness value of the position of the slit image in the first image; A three-dimensional shape measuring device comprising a decoding means for measuring the position and shape of a three-dimensional object by decoding and associating it with the projected grid pattern. (2) The coded grid pattern projected by the pattern projection device is composed of a row of slits whose brightness is changed according to the code sequence, and the coded grid pattern is projected onto a three-dimensional object and photographed. A code extraction unit compares the luminance value of the detected slit image with the luminance value of the position of the slit image of the first image onto which a uniform white pattern is projected, By determining the brightness code of the slit image, a code sequence of the first image is extracted, which is decoded and associated with the projection coding grid pattern to measure the position and shape of the three-dimensional object to be measured. The three-dimensional shape measuring device according to claim (1). (3) The encoded lattice patterns are a plurality of encoded lattice patterns configured by sequentially changing the correspondence between the source of the code sequence and the color or brightness of the slit, and the plurality of encoded lattice patterns are is sequentially projected onto a three-dimensional object to be measured, a plurality of sequentially captured image data are stored in an image memory, and the color or brightness code of each slit image is extracted for each of the plurality of image data stored in the image memory. Claims (1) and (2) characterized in that the code of the slit image is determined by majority vote from a plurality of codes each extracted from the slit images located at the same position in the plurality of images. 3D shape measuring device described in ).