JPH03192474A - Three-dimensional shape measuring system - Google Patents

Abstract

PURPOSE:To suitably decode even the slit number of a slit image, which is interrupted or missed, as well by dividing a processing into plural steps, gradually decoding the slit image from secure one and decoding the indefinite image by referring the result of the already decoded slit image. CONSTITUTION:A slit picture element extracting means 102, picture element grouping means 103, grouped picture element decoding means 104 and slit number interpolating means 105 are provided. When executing a three-dimensional shape measurement using slit light divided in plural colors, the decoding processing is divided into plural steps even in the case the interruption or miss is generated when extracting the slit image from an observation object in the complicated shape, and the slit images are gradually decoded from reliable ones. Then, the indefinite slit image is decoded by referring the result of the already decoded slit image. Thus, the three-dimensional shape of the observation object can be exactly measured.

Description

[Detailed Description of the Invention] [Summary] This invention relates to a three-dimensional shape measurement method for measuring a three-dimensional shape such as a human face using a planar code method that characterizes a projected pattern using color. In the planar coding method using coded slit light, the aim is to enable decoding of color sequences that are resistant to discontinuities and omissions for slit images extracted from observed images. a slit pixel extracting means for extracting slit pixels and their colors that can constitute a slit pixel; a pixel grouping means for grouping pixels so that the pixels are classified into different groups; and a pixel grouping means for grouping the pixels so that the pixels are classified into different groups, and of each of the classified groups, the number of slit pixels in the group is k or more, and the color of each slit pixel is Detects a group whose arrangement is part of the color arrangement of the slit light on the projection side, and assigns to each slit pixel in the group the slit number assigned to the slit light on the projection side of the same color as the pixel color. By interpolating the slit number based on the grouped pixel decoding means to which the slit number is assigned and the slit pixel to which the slit number has been assigned, the grouped pixel decoding means determines the slit number among the slit pixels extracted from the slit pixel extraction means. slit number interpolation means for assigning a slit number to a slit pixel to which no slit number has been assigned and outputting a final decoded output together with the slit pixels to which a slit number has been assigned by the grouping pixel decoding means. do.

[Industrial Application Field] The present invention is a three-dimensional method for measuring three-dimensional shapes such as human faces using a planar code method that uses colors to characterize projected patterns.
Concerning dimensional shape measurement methods.

[Background Art] In recent years, processing for three-dimensional shapes such as human faces has been actively performed in many fields such as medicine, personal identification, and communications. Along with this, there is an increasing need to easily measure three-dimensional shapes such as human faces.

Conventionally, techniques for measuring three-dimensional shapes are broadly classified into two types: contact type and non-contact type.

Contact measurement methods have long been used to measure the shape of the human body, including the face. However, the contact type method has the following problems.

First, it takes time and effort to convert the obtained three-dimensional data into a format that can be handled by a computer. Second, it is difficult to accurately measure the shape of a soft object such as a human body. Thirdly, it is difficult to collect three-dimensional data densely on the target surface. Fourth, the subject must remain stationary in the pose for a long period of time. Fifth, the measuring device becomes large-scale.

Due to the problems mentioned above, it is difficult to say that the contact method is an easy method for measurement.

For this reason, non-contact methods are now being used.

Conventionally, there are various non-contact three-dimensional shape measurement methods, one of which is a method called a plane code method. This method is described in the document “Sansui, Tamune, Tamura: “Input and processing of distance images”, IEICE Technical Bulletin IE86-
128 (1987-3), J. The planar code method projects a single coded pattern of light onto an object and photographs it, thereby obtaining information equivalent to the information obtained from the direction of the light projection from the observed image of the object. It was made so that it could be obtained. Using this method, it is possible to obtain all the information necessary to calculate a three-dimensional shape from a single observation image. The planar code method has the following characteristics when compared with other non-contact methods.

First, there is no need to photograph an object from different directions and make correspondence between multiple images, unlike in multi-viewing. For this reason,
° There is no need to limit the posture of the object or place marks on the surface of the object. Second, unlike the scanning method using a light beam, there is no need to photograph the object multiple times, so the object does not need to remain stationary for a long period of time. Third, measurements can be performed using simpler equipment than methods that use lasers, ultrasound, etc.

In this way, the planar code method is a relatively simple device that does not place many restrictions on the subject's posture, and is particularly useful for measuring the shape of the human body. It is possible to obtain dimensional data and is a method suitable for measuring three-dimensional shapes such as human faces.

In the plane code method, a correspondence is created between the pattern light on the projection side and the pattern image in the observed image, and information included in the observed image that is equivalent to the information obtained when viewed from the direction of light projection is extracted. There is a need to.

At this time, if the patterned light to be projected is uniform, there will be no distinction between them, making it difficult to automatically perform the matching.

Therefore, a method has been proposed in which pattern light is characterized using colors and information necessary for correspondence is encoded based on the arrangement of colors. This method is described in the literature “Yonezawa.

Ball amount: “°Measurement of object shape using encoded grid”.

IEICE Transactions Volume, J61-D, 6. p
p, 411418 (197B-6), J and rK, L
,Boyer and A.C.Kak:”Co1or
-Encoded 5structured Lig
ht for Rapid Active Range
ng-IEEE Trans, Pattern An
al, &Machine Intel,,PAMI
-9, L, pp. 14-27 (1987-1). Using this method, by extracting a pattern image from an observed image and decoding the arrangement of its colors, it becomes possible to automatically associate the projected pattern light with the observed pattern image.

[Problem to be solved by the invention]

However, in the conventional method of characterizing patterned light using colors as described above, patterned light characterized by color is used to treat objects with complex shapes, highly uneven surfaces, and uneven surface colors, such as human faces. When projected, there is a problem in that the pattern image extracted from the observed image is likely to be interrupted or missing, making it difficult to accurately decode the color arrangement from the pattern image.

This problem cannot be solved simply by adding innovations to the pattern light encoding method or the pattern image extraction method.

The present invention makes it possible to decode a color sequence that is resistant to discontinuities and omissions in a slit image extracted from an observed image in a planar coding method that uses color-coded slit light as a projection pattern. purpose.

[Means for Solving the Problems] The present invention first projects a plurality of slit lights assigned with slit numbers onto an observation target, and from an observation image of the observation target, selects a slit corresponding to a slit image on the image. The premise is a three-dimensional shape measurement method that decodes the light slit number and measures the three-dimensional shape of the observation target based on this. A plurality of slit lights assigned slit numbers are each projected onto an observation target at a unique projection angle within a three-dimensional coordinate system, and
The mutual positional relationship between the camera, the object to be observed, and the slit light that captures the ri1 measurement image is set in advance. If we know which slit number the slit image on the observed image relates to the slit light, we can know at what projection angle that slit image was projected onto the observation target, and based on this and the above-mentioned positional relationship, The coordinates in the three-dimensional coordinate system of the part of the observation target that overlaps the slit image can be uniquely determined.

In this way, by determining the coordinates of multiple locations on the observation target, the three-dimensional shape of the observation target can be measured.

In this case, in order to be able to determine the slit number from the slit image of the observed image, each slit light is characterized by one of q colors (q is a natural number of 2 or more). Also,
Two adjacent slit lights do not have the same color, and the color of each slit light is arranged so that the colors of adjacent slit lights (k is a natural number of 2 or more) appear only once in all slit lights. is set from q colors.

A plurality of parallel slit lights, which are color-coded as described above and each assigned a slit number, are projected onto the observation target so as to be parallel to a predetermined coordinate axis within a predetermined three-dimensional coordinate system, and the state Then, an observation image of the observation object is photographed so that the predetermined coordinate axis is in the vertical direction. By detecting the slit image and its color corresponding to each slit light from the observation image taken in this way, and decoding the slit number from the color arrangement of each slit image, three of the observation targets are detected as described above. Measure dimensional shapes.

FIG. 1 shows a block diagram of the present invention based on the above three-dimensional shape measurement method.

First, the slit pixel extraction means 102 extracts the observed image 101.
The slit pixels that can constitute each slit image and their colors are extracted from the image. Here, the number of color types of the slit light mentioned above q
For example, if the three colors are red, green, and blue, the above-mentioned observation image 101 is three images, a red component image, a green component image, and a blue component image output from a video camera, for example. Then, the above-mentioned slit pixel extraction means 10
2 extracts pixels with maximum brightness values as slit pixels for each row on each image side of the red component image, green component image, and blue component image, and also extracts the pixels in each slit pixel. The color of the slit pixel is extracted as one of red, green, or blue, depending on which image among the red component image, green component image, and blue component image has the maximum value.

Next, the pixel grouping means 103 groups each slit pixel extracted as described above into each row on the observed image 101. In this case, if any two slit pixels are spaced apart from each other by a predetermined distance or more, or if they have the same color, the adjacent slit pixels are classified into different groups. Divide.

Further, the grouped pixel decoding means 104 determines that among each group classified as described above, the number of slit pixels in the group is equal to or more than the above-mentioned number, and the color arrangement of each slit pixel is detects a group that is part of the color sequence of the slit light on the projection side. Then, each slit pixel in the group is assigned the slit number assigned to the projection-side slit light of the same color as the pixel.

In this case, the means is a first method that causes, for example, decoding of the slit number of each slit pixel to cause an inversion of the slit number between the slit pixel and the slit pixel to which the slit number has already been assigned.
Or, if the slit number of each slit pixel is decoded, a second slit image in which slit pixels with different slit numbers coexist in the slit image generated by one slit light is generated.
If any of the following conditions occurs, the slit number of each slit pixel is not decoded.

Furthermore, the slit number interpolation means 105 interpolates slit numbers based on the slit pixels to which slit numbers have been assigned as described above. The decoding means 104 assigns slit numbers to slit pixels to which no slit numbers have been assigned, and the grouped pixel decoding means 104 outputs a final decoded output 106 together with the slit pixels to which slit numbers have been assigned. The means includes, for example, horizontal interpolation means and vertical interpolation means as described below. That is, in each row of the observed image 101, when a slit pixel column to which no slit number has been assigned is sandwiched between slit pixels to which slit numbers have been assigned, the horizontal interpolation means Based on the slit number, refer to the corresponding row of slit lights on the projection side, and for each slit pixel to which no slit number is assigned, assign the slit number assigned to the slit light on the projection side that corresponds to that pixel. Give. After the processing by the horizontal interpolation means, the vertical interpolation means, in the first case, connects 8 slit pixels to which no slit number has been assigned to two slit pixels that have been assigned the same slit number. When the slit pixels are connected by being sandwiched between them, the same slit number is assigned to the sandwiched slit pixels. In the second case, slit pixels to which no slit number has been assigned yet are sandwiched between the slit pixel at the end point to which no slit number has been assigned and the slit pixel to which a slit number has been assigned, and are connected in 8-connection. In this case, the slit number is assigned to the sandwiched slit pixel.

In addition to the above configuration, the following first processing ~
It may be configured to include pre-processing means for executing the fourth process and outputting the remaining slit pixels to the pixel grouping means 103. As a first process, this means deletes the currently focused slit pixel if there is a slit pixel having a different color within 8 neighborhoods of the focused slit pixel. Second
As the processing, if there are three or more other slit pixels within 8 neighborhoods of the slit pixel of interest, the pixel of interest is deleted. As the third process, among the eight neighbors of the slit pixel of interest, the row of the pixel of interest and its first
If there are two or more slit pixels in the pixel included in the row above the pixel, or if there are two or more slit pixels in the pixel included in the row of the pixel of interest and the row one row below, the pixel of interest is delete. Then, as a fourth process, if the length of the slit image formed by connecting 8 slit pixels after the above-mentioned first process to third process is shorter than a predetermined length, the slit All slit pixels constituting the slit image are deleted, assuming that the image is noise.

[For production]

Using color-coded slit light, which is the premise of the present invention 3
In the dimensional shape measurement method, measurement accuracy is greatly influenced by how accurately the slit image is decoded from the observed image. Generally, when observing an object such as a face that has an uneven surface color and a complex shape, interruptions and omissions occur when extracting a slit image, making correct decoding difficult.

Therefore, in the present invention, when decoding the slit number of the slit light corresponding to the slit image, the process is divided into multiple stages as shown in FIG. In contrast, decoding is performed by referring to the results of the slit images that have already been decoded.

That is, the slit pixel extraction means 1 from the observed image 101
After slit pixels that are candidates for forming a slit image are extracted by step 02, first stage decoding is performed by pixel grouping means 103 and grouped pixel decoding means 104. Of these, the pixel grouping means 103 groups the slit pixels separately at locations where there is a high possibility that the slit image is interrupted or missing. Then, the grouped pixel decoding means 104 performs decoding on a unit of slit pixel grouped in this way, so that two slit pixels in which a slit image is interrupted or missing between them are It is unlikely that the slit number will be given (can be, in this case,
If either the first state or the second state described above occurs, decoding is not performed. When a screen is set up behind the observation target, the slit number in the slit image on the observation image is almost never reversed. Further, if the slit image is generated by one slit light beam, the same slit number must be given to all pixels forming the slit image. Therefore, by determining the first and second states described above, it is possible to reduce errors during decoding.

Using the reliable decoding result obtained by the first stage decoding described above, the slit number interpolation means 105 performs the second stage decoding. Here, a slit number can be appropriately assigned to a slit pixel that has not been assigned a slit number by logical interpolation processing using the first-stage decoding result. In this case, in particular, by performing horizontal interpolation processing using the horizontal interpolation means, it is possible to appropriately interpolate slit pixels of another slit image sandwiched between slit pixels included in different slit images. Further, by performing vertical interpolation processing by the vertical interpolation means, interpolation processing of slit pixels within one slit image can be appropriately performed.

As mentioned above, by performing decoding in multiple stages,
It becomes possible to appropriately decode slit numbers of slit images that are interrupted or missing.

Further, in the present invention, the pixel grouping means 103 in FIG.
By providing the above-mentioned pre-processing means before the processing, errors in the subsequent decoding process can be further reduced.

〔Example〕

Embodiments of the present invention will be described below with reference to the drawings.

(Day 3 1) First, the configuration of an embodiment of the present invention will be explained.

FIG. 2 is a diagram showing the basic measuring device of the three-dimensional measuring system.

In the figure, the light emitted from the light source 201 is 203
1-203. Slit plate 2 with 8 color slits
02, it is converted into S types of color-coded slit lights 204I to 204°. Here, as the light source 201, a light source of an 0HP (overhead projector) device is used, and as the slit plate 202, an OH
A P sheet with eight colored slits 2041 to 2045 drawn at equal intervals is used. Color coded slit light 204. ~204. is projected onto a target 205 located, for example, approximately 1 meter away.

The object 205 on which the slit lights 2041 to 2045 are projected in this way is the image input device 20 which is a video camera.
Photographed at 6. As a result, a slit image of the object 205 projected by the slit light 204 is formed on the camera frame (imaging position) of the image input device 206, as conceptually shown in 207 of the same figure. become. As the image input device 206, a color ITV camera with 512 x 480 pixels and 8 bits per pixel (256 gradations) is used.
The device is installed, for example, at a position approximately 1 m away from the target 205.

Here, the distance Md (described later) between the focal points of the light source 201 and the image input device 206 is approximately 0.5 m, and the optical axis of the light source 201 and the optical axis of the image input device 206 are arranged so that they form an angle of approximately 30'. Set up Further, a matte gray screen is placed behind the object 205 at a position approximately 1.2 m from the light source 201 and the image input device 206. Further, the object 205 is placed immediately in front of the screen so that when the object 205 is photographed by the image input device 206, the object 205 does not protrude from the photographing screen.

Next, in this embodiment, slit image decoding processing is performed on the observation image taken by the image input device 206. The configuration of the decoding processing section for this purpose is shown in FIG.

In the decoding processing unit, a slit candidate pixel extraction unit 302, a preprocessing unit 303, a pixel grouping unit 304, a grouped pixel decoding unit 305, a horizontal Processing is performed in the order of the directional interpolation unit 306 and the vertical interpolation unit 307 to obtain a final decoded output 308 in which each extracted slit image is associated with a slit number. These detailed operations will be described later.

(3'3゛M beach) In the example of the three-dimensional shape measurement system with the above configuration,
First, the basic principle when measuring a three-dimensional shape will be explained below. In the following description, the light source 201 in FIG. 2 is simply referred to as a light source, the image input device 206 is simply referred to as a camera, and the slit light 204 . ~204s will simply be called slit light.

First, S types of slit light are emitted from the slit plate 202, but now each color slit 203 on the slit plate 202
A series of slit numbers are assigned to 1 to 203s, and each slit light is therefore uniquely determined accordingly.

Next, FIG. 4 shows a coordinate system used in the three-dimensional shape measuring system shown in FIG. 2. As shown in the figure, the origin 0 is set as the focus of the light source, and the focus of the camera is placed on the X-axis.

It is also assumed that S types of slit lights are projected onto an arbitrary point P on the object 205 so as to be substantially parallel to the y-axis.

Now, any point P on the target 205 onto which the slit light is projected
The coordinates P (xo, yo, Zo) of
) to (3).

xo=zocot θ1(1) yo=J XOd +zo ta
n θt (2) where, d = distance between the light source and the focal point of the camera θ, = angle between the X-axis and the optical axis of the camera θt = angle between the x-z plane and the optical axis of the camera θI'' This is the angle formed by the slit light.The specific value of the distance d is as described above, and if the characteristics of the light source and camera are measured in advance, it is possible to θ4
can be found. Furthermore, in the slit images extracted from the observed image, if the slit number of the slit image overlapping the point P can be determined, θ1 can be determined. This is because if the slit number can be determined, the slit light
This is because it is possible to know which slit on the slit plate 202 in the figure generated the light, and 61 in FIG. 3 can be uniquely determined from the positional relationship between the light source 201 in FIG. 2 and the corresponding slit on the slit plate 202.

As described above, a plurality of appropriate points P on the target 205 are
If the position of the object 205 can be determined, the position of the object 205 on the three-dimensional xyz coordinates can be determined.
The three-dimensional shape of 5 can be measured.

As can be seen from the above principle, the measurement accuracy of point P is as follows:
Determined by two factors. That is, the first factor is
The second factor is the measurement accuracy of the light source/camera characteristics, and the second factor is the decoding accuracy of the slit image extracted from the observed image.

Comparing the above two factors, the influence of the decoding error of the slit image extracted from the observed image is much larger than the influence of the measurement accuracy of the light source and camera characteristics.

The present invention particularly relates to a slit image decoding processing section that can accurately decode the slit number of a slit image, as will be described in detail later.

Here, in this embodiment, as a projection pattern onto the target 205 in FIG.
04. ~204s is used. This slit light is transmitted through two to two S type slits 203 on the slit plate 202 in FIG.
Generated by 03s.

The overall color scheme of the color-encoded slit lights 2041 to 204s is determined so that the slit numbers can be uniquely determined based on the way the colors of books are arranged in adjacent slit lights. Slit lights 2041 to 204 using q colors. If we characterize this, the slit light colored in this way is
It can be considered as a code sequence in which code length k is encoded using q elements. Slit light 204 used in this example
1-204. shall be encoded according to the following four conditions (1) to (2). This condition is based on the document ``Yamamoto, Tamo 2 Tamura: ``Input and processing of distance images''.

Institute of Electronics, Information and Communication Engineers Technical Bulletin IE86-128 (1987
-3), based on the description of J.

■Slit light 2041-204. is characterized using q colors.

01 slit light is characterized by one color.

■Adjacent slit lights do not have the same color.

■The arrangement of colors caused by adjacent book slit lights appears only once in all slit lights.

When encoding is performed according to the conditions of ■ to ■ above, the maximum length of the generated code sequence, that is, the total number S of slit lights, is: 5=qX, (q-1)k-' +)c-1 (4) becomes. Under ideal conditions, when the slit lights 2041 to 204s encoded in this way are projected onto an object, adjacent book slit images are extracted from the observed image, and by examining their color scheme, it is possible to identify the book slit images between them. The slit number of the image can be determined. However, in reality, the slit images extracted from the observed images often have breaks or omissions, and even if k color schemes are examined, the slit numbers cannot be uniquely determined or cannot be correctly correlated.

By increasing the number of colors 9 used for characterization, it becomes possible to decode the slit number with a smaller number of colors, and the slit image becomes discontinuous.
Less susceptible to omissions. However, it becomes difficult to distinguish the color of the slit image. In particular, when an object whose surface color is not uniform, such as a human face, is targeted, discrimination becomes more difficult because it is affected by the color of the object itself.

It is also conceivable to divide one slit light in the length direction and further encode it using q colors. However, when targeting an object with a complex shape, the image of one slit light may be interrupted or missing, so it may be observed as multiple slit images in the observation image, or the image of multiple slit light may be observed as multiple slit images. There is a possibility that it will be observed as a single slit image. In such a case, it is easier to associate the slit image with the slit light if each slit light is characterized by one color.

When the code lengths are equal, if adjacent slit lights are characterized by different colors, the maximum length of the generated code sequence will be shorter than if the same color is allowed. However, since it becomes easier to distinguish between adjacent slit lights, it is possible to increase the density when projecting the slit lights onto a target.

Considering the above, in this example, red (R), green (G
) ・Slit 20 encoded using three colors of blue (B)
31 to 2035 are drawn on the slit plate 202 of the OHP sheet, and the slit light 204. ~204, are characterized. Furthermore, the slits 203+ to 203. is encoded with code length = 6, and from equation (4) above, the total number of slits S is 3X
2'-'+6-1=101 pieces. FIG. 5 shows an example of color-coded slits.

FIG. 13(a) shows an example of an observation image captured by the image input device 206 of FIG. 2 by projecting color-encoded slit light.

(Decoding Processing Unit (2) Operation) Next, the operation of the decoding processing unit that is particularly relevant to the present invention will be described.

In the decoding processing unit, the color-encoded slit light 20 shown in FIG.
41-204. is projected onto the object 205, and the slit image generated by the slit light is extracted from the observation image 301 captured by the image input device 206, and the slit number is decoded. One observation image 301 output from the image input device 206 is composed of three pieces of image data corresponding to red, green, and blue components. This is the standard output of a typical video camera. After that, each image data is converted to R-G/B.
I will abbreviate it as Also, the R-G-B coordinate system is the 13th
As shown in the example of FIG. 2, the slit lights 204I to 204. is projected to be approximately parallel to the V axis.

Observation image 301 consisting of the above RGB image data of slit 302
is first input to the slit candidate pixel extraction unit 302, where the slit candidate pixels are extracted.

For this purpose, each R-G-B image data is
The pixels are scanned line by line in the U-axis direction (in the example shown in FIG. 13), and a pixel with a maximum brightness value is searched for in each line. Here, a pixel with a maximum value refers to a pixel that has a brightness value greater than the brightness values of the pixels before and after it in the row direction. As mentioned above, the slit lights 2041 to 204. is projected substantially parallel to the V axis, so each row is substantially perpendicular to the slit image generated based on the slit light.

Therefore, the areas corresponding to the slit images become brighter (pixel brightness values become larger), and the areas corresponding between the slit images become darker (pixel brightness values become smaller).
A pixel with a maximum brightness value is considered to be a candidate pixel constituting a slit image. The pixel having the maximum value extracted here will be called a slit candidate pixel. It is assumed that one slit image is composed of eight slit candidate pixels connected to each other.

Each slit candidate pixel is given corresponding color information among RGB, depending on which image data among RGB has the maximum value. If the slit candidate pixel is the maximum among a plurality of image data, the color of the image data with the maximum brightness value is set as the color of the slit candidate pixel.

Φ of IyL 1.1303 Next, the slit candidate pixels extracted as described above are input to the preprocessing unit 303, and in order to easily and accurately decode the slit image to be performed later, the following is performed. Pre-processing as shown in Process 1 to Process 4 is performed in order,
Slit images generated by different slit lights are prevented from being connected to each other in 8-connection. First, a thrift candidate pixel to be currently processed is set as <po as shown in FIG. 6, and a number P+=Ps shown in the figure is assigned to each of the eight pixels in the vicinity of that pixel.

*Processing 1... slit candidate pixel P. Within 8 neighborhoods of (
P+-Pa), if there is a slit candidate pixel to which color information different from P is given, the pixel is P.

Delete.

One ray of Suri-nod light is characterized by one color.

Therefore, when a single slit image includes pixels to which different color information is added, this slit image is considered to be a combination of images generated by different slit lights. Therefore, by the above-mentioned process 1, one slit image composed of multiple colors is divided into multiple slit images,
The pixels forming each pickpocket image have the same color.

*Processing 2... slit candidate pixel P. If there are three or more other slit candidate pixels within the 8 neighborhood of pixel P. Delete.

*Processing 3...Pa is the slit candidate pixel, P1 to P5
There are two or more slit candidate pixels within, or P and P5 to P8
If there are two or more slit candidate pixels within the pixel P0, the pixel P0 is deleted.

It is a very special case that slit images generated by the same slit light beam intersect or diverge, and it can be said that it is extremely rare. Therefore, it is considered that slit images generated by different slit lights are connected at locations where intersections and branches occur. Therefore, by performing Process 2 and Process 3 described above, pixels at locations where the slit images intersect or diverge are removed, and one slit image is divided into a plurality of slit images.

After performing Processes 1 to 3 described above, Process 4 is performed in order to remove slit candidate pixels that are erroneously detected due to noise superimposed on the observed image 301 due to some reason.

*Process 4: Check the length of each slit image (the number of slit candidate pixels passed from one end point to the other end point), and if it is shorter than a certain value, it is considered to be noise, and the slit image is All slit candidate pixels forming the image are deleted.

After the above processing is completed, a slit image made up of the remaining slit candidate pixels is output as the final extraction result. The extraction results output in this way satisfy the following conditions (1) to (2).

The pixels constituting the 01 slit image are connected to each other in 8-connection.

The thickness of the 01 slit image is 1 pixel.

(2) Pixels forming one slit image have the same color information. This is based on Process 1.

■Each slit image with different color information is separated by one or more pixels. This is also based on process 1.

■Slit images do not intersect or diverge. This is based on Process 2 and Process 3.

■The end point of the slit image is located at the top (the value of the V coordinate is the minimum) or at the bottom (the value of the V coordinate is the maximum) among the pixels that make up the slit image. , according to process 3.

As a result of the above processing in the preprocessing unit 303, as shown in FIG. 13(a).
An example of a slit image (image based on slit candidate pixels) extracted from the observed image 301 illustrated in FIG. 13(b)
Shown below. In this example, in the process 4 described above, the length is 1.
Slit images of 5 pixels or less are deleted. Also, in the example in the same figure, the slit light is projected from the right side, so
There is a shadow on the left side of the face and the left side of the nose, and the slit is missing. Also, the area under the nose and around the lips is particularly uneven, so the slit is interrupted.

′ no-・na. As described above, the slit candidate pixels extracted by the preprocessing unit 303 are processed by the pixel grouping unit 304, grouped pixel decoding unit 305, horizontal interpolation unit 306, and vertical interpolation unit 307 in FIG. Processing is performed sequentially, and decoding processing is executed. Here, an image in which each slit image is expressed by slit candidate pixels is called a slit image.

Here, before explaining the specific decoding process, we will explain the general problems when decoding the slit number of each slit image from the slit image, and the third
The relationship with each processing unit in the figure will be mentioned.

Under ideal conditions, (basic principle of 3D shape measurement)
As described above, the slit number to be assigned can be determined by extracting adjacent slit images and examining the color information assigned to the slit candidate pixels composing them. However, if the following situation occurs in the extracted slit image, simply checking the color scheme of the slit image may not be able to uniquely determine the slit number to be assigned, or the wrong number may be assigned. Become.

*Situation 1: Inversion of the slit image occurs.

*Situation 2: Slit images are interrupted or missing.

*Situation 3: Slit images generated by different slit lights are connected to each other in 8-connection, and are extracted as one slit image.

FIG. 7 shows examples of each of the above-mentioned situations. In the figure, each square represents a pixel constituting a slit image, and the number inside the square is a slit number given to each slit candidate pixel.

Situation 1 described above is development in which the order of the slits is reversed between the light source 201 side and the image input device 206 side, as shown in FIG. 7(a) or FIG. 8(a). This problem can be significantly prevented by shortening the distance between the object 205 and the screen behind it, as shown in FIG. 7(b).

Therefore, in the observed image 301 (FIG. 3) assumed in this embodiment, decoding is performed on the assumption that no inversion of the slit image has occurred.

Further, in the above-mentioned situation 2, the light source 201 and the image input device 20
6, there is an invisible area on the target 205,
When extracting the slit image from the observation image 301, there may be interruptions.
Occurs due to omissions.

As a result, the slit numbers of adjacent slit images become discontinuous, or multiple slit images generated by one slit light beam are generated, as shown in Figure 7(b) (broken) and (C) (missing). It is extracted as a slit image.

It is not possible to prevent this situation from occurring simply by improving the processing before decryption. This problem can be mainly addressed by the pixel grouping unit 304, which will be described later.

Furthermore, the above-mentioned situation 3 occurs when the object 205 has severe irregularities, and as shown in FIG. 7(d), the slit number to be given differs depending on the pixel even if the pixels constitute the same slit image. . In the process 1 in the preprocessing unit 303 described above, when the slit images generated by a plurality of slit lights characterized by different colors are connected to each other in an 8-connection, they are finally converted into one slit image. It is never extracted. However, when 8 slit images generated by different slit lights characterized by the same color are connected, processing 1 cannot deal with the problem, and the above-mentioned phenomenon occurs. This problem can be mainly dealt with by the grouped pixel decoding unit 305.

In the decoding process below the pixel grouping unit 304 in FIG. 3, the process is divided into a plurality of stages, as can be seen from the figure. That is, the slit candidate pixels are numbered in order of certainty, and for pixels whose numbers have not been determined, they are sequentially determined by referring to the numbers of neighboring slit candidate pixels. This is a major feature of this embodiment.

First of the group 304, the pixel grouping unit 304 processes the output of the preprocessing unit 303 in FIG. 3, that is, the slit candidate pixels constituting the slit image. In the following description, this pixel will be referred to as a slit pixel. In a slit image, if the interval between adjacent slit pixels is large, it is considered that there is a high possibility that the slit image is interrupted or missing between them. Furthermore, since adjacent slit lights do not have the same color, even if two adjacent slit pixels have the same color, there is a high possibility that a break or omission will occur between them. If this happens, it falls under situation 2 above.

Therefore, the pixel grouping unit 304 scans the slit image horizontally line by line and groups adjacent slit pixels. However, if the interval between two adjacent slit pixels (P□pb) is larger than a predetermined darkness value Dt, or P,,P.

If the color information given to is the same, P-,
Classify Pb into different groups. It is assumed that the darkness value Di is the average interval between adjacent slit pixels in the entire slit image.

Groups are separated at places where there is a high possibility of interruptions or omissions, so by using grouped slit pixels as the processing unit for decoding, it is possible to avoid interruptions or omissions in the slit image between them. For two pixels,
It is possible to reduce the possibility that consecutive slit numbers will be given.

FIG. 9 shows an example of grouping of pixels in the pixel grouping unit 304. In the same figure, Dt""2 and R-
It is assumed that G-B represents a color determined by color information given to a slit pixel, and G1-G5 represents a group.

Since the spacing between the slit pixels between G2 and Ge1 is larger than that of the slit pixels, the slit pixels are divided into two groups. G4=0
5, adjacent slit pixels have the same color, so 2
divided into two groups.

The slit image in which each slit pixel is grouped as described above in the group "' 305 is input to the grouped pixel decoding unit 305,
Here, the grouped slit pixels are decoded.

First, record the groups in which the number of slit pixels in each group is greater than or equal to the code length (see equation (4), etc. above), and the arrangement of colors of the slit pixels is part of the code sequence on the slit light projection side. put.

Then, decoding is performed in order from the groups containing the largest number of slit pixels, and a corresponding slit number is given to the slit pixels included in each group. For example, for a slit pixel array having a color arrangement as shown in FIG. Numbers are assigned as shown in Figure (a).

However, if the following two conditions occur, the group will not be decoded.

*State 1: When decoding is performed, the slit number is reversed with respect to the slit pixel to which the slit number has already been assigned.

*State 2: When decoding is performed, slit pixels with different slit numbers coexist within the slit image generated by one slit light.

As described above, in the observation image 301 used in this embodiment, there is almost no inversion of the slit image.

Also, if the slit image is generated by one slit light, for all pixels that make up the slit image,
The same slit number must be given. Therefore, if a certain decoding causes state 1 or 2 as described above, that decoding is incorrect.

In order to investigate whether state 2 has occurred, the following test is performed to determine whether the slit image is generated by a single slit beam or by a plurality of slit beams.

When slit images generated by multiple slit lights are connected to each other in an 8-connected manner, it is very rare that the same condition occurs in the slit images on both sides, and in most cases, the slit images are interrupted or missing. is occurring. Now, suppose that there are three adjacent slit images S□S, , Se, and the slit pixels included in each are Pa (us, vb )
, Pb (ub, Vb).

Let it be Pc (uc, Vc). If there exists a quantity V,, Vd, and V,<v,, vb+VC<vd, then P
,,Pb,PC satisfy the following conditions (■) and (■), it is assumed that there is no discontinuity or omission in the slit image Sb and the slit images S, , Sc on both sides of it within this range, and Sb This part of (V,<v,.

<V, ) is determined to be generated by one slit beam.

■u @ < ub < Llc and ub (1m+u
cu ub<D holds true. However, v, = vb = v
c=v (same value). Further, Di is the darkness value used in the pixel grouping unit 304.

■When v, = vb = vc = v (same value), P.

There are no other slit pixels between Pb and between Pb and Pe.

When the decoding of the grouped slit pixels is completed in the above manner, the slit number of the pixel determined by this process is propagated to the upper and lower slit pixels along the slit image, unless the above-mentioned state 2 occurs. I'll let you do it.

FIG. 13(C) shows an example in which grouped pixels are decoded. Further, FIG. 2D shows an example in which the slit number is propagated to the upper and lower slit pixels (in the vertical direction) along the slit image. In the figure, pixels for which slit numbers have been determined are shown in black.

The above decoded output of 306 is further input to the horizontal interpolation unit 306 . As can be seen by comparing FIG. 13(1)) with FIG. 4(d) and FIG. 4(c), even if the grouped slit pixels are only decoded, there are still many unnumbered slit pixels. Therefore, the horizontal interpolation unit 306 performs interpolation of slit numbers in the horizontal direction using adjacent slit pixels that have not yet been assigned a number as units.

As shown in Figure 11(a), when a slit pixel row whose number has not yet been determined is sandwiched between pixels whose numbers have already been determined, the color difference is compared with the slit pattern row on the projection side in Figure 11(g)). It is assumed that the slit pixel number can be uniquely determined even if the arrangement is consistent with a part of the pattern row on the projection side.

FIG. 13(e) shows an example of the processing results in the horizontal interpolation section 306. In the figure, pixels for which slit numbers have been determined are shown in black.

Note that once the horizontal interpolation is completed, the slit number is propagated in the vertical direction, as in the case of the grouped pixel decoding unit 305.

Dropopia n Day, 307-111 The slit pixels that have passed through the horizontal interpolation unit 306 described above are:
Finally, it is input to the vertical interpolation section 307.

Here, slit pixels that are not given slit numbers in the processing up to the horizontal interpolation unit 306 are numbered according to the following procedure.

*If a slit pixel whose slit number has not been determined is sandwiched between two slit pixels waiting for the same slit number N, give the slit number N5 to the undetermined slit pixel in between.

*If a slit pixel whose slit number has not been determined is sandwiched between an end point whose number has not been determined and a slit pixel whose slit number has already been determined, give the slit number N to the undetermined slit pixel in between. .

FIG. 12 shows the operation of the above processing. In the same figure,
N1 to Nc represent the slit numbers of slit pixels determined by the processing up to the horizontal interpolation unit 306, and Pa
-PC represents a slit pixel whose slit number has not yet been determined. Through the above-described processing, slit numbers N and -N are given to pixels P3 to PC, respectively.

Figure 13 shows an example of the decoded output 308 (Figure 3) obtained by performing all the processing up to the vertical interpolation unit 307.
Shown below. In the figure, pixels for which slit numbers have been determined are shown in black. As can be seen from the same figure, Figure 13 (
Pre-processing section 303 (Fig. 3) shown in black part in b)
For the slit image obtained in , there are very few pixels for which the slit number was not determined until the end, which shows that the decoding was performed appropriately.

(i `` of 3 ゛ - ta '') Through the processing in the decoding processing unit shown in FIG. Obtainable.

Based on this, as mentioned above in the section of (3゛-)8渭, The three-dimensional coordinates (Xo
, yo, Zo) as mentioned above (1)2~(3)
It can be calculated from the formula.

3 calculated from the example of the decoded output 308 in FIG. 13(f)
FIG. 14 shows a three-dimensional shape obtained by rotating the dimensional coordinates.

In this case, the three-dimensional coordinates of the part where the thrift image is extracted cannot be determined from the observed image 301 in FIG.
In the example of Fig. 14, the obtained three-dimensional coordinates are B −5pl
It is complemented by the surface of ine.

This interpolation technique is described in the literature rW, J. Gordon
andR,F, Riesenfeld:'B-5pl
ine Curves and 5 surfacesin
Computer Aided Geometry
c Design, eds, R.

E, Barnhill and R, F, Ries
enfeld, pp, 95-126°Academ
ic Press, New York (1974),
It is disclosed in J.

(Effects of the Invention) According to the present invention, in a three-dimensional shape measurement method using color-coded slit light, even if breaks or omissions occur when extracting a slit image from an observation object with a complex shape,
By dividing the decoding process into multiple stages, gradually decoding things that are certain, and decoding uncertain things by referring to the results of the slit images that have already been decoded, it is possible to It becomes possible to accurately measure the three-dimensional shape of

As a result, automatic measurement can be achieved using a simple device without forcing the subject into special situations, and the three-dimensional shape of the face can be measured more easily than with conventional methods.

As a specific effect, the slit pixels are grouped separately in advance by the pixel grouping means at the places where there is a high possibility of interruptions or omissions in the slit image, and then the grouped pixel decoding means decodes the slit pixels. It is possible to reduce the possibility that consecutive slit numbers will be assigned to two slit pixels whose slit images are interrupted or missing.

Further, when decoding is performed by the grouped pixel decoding means, it is possible to reduce errors during decoding by logically determining a state that is unlikely to occur and not performing decoding.

Furthermore, by performing logical interpolation processing using the decoding results of the grouped pixel decoding means by the slit number interpolation means, it is possible to appropriately assign slit numbers to slit pixels to which no slit numbers have been assigned. . In this case, in particular, by performing horizontal interpolation processing using the horizontal interpolation means, it is possible to appropriately interpolate slit pixels of another slit image sandwiched between slit pixels included in different slit images. Further, by performing vertical interpolation processing by the vertical interpolation means, it becomes possible to appropriately perform interpolation processing of slit pixels within one slit image.

In addition to the above effects, by providing a preprocessing means before the processing by the pixel grouping means, it is possible to further reduce errors in the subsequent decoding process.

[Brief explanation of drawings]

FIG. 1 is a block diagram of the present invention. FIG. 2 is a configuration diagram of a three-dimensional measurement system based on this embodiment. FIG. 3 is a configuration diagram of a decoding processing unit in this embodiment. is an explanatory diagram of the coordinate system of the three-dimensional measurement system; FIG. 5 is a diagram showing an example of a color-coded slit; and FIG. 6 is a diagram of a pixel P. Figure 7 is an explanatory diagram of a situation where the slit image is decoded incorrectly. Figures 8 (a) and (b) are diagrams showing the relationship between the projected slit light and the detected slit image. , Fig. 9 is an explanatory diagram of pixel grouping, Fig. 1O is an explanatory diagram of decoding processing of grouped pixels, Fig. 11 is an explanatory diagram of slit number interpolation processing (horizontal direction), Fig. 12 is an explanatory diagram of slit number interpolation processing (vertical direction), Figure 13 is a diagram showing observed images, slit images, and decoding, and Figure 14 is an example of measurement results of three-dimensional shapes. It is a diagram. 101...Observation image, 102 103 04 05 06 slit pixel extraction means, pixel grouping means, grouped pixel decoding means, slit number interpolation means, decoding output.

Claims (1)

[Claims] 1) Let q and k be predetermined natural numbers of 2 or more, each slit light is characterized by one of q colors, and two adjacent slit lights are not the same color, but the adjacent slit lights are The arrangement of colors from k slit lights appears only once in all slit lights, and each slit light is assigned a slit number, and multiple parallel slit lights are arranged on a predetermined coordinate axis in a predetermined three-dimensional coordinate system. A slit image and its color corresponding to each of the slit lights are obtained from an observation image (101) which is projected onto an observation object so that the light is parallel to In a three-dimensional shape measurement method that measures the three-dimensional shape of the observation target by detecting the color sequence of each slit image and decoding the slit number from the color arrangement of each slit image, slit pixel extraction means (102) for extracting slit pixels that can be configured and their colors;
), pixel grouping means (
103) Among the classified groups, the number of slit pixels in the group is k or more, and the color arrangement of each slit pixel is a part of the color arrangement of the slit light on the projection side. Grouped pixel decoding means (
104) and the grouped pixel decoding means (104) among the slit pixels extracted from the slit pixel extraction means (102) by interpolating the slit number based on the slit pixel to which the slit number has been assigned. A slit number is assigned to a slit pixel to which a slit number has not been assigned by
A three-dimensional shape measuring method characterized by comprising: a slit number interpolation means (105) that outputs a final decoded output (106) together with the slit pixels to which the slit numbers have been assigned according to 04). 2) The grouped pixel decoding means is configured to perform a first state in which decoding the slit number of each slit pixel results in an inversion of the slit number between the slit pixels to which a slit number has already been assigned, or When decoding the slit number of a slit pixel, if one of the second states occurs in which slit pixels with different slit numbers coexist in the slit image created by one slit light,
A three-dimensional shape measuring device characterized in that the slit number of each slit pixel is not decoded. 3) The slit number interpolation means performs the following: in each row of the observation image, when a slit pixel column to which the slit number has not yet been assigned is sandwiched between slit pixels to which the slit number has been assigned, Based on the slit number of the slit pixel, refer to the corresponding one of the array of slit lights on the projection side, and apply the slit light on the projection side corresponding to each slit pixel to which the slit number is not assigned. a horizontal interpolation means for assigning a slit number, and after processing by the horizontal interpolation means, in a first case,
When a slit pixel to which the slit number has not yet been assigned is sandwiched and connected to two slit pixels that have been assigned the same slit number in a 8-connection manner, the sandwiched slit pixel is assigned the same slit number. As a second case, the slit pixel to which the slit number has not yet been assigned is sandwiched between the slit pixel at the end point to which the slit number has not yet been assigned and the slit pixel to which the slit number has been assigned. 3. The three-dimensional shape measuring method according to claim 1, further comprising vertical interpolation means for assigning the slit number to the sandwiched slit pixels when the slit pixels are connected in an 8-connected manner. 4) The color types q are three colors: red, green, and blue, and the observed images are three images output from a video camera: a red component image, a green component image, and a blue component image; The slit pixel extraction means extracts, as the slit pixel, a pixel having a maximum brightness value for each row on each image, for each of the red component image, green component image, and blue component image, and The color of the slit pixel is extracted as one of red, green, or blue, depending on which image of the red component image, green component image, or blue component image the pixel becomes maximum. The three-dimensional shape measuring device according to any one of claims 1 to 3, characterized in that: 5) With respect to the slit pixel extracted by the slit pixel extracting means, if there is a slit pixel having a different color within 8 neighborhoods of the slit pixel of interest, a first step of deleting the pixel of interest; a second process of deleting the pixel of interest if there are three or more other slit pixels within the 8 neighborhoods of the slit pixel of interest; Among them, if there are two or more slit pixels in the pixels included in the row of the pixel of interest and the row one row above it, or if there are slit pixels in the pixels included in the row of the pixel of interest and the row one row below it. a third process of deleting the pixel of interest if there are two or more; and after the first to third processes, a slit image formed by connecting 8 of the slit pixels; If the length is shorter than a predetermined length, the slit image is considered to be noise and all the slit pixels constituting the slit image are deleted. 5. The three-dimensional shape measuring device according to claim 1, further comprising pre-processing means for outputting to the converting means.