JP2002054913A - Three-dimensional data generating system and projector - Google Patents

Abstract

PROBLEM TO BE SOLVED: To generate accurate three-dimensional data by producing proper contrast on two-dimensional images when generating three dimensional data from plural two-dimensional images and by accurately keeping corresponding relations. SOLUTION: This three-dimensional data generating system is equipped with plural image pickup means 25, plural projectors each of which projects pattern light, a diffusion detection part 103 and a luminance detecting part 104 serving as detecting means to detect the condition of the pattern light on an object, a focus control part 101 and a luminous energy control part 102 serving as control means to individually control each projector 26 based on the detected result of the detecting means, and a computing means 105 to generate three-dimensional data DT based on plural image data FD picked up by the image pickup means when the pattern light is projected by the projector 26.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to a system for generating three-dimensional data from a plurality of images obtained by imaging a subject, and to a light projecting device used to project pattern light onto the subject.

[0002]

2. Description of the Related Art Conventionally, three or more images having parallax have been used.
Various systems for generating dimensional data have been proposed.

For example, in a system for obtaining three-dimensional data of a relatively large component, first, the component is placed on a moving stage as a subject. A pseudo texture is projected toward the subject by using an illumination light projecting unit in which a minute point light source is arranged so as to surround the subject. The subject is photographed by a plurality of cameras, and a plurality of
Obtain a two-dimensional image. For a plurality of two-dimensional images, corresponding points are obtained using a correlation method or an optical flow method,
Three-dimensional data is calculated by calculation.

[0004]

A three-dimensional data generation system 80 as shown in FIG. 12 is conceivable. Figure 1
2 is a perspective view showing a configuration example of a three-dimensional data generation system 80, and FIG. 13 is a view for explaining a projection state of pattern light.

In FIG. 12, a three-dimensional data generating system 80 includes three trinocular cameras 81a, b, c, two pattern light projectors 82a, b, a system control box 83 for controlling these, and a system control box 83. It comprises a computer 84 for controlling the box 83.

The three-lens cameras 81a, 81b, and 81c are respectively
It has three cameras arranged such that the imaging plane is located at the vertex position of the triangle. Each of the three-lens cameras 81a, 81b, and 81c is disposed at a predetermined distance in the horizontal direction so as to surround the head HD of a person who is the subject Q, and faces the head HD.

The pattern light projecting devices 82a, 82b are arranged between the three-lens cameras 81a, 81b, 81c, respectively.
An appropriate pattern light is projected toward D. As shown in FIG. 13, the pattern lights PKa, b projected from the pattern light projecting devices 82a, b form patterns PT on the right and left faces of the head HD, respectively. This produces contrast over the entire face.

[0008] The system control box 83
It controls the photographing operation of the eye camera 81, accumulates photographed image data, and transfers it to the computer 84. The computer 84 instructs the three-lens camera 81 to perform a shooting operation via the system control box 83, and instructs the pattern light projecting device 82 to turn on or off. The positional relationship between the three cameras and the three trinocular cameras 81a, b, and c is known in advance, and the computer 84 recognizes them as positional information. The three-dimensional shape (three-dimensional data) of the head HD, which is the subject, is obtained from the positional information and the correspondence between the pixels in each captured image.

In order to accurately determine the correspondence between the images, the images need to have a contrast. Therefore, as described above, the pattern light PKa, b is projected onto the face of the subject by the pattern light projecting device 82 during photographing.

However, since the human face has irregularities and a complicated shape, only a part of the pattern light PKa, b is focused in the above-described pattern light projecting device 82. That is, as shown in FIG. 13, only two points Ta and Tb of the pattern light PKa and b are focused, and the other portions are out of focus.

Further, depending on the color and condition of the surface of the subject, a sufficient contrast may not be obtained. In the above example, the color of the facial skin is significantly different from the color of the hair, so that even if the same amount of pattern light PK is projected, it is suitable for only one of them, and sufficient contrast cannot be obtained for both.

As a result, it is not possible to accurately determine the correspondence between images, and it may not be possible to generate accurate three-dimensional data. SUMMARY OF THE INVENTION The present invention has been made in view of the above-described problems, and in generating three-dimensional data from a plurality of two-dimensional images, an appropriate contrast is generated in the two-dimensional image, and a three-dimensional image with high accuracy for accurate correspondence. The purpose is to generate data.

[0013]

A system according to the present invention comprises a plurality of image pickup means, a plurality of light projecting means each projecting pattern light, a detecting means for detecting a state of pattern light on a subject, Control means for individually controlling each of the light projecting means based on the detection result of the detecting means.

According to a preferred embodiment, there is provided a calculating means for generating three-dimensional data based on a plurality of image data picked up by the image pick-up means when the light projecting means projects the pattern light.

According to a preferred embodiment, the detecting means detects a blur amount of the pattern light on the subject, and the control means controls a focus of each light projecting means based on the detected blur amount. .

The detecting means detects a luminance state of the pattern light on the subject, and the control means controls the light quantity of each light projecting means based on the detected luminance state.

A light projecting device according to the present invention comprises a main body frame, a plurality of light emitters mounted on the main body frame so as to project pattern light toward a subject, and the brightness and focus of each of the light emitters. And a control device that can be individually controlled.

[0018]

FIG. 1 is a perspective view showing the configuration of a three-dimensional data generation system 1 according to the present invention, FIG. 2 is a side view showing the configuration of the three-dimensional data generation system 1, and FIG. FIG. 4 is an enlarged perspective view showing the projector 26, and FIG.

In FIG. 1 and FIG. 2, the three-dimensional data generation system 1 includes a base 10, a photographing unit 11, a system control box 13, a computer 14, and a chair CR.

The photographing unit 11 can be opened and closed like a fan in the vertical direction, and integrates functions of projecting pattern light and photographing with a multi-lens camera. It is attached to a support table SD placed on the base 10 so that the height can be adjusted.

Referring also to FIG. 3, the photographing unit 11
The band base material 21, a plurality of foldable bands that are rotatably mounted on the rotating shafts 22 a, 22 b, 23 a, 23 b attached to both ends of the band base material 21 and the rotating shafts 22 a, b or 23 a, b. .., Three trinocular cameras 25 a, 25 b, 25 c attached to the band base material 21, and a large number of light emitters 26 attached to each band member 24,
26 ... The three-lens cameras 25a, 25b, 2
5c may be referred to as the “three-lens camera 25” as a whole or a part thereof. The same applies to other cases.

The band base 21 is formed in an arc shape using a synthetic resin or a metal material, and has an appropriate rigidity. Each band member 24 is formed in the same arc shape as the band base material 21 using the same synthetic resin or metal material. The many band members 24 are arranged so as to sequentially overlap the back surface of the band base material 21, and can be opened or closed around the rotation shaft 22 or 23. The upper and lower rotation shafts 22a and 23a or 22b and 23b are not engaged with each other and can rotate independently. That is, the upper band member 24 and the lower band member 24 are independently foldable.

The band base member 21 and the band member 24 are black so that light is not reflected. Each three-lens camera 2
5 has the same configuration and function as those conventionally used. That is, each camera has three cameras arranged such that the imaging planes are located at the vertices of a triangle.

Each three-lens camera 25 has a subject, that is, a chair C
As in the case of photographing the head HD of a person sitting on the
Are mounted at a predetermined distance from each other in the horizontal direction on the surface. By arranging the three trinocular cameras 25, occlusion is eliminated.

A plurality of light projectors 26 are mounted in a mounting portion 24a provided on each band member 24 in such a manner as to be arranged in a row in a horizontal direction. By opening each band member 24 in the vertical direction, a large number of projectors 26 are arranged in the horizontal and vertical directions.

As shown in FIG. 4, each of the projectors 26
It comprises a lens barrel 41, a light source 42, a pattern mask 43, a projection optical system 44, and a focus adjustment mechanism 45. As the light source 42, for example, a high-brightness LED is used. The drive current supplied from the system control box 13 flows through the high-brightness LED. By controlling the drive current,
The light emission amount of the high-brightness LED, that is, the light amount of the light source 42 is controlled.

The pattern mask 43 determines the pattern of the pattern light PK projected from the light projector 26. The pattern of the pattern mask 43 is projected onto the head HD as the subject, and in this state, the subject is
Is imaged. It is desirable that the pattern has a change in luminance for each pixel pitch of an image obtained by imaging.

The projection optical system 44 enlarges the pattern light PK. The projection optical system 44 is moved back and forth by the focus adjustment mechanism 45 to adjust the focal length. Focus adjustment mechanism 4
5 is controlled by the system control box 13.

That is, a command from the computer 14 is transmitted to the focus adjustment mechanism 45 via the system control box 13, and the focus adjustment mechanism 45 drives the projection optical system 44 to adjust the focus.

Next, control of the projector 26 when projecting the pattern light PK onto the object will be described. FIG. 5 shows the pattern light PK from the projector 26 to the head HD of the person who is the subject.
6 is a plan view showing a state where the pattern PT is projected, FIG. 6 is a view showing a state of the pattern PT projected on the head HD, FIG. 7 is a view showing a difference in the state of the pattern PT depending on the presence or absence of focusing, and FIG. FIG. 9 is a diagram for explaining a method for obtaining the in-focus position of PK, FIG. 9 is a diagram for explaining a method for adjusting the light amount of the pattern light PK, and FIG. 10 is a diagram for explaining a method for determining the light amount of the pattern light PK.

As shown in FIG. 5, the pattern light PK is projected from each projector 26 toward the face of the head HD.
Each pattern light PK has a small projection angle and a narrow projection range. That is, each of the light projectors 26 projects the pattern light PK on the subject in a spot manner. Therefore, it is easy to focus on the surface of the subject Q, and a clear pattern PT having contrast can be formed on the subject Q.

That is, irrespective of the shape of the subject Q, it is possible to completely focus on at least a portion equal to the number of the light projectors 26. Since the projection range of the pattern light PK is narrow even for a part that is not perfectly focused, the degree of defocus is small. In this regard, in the case described with reference to FIG. 13, the number of perfectly focused points is about two, and the degree of defocus is large because the projection range is wide.

As described above, a clear pattern PT with contrast can be formed on the subject Q.
Even if the shape of the subject Q is complicated or the color or surface state of the subject Q is complicated, it is possible to accurately determine the corresponding points of the obtained image.

FIG. 6A shows an image FD1 of the object Q when the pattern light PK is not projected. FIGS. 6B and 6C show images FD2 and FD3 obtained by photographing the subject Q when the pattern light PK is projected.

In the image FD1 shown in FIG. 6A, the head HD, which is the subject Q, is roughly composed of a hair portion HK, which is a portion of hair including hair and eyelashes, and a skin portion, which is a portion of skin. HF.

Generally, the hair part HK and the skin part HF differ in color and surface condition. Image FD shown in FIG.
In the case of projecting the pattern light PK having an amount of light such that the pattern PT having an appropriate contrast is obtained on the skin portion HF as in 2, the light amount is insufficient at the hair portion HK, and the image is crushed black. No contrast can be obtained.

Conversely, when the pattern light PK having such an amount of light as to obtain a pattern PT having an appropriate contrast with respect to the hair HK is projected as shown in an image FD3 shown in FIG. With HF, the amount of light is excessive, and the image jumps white, and no contrast is obtained.

Therefore, the light amount of the pattern light PK is made different between the hair portion HK and the skin portion HF, and control is performed such that a pattern PT having a contrast appears in each of them. In the present embodiment, the focus and the light amount of each projector 26 are controlled independently.

First, a focus control method will be described.
As described above, the focus of the pattern light PK is adjusted by moving the projection optical system 44 back and forth by the focus adjustment mechanism 45.

First, only the light emitter 26 to be focused is turned on, and the pattern light PK is projected on the subject Q. The focus adjusting mechanism 45 is controlled by a command from the computer 14, and is sequentially moved so that the focus position of the pattern light PK by the projection optical system 44 becomes three different positions. At each focus position, the pattern PT at that time is photographed by the three-lens camera 25. Each focus position (shooting position)
ST1, ST2, and ST3.

At the in-focus position, the size of each image of the pattern PTa becomes minimum as shown in FIG. As the focus position moves away from the focus position, each image of the pattern PTb blurs and spreads as shown in FIG. 7B. By measuring the magnitude of image blur at three points, the in-focus position can be obtained.

In FIG. 8, the horizontal axis indicates the focal length, and the vertical axis indicates the magnitude of image blur. As can be understood from FIG. 8, the in-focus position is obtained from the magnitude of blur at each of the photographing positions ST1, ST2, ST3.

With such a procedure, the focal positions of all the light projectors 26 are adjusted. According to this method, the focus position of the pattern light PK of the light projector 26 can be adjusted only by an imaging system such as the three-lens camera 25 and the control of the projection optical system 44 of the light projector 26 without providing an autofocus mechanism. it can.

Next, a method for controlling the amount of light will be described.
First, only the light emitter 26 whose light quantity is to be adjusted is turned on, and the pattern light PK is projected on the subject Q. In this state, the pattern PT is photographed by the three-lens camera 25. In addition, the subject Q is photographed by the three-lens camera 25 with the light emitter 26 turned off.

Thus, as shown in FIGS. 9A and 9B, an image FD4 when the light projector 26 is turned on and an image FD5 when the light projector 26 is turned off are obtained. From the difference between these two images FD4 and FD5, as shown in FIG. 9C, an image FD6 of a partial area AR of the subject Q including the pattern PT
Cut out.

The contrast of the clipped area AR is checked. That is, the luminance value of each pixel in the area AR is measured. When the luminance is 256 gradations, the luminance value takes a value from 0 to 255. Look at the distribution of pixels for each luminance value.

In FIG. 10, the horizontal axis represents the luminance value, and the vertical axis represents the number of pixels. When the amount of the pattern light PK is insufficient and the image becomes dark and the image is blackened, the distribution of the luminance values is biased to the left (close to 0) as shown in FIG. vice versa,
When the amount of the pattern light PK is too large and jumps white, the distribution of the brightness values is biased to the right (toward 255) as shown in FIG. When the light quantity of the pattern light PK is appropriate and the contrast is sufficient, FIG.
As shown in (1), the distribution of luminance values is not biased to both ends but is distributed in the middle.

Therefore, until the average of the brightness values in the area AR is close to 128, the light is emitted from the projector 26 while changing the light quantity. An optimum amount of light can be obtained by repeatedly obtaining the average of the luminance values.

The system control box 13 controls the three-lens camera 25 and the projector 26 according to instructions from the computer 14. The image captured by the three-lens camera 25 is transferred to the computer 14.

Computer 14, as described above,
Control is performed so that each projector 26 has an optimum focus position and light amount. Further, control is performed such that the subject Q is photographed by the three-lens camera 25 in a state where the pattern PT is projected on the subject Q. Then, based on the plurality of images FD acquired by the three-lens camera 25, three-dimensional data DT is generated by a known method.

FIG. 11 is a block diagram functionally showing the three-dimensional data generation system 1. As shown in FIG. 11, the three-dimensional data generation system 1 includes a focus control unit 101, a light amount control unit 102, a blur amount detection unit 103, a brightness detection unit 104,
It comprises a dimension data calculation unit 105, an interface 106, a three-lens camera 25, and a large number of projectors 26.

The blur detector 103 detects the blur of the pattern light PK on the subject Q based on the image FD captured by the three-lens camera 25. Brightness detector 104
Detects the luminance state of the pattern light PK on the subject Q. The focus control unit 101 controls the focus position of each projector 26 based on the detected blur amount. The light amount control unit 102 controls each of the light projectors 2 based on the detected brightness state.
6 is controlled. The three-dimensional data calculation unit 105 controls the three-lens camera 2 when the projector 26 emits the pattern light PK.
The three-dimensional data DT is generated based on the plurality of images FD captured in Step 5. The details of the operations of these units are as described above.

In the above embodiment, various patterns can be used as the pattern PT by the light projector 26. The number and arrangement of the light projectors 26 may be various other than those described above. Trinocular camera 2 as imaging means
Although 5 is used, a twin-lens camera may be used. To detect the state of the pattern light PK on the subject Q, the image FD taken by the three-lens camera 25 was used.
Instead of the above, imaging means separate from them may be used.

In addition, the structure, shape, dimensions, number, material, control contents or order of the whole or each part of the photographing unit 11, the system control box 13, the computer 14, or the three-dimensional data generating system 1 are as follows.
It can be changed appropriately according to the spirit of the present invention.

[0055]

According to the present invention, when generating three-dimensional data from a plurality of two-dimensional images, an appropriate contrast is generated in the two-dimensional image, and accurate three-dimensional data is generated because the correspondence is accurate. Can be.

[Brief description of the drawings]

FIG. 1 is a perspective view showing a configuration of a three-dimensional data generation system according to the present invention.

FIG. 2 is a side view illustrating a configuration of a three-dimensional data generation system.

FIG. 3 is an enlarged perspective view showing a photographing unit.

FIG. 4 is an enlarged perspective view showing a light projector.

FIG. 5 is a plan view showing a state in which pattern light is projected from a light projector to a subject.

FIG. 6 is a diagram showing a state of a pattern projected on a subject.

FIG. 7 is a diagram illustrating a difference in a state of a pattern depending on the presence or absence of focusing.

FIG. 8 is a diagram illustrating a method of obtaining a focus position of pattern light.

FIG. 9 is a diagram for explaining a method of adjusting the amount of pattern light.

FIG. 10 is a diagram illustrating a method for determining the light amount of pattern light.

FIG. 11 is a block diagram functionally showing the three-dimensional data generation system.

FIG. 12 is a perspective view illustrating a configuration example of a three-dimensional data generation system.

FIG. 13 is a diagram illustrating a projection state of pattern light according to the configuration example of FIG. 12;

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 3D data generation system 11 Photographing unit (light projection device) 13 System control box 14 Computer (control device, control means, calculation means) 21 Band base material (main frame) 24 Band member (main frame) 25 Three-lens camera ( Imaging means) 26 light emitter (light emitting means) 101 focus control unit (control device) 102 light amount control unit (control device) 103 blur amount detection unit 104 luminance detection unit 105 three-dimensional data calculation unit PK pattern light

 ──────────────────────────────────────────────────続 き Continued on the front page F term (reference) 2F065 AA53 BB05 CC16 DD09 FF04 GG07 GG14 HH07 JJ03 JJ05 JJ19 NN02 QQ31 QQ42 5B047 AA07 BA02 BB06 BC04 BC05 BC12 CA17 CA19 CB11 5C061 AA21 AB04 AB24

Claims (5)

[Claims] 1. A plurality of image pickup means, a plurality of light projecting means each projecting pattern light, a detecting means for detecting a state of pattern light on a subject, and a plurality of light projecting means based on a detection result of the detecting means. A control means for individually controlling the light means; and a three-dimensional data generation system comprising: 2. The three-dimensional data generation system according to claim 1, further comprising an arithmetic unit that generates three-dimensional data based on a plurality of image data captured by the imaging unit when the projection unit projects the pattern light. . 3. The method according to claim 1, wherein the detecting unit detects a blur amount of the pattern light on the subject, and the control unit controls a focus of each of the light projecting units based on the detected blur amount. 2. A three-dimensional data generation system according to item 2. 4. The apparatus according to claim 1, wherein the detecting means detects a luminance state of the pattern light on the subject, and the control means controls a light amount of each of the light projecting means based on the detected luminance state. 2. A three-dimensional data generation system according to item 2. 5. A light projecting device for projecting pattern light for photographing onto a subject, comprising: a main body frame; and a plurality of light sources mounted on the main body frame, each projecting the pattern light toward the subject. A light projector comprising: a plurality of light projectors; and a control device capable of individually controlling the luminance and the focus of each of the light projectors.