JP2002300603A - Image pickup device for three-dimensional input - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable an image pickup device which is capable of making a part of the image in a shot optimal in resolution independently of the shot distance and size of a subject. SOLUTION: An image pickup device picks up the image of a subject as it is automatically regulated in image formation magnification so as to make a part of the image corresponding to the subject in a shot proper in size.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a photographing apparatus used for three-dimensional input for converting a shape into data based on a photographed image of an object.

[0002]

2. Description of the Related Art An optical three-dimensional digitizer for measuring a three-dimensional shape of an object in a non-contact manner is used for data input to a CG system or a CAD system, body measurement, and the like. Internet users are also attracting attention. In modeling the entire circumference of the object, three-dimensional inputs are made at least for the front and back surfaces, and two shape data (three-dimensional model) are integrated. If the object is not the entire circumference but is larger than the maximum measurable range, the three-dimensional measurement must be performed a plurality of times.

[0003] As a measuring method for performing photographing, a light section method,
There are a pattern coding method, a stereo vision method, and the like. In particular, in the stereoscopic method, since the shape is obtained based on a plurality of images obtained in a momentary photographing, the measurement can be performed with the same ease as photographing.

In the stereo viewing method, a corresponding point search for finding a pair of pixels corresponding to the same part in a subject is performed on a plurality of captured images having parallax. The shape model can be obtained by calculating the distance by applying the principle of triangulation to the corresponding points. The number of vertices of the model is determined by the number of corresponding points, and does not exceed the resolution (number of pixels) of the captured image.
The more correspondence points, the more faithfully the object can be modeled.

[0005]

Conventionally, the resolution of a subject portion in a photographed image, that is, the density of sampling positions on the subject changes each time a photograph is taken, and the sophistication of a shape model varies with each photograph. There was a problem. For example, in the modeling of the entire circumference, the photographing distance was not constant, so that an uneven model such as a precise front surface and a slightly rough rear surface was sometimes obtained. Also, even when the shooting distance is constant, in the modeling of the face of a person, since the number of pixels of the face in the shot image is smaller in a person with a small face than in a person with a large face, a low-pass filter processing is performed. There was a problem that it became a blurred model. In particular, if an appropriate amount of corresponding points is not obtained in a region having a high spatial frequency, such as the eyes, facial features cannot be represented.

SUMMARY OF THE INVENTION An object of the present invention is to equalize the resolution of a subject portion in a photographed image when performing photographing a plurality of times. Another object is to optimize the resolution of a subject portion in a captured image regardless of the shooting distance and the size of the subject.

[0007]

According to the present invention, a photographing is performed by automatically adjusting an image forming magnification so that a portion corresponding to an object in a photographed image becomes appropriate in size. If an appropriate value is set for each shooting based on the conditions of the previous shooting, a series of actions such as modeling one object for multiple parts or modeling multiple people individually is possible. In the case of performing shooting twice, the resolution of the portion corresponding to the object becomes uniform. If the appropriate value is fixedly determined, a photographed image suitable for modeling can be obtained regardless of the photographing distance and the size of the subject.

[0008]

DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 1 is a block diagram of a three-dimensional input system according to the present invention, FIG. 2 is a schematic diagram of a twin-lens camera, and FIG.

The three-dimensional input system 100 comprises a twin-lens camera 1 in which two digital cameras are integrated, and a computer 8 in which software for three-dimensional input and modeling by stereo vision is incorporated. A display and a keyboard are connected to the computer 8 as a user interface. Between the twin-lens camera 1 and the computer 8, online data transfer by cable or infrared communication and offline data transfer by a portable recording medium are possible.

Two light receiving sections 1 are provided on the front of the twin-lens camera 1.
1 and 12 are arranged side by side, and two electronic viewfinders 21 and 22 are arranged on the back in accordance with the arrangement. The twin-lens camera 1 repeats the photoelectric conversion of the subject 90 at regular intervals in the standby state. Electronic finder 21,
Numeral 22 sequentially displays the subject image that has been photoelectrically converted on a monitor. When the user presses the release button 24, the subject image at that time is recorded as a photographed image. In this specification, “photographing” means recording a subject image in response to a photographing instruction.

A pair of captured images (stereo images) having different viewpoints obtained by the twin-lens camera 1 is used by the computer 8 for modeling a subject. At the time of modeling, as shown in FIG.
Is displayed on the monitor. There is a corresponding point search as a pre-process for reconstructing three-dimensional data based on a stereo image. This is data processing for searching for a corresponding point in another image for all coordinate points of a portion indicating a subject in a reference image of two images. The shape of the object is reconstructed by obtaining the three-dimensional coordinates of the point on the object by applying the stereoscopic principle for all the corresponding points.

FIG. 5 is a functional block diagram of a main part of the twin-lens camera. The twin-lens camera 1 has light receiving units 11 and 12 and an image processing unit 30. The light receiving unit 11 includes an optical system 111 having a variable imaging magnification and an imaging device 112 represented by a CCD sensor. Similarly, the light receiving unit 12 also includes an optical system 121 whose imaging magnification is variable and an imaging device 122. In searching for a corresponding point, for example, an image captured by the light receiving unit 11 is used as a reference image, and an image captured by the light receiving unit 12 is used as a reference image. The image processing unit 30 includes memories 31 and 32 and a CPU 33.
Consists of

The subject images formed on the light receiving surfaces of the imaging devices 112 and 122 are photoelectrically converted and temporarily stored in the memories 31 and 32. The CPU 33 instructs the optical systems 111 and 121 to change the imaging magnification as needed, as described later.

FIG. 6 is a diagram showing a photographing situation according to a first example of the control of the optical system. In FIG. 6, subject 9
1 is a car. In order to model the entire car, multiple shots are taken from different viewpoints. In this case, the two cameras 1
It is assumed that the second photographing is performed, and then the second camera 1 is moved to perform the second photographing at an obliquely rear position of the automobile.

FIG. 7 is a diagram showing a stereo image in the first shooting, FIG. 8 is a diagram showing a monitor image in the second shooting, and FIG. 9 is a diagram showing a stereo image in the second shooting.

In the first photographing, images G11 and G21 are recorded. The images G11 and G21 include subject images 911 and 912 showing the first half of the vehicle. After the first shooting, the photographer moves the twin-lens camera 1 to the second shooting position.
Place. At this time, when the second photographing position is farther from the car than the first photographing, the subject image 9 in the monitor images G12 ′ and G22 ′ captured by the light receiving units 11 and 12 is obtained.
13,914 is small. If the second shooting is performed as it is, subject images 911 and 91 to be noticed in the corresponding point search
The resolution differs between the pair 2 and the pair of the subject images 913 and 914, and the precision of the shape model becomes uneven. Therefore, the CPU 33 adjusts the imaging magnification of the optical systems 11 and 12 so that the resolution of the corresponding point search becomes uniform regardless of the shooting position. In the second photographing performed after the adjustment, the image G
12, G22 are recorded. In the images G12 and G22, the magnifications of the subject images 915 and 916 showing the latter half of the vehicle are almost the same as those of the subject images 911 and 912.

FIG. 10 shows the first operation of the three-dimensional input system.
It is a flowchart which shows an example. In the case of a series shooting mode in which a plurality of shootings is a series of operations related to each other, the twin-lens camera 1 detects a change in distance to the subject in a standby period after the first shooting, and responds to the distance. By controlling the optical system, the imaging magnification is adjusted to the first photographing (# 11 to # 13). Regarding the mode setting, there are a mode in which a series of operations from power-on to a reset operation are performed in a series of shooting modes, and a mode in which a mode switch is provided to switch modes. The optical system is similarly controlled after the second shooting (#
14, # 15). The computer 8 reconstructs a three-dimensional shape by searching for corresponding points for each of a plurality of sets of captured images, and integrates a plurality of partial models (# 16 to # 1).
8). Note that the modeling function of the computer 8 may be incorporated in the twin-lens camera 1.

The method of detecting the distance to the object includes a method of obtaining the distance from the focal length of the optical system or focus information, and a method of recognizing the distance from the low-resolution object images 913 and 914 obtained from the monitor image.

[Embodiment 2] FIG. 11 shows a second example of the control of the optical system.
It is a conceptual diagram of an example. When creating a model of a person's face, it is necessary to consider individual differences in face size. In FIG. 11, a portion (face area) 901 ′ corresponding to a person's face in the monitor image G10 ′ is small. At the time of photographing, the imaging magnification is adjusted by controlling the optical system so that the face area 901 of the photographed image G10 has an appropriate size.

FIG. 12 shows the second operation of the three-dimensional input system.
It is a flowchart which shows an example. In the twin-lens camera 1, the CPU 33 cuts out a face area from the monitor image input to the memory 31 (# 21, # 22). And C
The PU 33 controls both of the two optical systems 111 and 121 so that the area of the face region (that is, the number of pixels) becomes an appropriate value exceeding the threshold (# 23, # 24, # 27). The computer 8 performs a corresponding point search on the stereo image photographed at an appropriate magnification to reconstruct the three-dimensional shape of the face (# 2).
5, # 26).

With respect to the extraction of the face area, there are a method of extracting a skin color area by performing a clustering process and a method of extracting an area having a relatively small luminance gradient. Generally, humans have low skin contrast, and the undulation of the face is smooth. Therefore, the luminance gradient of the face is small.

[Embodiment 3] FIG. 13 shows a third embodiment of the control of the optical system.
FIG. 14 is a diagram illustrating a monitor image according to an example, and FIG. 14 is a diagram illustrating a captured image according to a third example of control of the optical system. Example 3
This ensures a sufficient number of corresponding points for high spatial frequency parts such as the eyes, nose, and mouth, and is suitable for face modeling as in the second embodiment.

In FIG. 13, a portion (eye area) 90 corresponding to the pupil and iris of the person in the monitor image G30 'is shown.
2 'is small. At the time of photographing, the imaging magnification is adjusted by controlling the optical system so that the eye region 902 of the photographed image G30 in FIG. 14 has an appropriate size.

FIG. 15 shows the third operation of the three-dimensional input system.
It is a flowchart which shows an example. In the twin-lens camera 1, the CPU 33 cuts out an eye area from the monitor image input to the memory 31 (# 31, # 32). Usually, the color difference between the iris and its surroundings is remarkable, so that it is relatively easy to cut out the eye region by image processing. And CPU
Reference numeral 33 controls both of the two optical systems 111 and 121 so that the number of pixels in the eye region becomes an appropriate value exceeding the threshold (# 3).
3, # 34, # 37). The computer 8 performs a corresponding point search on the stereo image taken at an appropriate magnification,
The three-dimensional shape of the face is reconstructed (# 35, # 36).

As an application of the reconstructed shape data, there is a fitting for deforming a standard model according to the shape data. The fitting streamlines the generation of a model in a specified format such as animation. According to the third embodiment, the fitting result of a portion having a high spatial frequency becomes uniform irrespective of the individual difference of the subject and the photographing distance, and it is possible to obtain an elaborate model representing the characteristics of each subject.

In the above embodiment, the multi-lens camera 1b shown in FIG. The multi-lens camera 1b has a third light receiving unit 13. 3
By applying the stereoscopic vision, the accuracy of the corresponding point search is improved.

[0027]

According to the first to fourth aspects of the present invention, it is possible to equalize the resolution of a subject portion in a photographed image when performing photographing a plurality of times.

According to the invention of claims 2 to 4,
Regardless of the shooting distance and the size of the subject, the resolution of the subject portion in the captured image can be optimized.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of a three-dimensional input system according to the present invention.

FIG. 2 is a schematic view of a twin-lens camera.

FIG. 3 is a diagram illustrating an example of a shooting situation;

FIG. 4 is a diagram showing a monitor display in modeling.

FIG. 5 is a functional block diagram of a main part of the twin-lens camera.

FIG. 6 is a diagram illustrating a shooting situation according to a first example of control of the optical system.

FIG. 7 is a diagram showing a stereo image in the first shooting.

FIG. 8 is a diagram showing a monitor image according to a second photographing.

FIG. 9 is a diagram showing a stereo image in the second shooting.

FIG. 10 is a flowchart illustrating a first example of the operation of the three-dimensional input system.

FIG. 11 is a conceptual diagram of a second example of control of the optical system.

FIG. 12 is a flowchart illustrating a second example of the operation of the three-dimensional input system.

FIG. 13 is a diagram showing a monitor image according to a third example of the control of the optical system.

FIG. 14 is a diagram illustrating a captured image according to a third example of control of the optical system.

FIG. 15 is a flowchart illustrating a third example of the operation of the three-dimensional input system.

FIG. 16 is a schematic diagram of a multi-lens camera.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Two-lens camera (photographing device) 1b Multi-lens camera (photographing device) G11, G21, G12, G22 Photographed image 111, 121 Optical system 33 CPU (distance detection means) 30 Image processing part (image processing means)

Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat II (reference) G03B 15/00 G03B 15/00 H H04N 5/225 Z H04N 5/225 5/232 A 5/232 G01B 11/24 K F term (reference) 2F065 AA04 AA53 BB05 CC16 DD03 FF05 JJ03 JJ26 KK03 MM06 QQ04 QQ24 QQ28 QQ33 2F112 AC06 BA06 CA08 CA12 FA03 FA21 FA39 FA45 2H044 DA01 DA02 DA04 DB02 DC02 5C022 AB62 AB66 CA02 5C06 AB

Claims (4)

[Claims] An imaging apparatus for three-dimensional input for converting a shape into data based on a captured image of an object, comprising: a series of imaging modes in which a plurality of imaging operations are performed in association with each other; It has an optical system for photographing that can change the magnification and a distance detecting unit that detects the distance to the object. It corresponds to the object in the photographed image at the time of the second or subsequent photographing in the series photographing mode. An imaging device for three-dimensional input, wherein an imaging magnification is adjusted in accordance with a detected distance so that a size of a portion to be adjusted is matched with the size of the first imaging. 2. An imaging device for three-dimensional input for converting a shape into data based on a captured image of an object, comprising: an optical system capable of changing an imaging magnification; Image processing means for detecting the resolution of the portion corresponding to the target object, and adjusting the imaging magnification so that the resolution of the portion corresponding to the target object in the captured image becomes a value within the set range. An imaging device for three-dimensional input characterized by the following. 3. A photographing apparatus for three-dimensional input for converting a shape into data based on a photographed image of an object, comprising: an optical system for photographing capable of changing an imaging magnification; Image processing means for detecting a specific color portion, and adjusting the imaging magnification so that the size of the specific color portion becomes a value within a set range. Shooting equipment. 4. An imaging device for three-dimensional input for converting a shape into data based on a captured image of an object, comprising: an optical system capable of changing an imaging magnification; Image processing means for detecting a portion having a specific luminance gradient, and adjusting the imaging magnification so that the size of the portion having the specific luminance gradient becomes a value within a set range. An imaging device for three-dimensional input.