JP2001148866A - Image input device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enhance the accuracy of a three-dimensional image in its reconfiguration or to decrease the sense of incongruity in a panorama image by using a single distance obtained through measurements so as to control the respective foci of a plurality of image pickup sections. SOLUTION: The image input device 1 picks up an object by using a plurality of image pickup sections 11, 21 each including a light receiving lens and a photoelectric conversion element so that the picked up images have at least partly overlapping parts, and receives the resulting image. The device is provided with a distance detection section 31 that measures the distance up to the object and control means 12, 22 that use a single distance based on the measurements by the distance detection section 31 so as to control the respective foci of a plurality of the image pickup sections 11, 21.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image input apparatus for picking up an object using a plurality of image pickup units including a light receiving lens and a photoelectric conversion element in order to obtain three-dimensional data or a panoramic image of the object. About.

[0002]

2. Description of the Related Art Conventionally, an image input device for inputting a plurality of images of a target (subject) by a plurality of image pickup units has been used. Such a device is used for obtaining a panoramic image by capturing images from adjacent viewpoints that can be regarded as virtually the same point, for viewing the same target object from multiple viewpoints, or based on an image obtained from multiple viewpoints. Dimensional image (3
It is used for various purposes such as displaying or reconstructing (dimensional data or stereoscopic image).

In an application for obtaining a panoramic image, in order to obtain a wider range of images of an object, it is preferable that the overlapping portion of each image is small. On the other hand, in an application for obtaining a three-dimensional image, since the three-dimensional image is reconstructed by the overlapping part of the images, it is preferable that the overlapping part of each image is large.

Conventionally, as an apparatus which can be used for such a purpose, there is an apparatus disclosed in Japanese Patent Application Laid-Open No. 9-305796. In this conventional apparatus, two imaging lenses (light receiving lenses) are used, and a control distance X of a focus is determined based on a distance X1 to an object and a distance X2 to a background. Determining the aperture value so that the distance X1 and the distance X2 fall within the depth;
Determining the focal length at which the amount of overlap of the image is maximum is executed for each imaging lens.

Japanese Patent Application Laid-Open No. 8-116553 discloses that
There is disclosed a system configured to set a desired convergence angle and a focus based on a distance between images of an object obtained by two sensors and a current convergence angle.

[0006]

However, when an image is input through a plurality of light-receiving lenses, even if each of the light-receiving lenses is in focus at some point on the object, it is focused at the same point. Not necessarily.

[0007] Therefore, in the related art, even if the images picked up by a plurality of cameras are at the same point on the object, the sizes of the images may be different from each other or the blur states may be different from each other.

[0008] These differences in magnification or blur state among a plurality of images cause errors when searching for corresponding points between images in the reconstruction of a three-dimensional image. Therefore, it is desirable from the viewpoint of the accuracy of the generated three-dimensional image that the same point on the object be obtained as image data at the same magnification and in the same blur state in each image.

In addition, when a plurality of images are connected to obtain a panoramic image, a difference in magnification or blur between the plurality of images causes discomfort in the obtained images.

SUMMARY OF THE INVENTION The present invention has been made in view of the above-described problem, and controls the focus of each of a plurality of image pickup units using one distance obtained by measurement, thereby achieving three-dimensional image reconstruction. An object of the present invention is to improve accuracy or reduce discomfort in a panoramic image.

[0011]

According to a first aspect of the present invention, an object is imaged by a plurality of imaging units including a light receiving lens and a photoelectric conversion element so that at least a part of the image has a portion overlapping each other. An image input device for inputting an image by using a distance measuring unit for measuring a distance to the object, and each of the plurality of imaging units using one distance based on the measurement by the distance measuring unit. And control means for controlling the focus.

In the apparatus according to the second aspect of the present invention, the control means can independently control each of the plurality of imaging units. In the apparatus according to the third aspect of the present invention, when the focus position under the control of the control unit differs depending on the imaging unit, the magnification of the captured image is corrected.

According to a fourth aspect of the present invention, the control means controls the attitude of each of the plurality of imaging units using the one distance. In the apparatus according to a fifth aspect of the present invention, the control unit controls the apertures of the plurality of imaging units using the one distance.

In the apparatus according to a sixth aspect of the present invention, a means for obtaining an aperture value Aa based on the measurement result by the distance measuring means, a means for obtaining an aperture value Ar based on an exposure amount by photometry, and a method for obtaining the aperture value Aa Means for selecting an aperture value A for control based on the aperture value Ar.

[0015]

FIG. 1 is a block diagram showing an image input apparatus 1 according to a first embodiment of the present invention, FIG. 2 is a view showing an image pickup state by the image input apparatus 1, and FIGS. It is a figure for explaining depth in imaging.

In FIG. 1, an image input apparatus 1 includes image pickup units 11 and 21, focus control units 12 and 22, and a zoom control unit 1.
3, 23, aperture control units 14, 24, attitude control units 15, 2
5, a CPU 30, a distance detection unit 31, a memory 32, a magnification correction value calculation unit 33, a magnification correction unit 34, and the like.

The image pickup section 11, focus control section 12, zoom control section 13, aperture control section 14, and attitude control section 15 constitute a first camera 10, and include an image pickup section 21, a focus control section 2
2. The zoom control unit 23, the aperture control unit 24, and the attitude control unit 25 constitute the second camera 20. Since the first and second cameras 10 and 20 have the same basic configuration and function, only one of them will be described.

The first and second cameras 10, 20
May be described as a first imaging unit 11, a second imaging unit 21, and the like. As shown in FIG. 2, the first and second imaging units 11 and 21 are vertically symmetrically arranged with respect to the optical axis of the distance detection unit 31.
The optical axes of the first and second imaging units 11 and 21 and the distance detection unit 31 are on the same plane, and the first and second imaging units 11 and 21 are arranged such that their optical axes are on the same plane. Movable to be within. Angles formed between the optical axis of the distance detection unit 31 and the optical axes of the first and second imaging units 11 and 21 are convergence angles α1 and α2.

The first imaging unit 11 and the second imaging unit 2
1 is arranged so that the base line length LA, which is the distance between them, is known. Returning to FIG. 1, the imaging unit 11 includes a light receiving lens 101 and a photoelectric conversion element 102. The light receiving lens 101 is a zoom lens formed by combining a plurality of lenses. Photoelectric conversion element 102
Is composed of, for example, a CCD and outputs image data D1 and D2 corresponding to an image formed by the light receiving lens 101.

The focus control section 12 drives the light receiving lens 101 to control the focus of the image pickup section 11. That is, the light receiving lens 101 is moved based on the distance LB measured by the distance detection unit 31 so that the imaging unit 11 is in focus. Normally, control is performed so that the intersection between the optical axis of the imaging unit 11 and the surface of the object Q is focused.

Therefore, control data for controlling the light receiving lens 101 is stored in the memory 32 in advance. The control data includes, for example, focal lengths LB1 and LB2 which are control targets for focusing of the light receiving lens 101, and convergence angles α1 and α which are control targets of the attitudes of the imaging units 11 and 21.
2, or a drive control value for positioning to the control target.

Such control data is transmitted to the distance detecting section 31.
Is calculated in advance in accordance with the base line length LA and the method of controlling the attitude of the imaging units 11 and 21, and is stored in the memory 32 in, for example, a table format.

For example, as shown in FIG.
The optical axes of the first and second imaging units 11 and 21 are symmetric with respect to one optical axis, and the optical axes of the first and second imaging units 11 and 21 are directed to the intersection between the optical axis of the distance detection unit 31 and the surface of the target object Q. When the attitude is controlled in the following manner, the focal length LB1, which is the control target of each light receiving lens 101 of the first and second imaging units 11 and 21, is controlled.
LB2 is equal to each other and is calculated from the distance LB0 and the base line length LA. Similarly, the convergence angles α1 and α2 are calculated.

When an image is input by the image input device 1, control data is read from the memory 32 based on the output of the distance detecting unit 31, and based on the control data, each light receiving lens 1 is read.
01 is performed.

Since each light receiving lens 101 has an individual difference, the above control data is determined in consideration of the individual difference. Therefore, by controlling based on the control data read from the memory 32, the individual difference of each light receiving lens 101 is also corrected.

Next, the zoom controller 13 controls the light receiving lens 1
01 is driven to control the optical magnification of the imaging unit 11. Generally, when the same object is imaged by a plurality of light receiving lenses, an error occurs in the imaging magnification because the distance from each light receiving lens to the object is not the same. Even when the distance from each light receiving lens to the object is the same, a slight error occurs in the imaging magnification due to a substantial difference in focal length depending on the focus state.

First and second zoom controllers 13 and 23
These errors are reduced by controlling the optical magnification. Further, the angle of view is controlled by the zoom control units 13 and 23 so that the target object Q is imaged as large as possible in the imaging plane of each photoelectric conversion element 102. In addition,
The correction of the imaging magnification error is mainly performed by a magnification correction unit 34 described later.

The aperture control unit 14 controls the aperture of the imaging unit 11. By controlling the aperture, the first and second
The amount of blur in the imaging units 11 and 21 is minimized, and the blur states can be regarded as equivalent to each other.

That is, the imaging unit 11 is provided with a variable or fixed aperture mechanism. The aperture value is calculated so that the depth of field satisfies the following condition. That is, the maximum range (maximum depth) in the depth direction is set as the depth length (length LC shown in FIG. 3) of the range AE that fits in the common visual field of the first and second imaging units 11 and 21. The range (minimum depth) is determined by the first and second imaging units 11 and 21.
Are set as ranges that can be regarded as the same blur state. The range that can be regarded as the same blur state generally refers to a range where the blur is one pixel or less of the photoelectric conversion element 102.

That is, in FIG. 4, the minimum depth dal
It is shown. First, focus on one point P1 on the object Q. When both the imaging units 11 and 21 are focused on the distance LD1, the imaging unit 11 is in focus because the point P1 is located at the distance LD1. However, in the imaging unit 21, the vertical distance which is the optical axis direction component of the distance to the point P1 is LD2.
Is LD2 = LD1-dal.

Therefore, the depth of field of the imaging unit 21 needs to be at least as long as the distance dal. When this condition is satisfied, at least the point P1 indicates that both the imaging units 11 and
At 21 it can be considered that the subject is in focus.

The depth of field refers to the distance in the depth direction where a point can be regarded as a point, and it is necessary to determine the area which can be regarded as a point. Here, a value determined with respect to the minimum unit area of the photoelectric conversion element 102 is, for example, half of the minimum unit area from the sampling theorem.

With the above configuration, in the images obtained from the two imaging units 11 and 21, the same point on the object Q has the same size and the same blur state.
The value actually used is determined by comparing the calculated aperture value with the range of the aperture value obtained from the exposure amount measurement. Details will be described in a flowchart described later.

The attitude control section 15 controls the attitude of the image pickup section 11 and varies the convergence angle α. That is, the attitude of the imaging unit 11 is controlled based on the distance LB measured by the distance detection unit 31 so that the convergence angle α of the imaging unit 11 becomes a predetermined value. Normally, control is performed so that the optical axes of the first and second imaging units 11 and 21 are directed to the intersection between the optical axis of the distance detection unit 31 and the surface of the target object Q.

For this purpose, the memory 32 includes an image pickup unit 1
Control data for the attitude control of 1, 21 is stored in advance. The control data is, as described above, the convergence angle α corresponding to the distance LB measured by the distance detection unit 31 or a drive control value that sets the convergence angle α as a control target.

When an image is input by imaging, control data is read from the memory 32 based on the output of the distance detecting unit 31, and the respective imaging units 11 and 21 are controlled based on the control data. In the present embodiment, the convergence angles α1, α2
Are controlled to be equal to each other.

Further, by controlling the convergence angle α,
The overlapping portion of the two images obtained by the imaging units 11 and 21 can be set to an appropriate size depending on the application. For example, when imaging for obtaining a three-dimensional image is performed, the convergence angle α and the angle of view are controlled so that the image of the object Q enters the center of each imaging region as much as possible. This makes it possible to accurately search for a corresponding point between the images in the reconstruction of the three-dimensional image, and to obtain a highly accurate three-dimensional image.

In the case of performing imaging for obtaining a panoramic image, it is possible to make an adjustment so that the overlapping portion is minimized. The control described above is performed based on a command from the CPU 30. Note that the first camera 10 and the second camera 20 can be controlled independently of each other.

In FIG. 1, the magnification correction value calculation unit 33
As described above, a magnification correction value for correcting the magnification of a captured image is calculated. The magnification correction unit 34 performs enlargement processing or reduction processing on one or both of the image data D1 and D2 output from the imaging unit 11 or the imaging unit 21 based on the calculated magnification correction value. .

That is, even when the focus is controlled in the state shown in FIG. 1, the focus position is different due to individual differences of the respective light receiving lenses 101, and the magnification is slightly different. In this case, a magnification ratio is determined from the distance LB0 obtained by the distance detection unit 31 and the focal lengths LB1 and LB2 calculated in advance and stored in the memory 32, and this is set as a magnification correction value.

Since the magnification correction is performed by the magnification correction unit 34, the accuracy of searching for a corresponding point when reconstructing a three-dimensional image is improved. In addition, in the panoramic image, an adjacent image without a sense of discomfort can be obtained.

The magnification of the image is the same only in the vicinity of the distance measurement point used for calculating the magnification correction value. In other words, in the end of the image, the focus distance is shifted by the imaging units 11 and 21. Although the magnification will be slightly different due to this displacement, the target object Q is usually arranged near the center of the imaging area in the three-dimensional image input, so that a sufficient effect can be obtained by the magnification correction of the present embodiment. it can.

The CPU 30 controls the above control units and controls the entire image input apparatus 1. The three-dimensional reconstruction unit 3 generates a three-dimensional image (three-dimensional data) D3 based on the image data D10 and D20 output from the image input device 1.

Next, the flow of processing for controlling the aperture will be described with reference to the flowchart shown in FIG. In FIG. 6, first, the distance LB0 to the target object Q is measured by the distance detecting unit 31 (# 11). Measured distance LB
The aperture value Aa corresponding to 0 is read from the memory 32 (# 12). The aperture value Aa is a value that can secure the depth with respect to the distance LB0, and is obtained in advance and stored in the memory 32.

That is, in FIG. 5, the aperture value table ST stores an aperture value Aa capable of securing a depth corresponding to each distance LB0 and each angle of view.
The larger the aperture value Aa, the better the aperture of the light receiving lens 101, and therefore the greater the depth, and the longer the depth that can be imaged. The angle of view here is determined by the focal length of the light receiving lens 101. If the angle of view is constant, only data corresponding to the angle of view need be stored.

Returning to FIG. 6, control by the aperture control unit 14 is performed based on the read aperture value Aa, and in this state, the exposure EAa is measured by a photometer (not shown) (# 1).
3).

On the basis of the measured exposure amount EAa, the minimum value of the aperture value A Armi is determined from the combination of the shutter speed (exposure time) and the aperture value Ar for obtaining the required exposure amount.
n and the maximum value Armax are determined (# 14). That is, in steps # 13 and # 14, the range of the appropriate aperture value A is limited based on the exposure amount EAa.

If the aperture value Aa is smaller than the minimum value Armin (Yes in # 15), the minimum value Armin is adopted as the aperture value A (# 16). Thus, the exposure amount E
A proper exposure to Aa is secured, and the aperture value A
Since the minimum value Armin narrower than a is adopted, the depth is also secured.

When the aperture value Aa is larger than the minimum value Armin (No in # 15) and smaller than the maximum value Armax (Yes in # 17), the aperture value Aa is set to the exposure amount EA.
Since it is within the proper exposure range for securing a, the aperture value Aa is directly used as the aperture value A (# 18).

If the aperture value Aa is larger than the maximum value Armax (No in # 17), the aperture value Aa is adopted as the aperture value A to secure the depth (# 19), and the aperture value A In this case, since the exposure amount EAa cannot be secured due to the excessive aperture, the auxiliary light is emitted to secure the exposure amount EAa (# 20).

In the image input device 1 of the first embodiment described above, only one distance detecting section 31 is provided. However, a plurality of distance detecting sections 31 are provided, and a plurality of distance detecting sections 31 are obtained by measuring them. May be determined based on the distance. In this case, one optimal distance LB0 may be selected from a plurality of distances obtained by measurement, or one optimal distance LB0 may be obtained by calculation based on the plurality of distances. . Note that a plurality of distance detection units 31
May be provided corresponding to the respective imaging units 11 and 21, or may be provided independently of the imaging units 11 and 21.

Next, a second embodiment of the present invention will be described. FIG. 7 is a diagram illustrating a state of imaging by the image input device 1B according to the second embodiment of the present invention. In addition, FIG.
FIG. 3 is a diagram corresponding to FIG. 2 of the first embodiment.

In the image input device 1B of the second embodiment,
The basic configuration is the same as that of the image input device 1 of the first embodiment. Hereinafter, only the differences will be described. In FIG. 7, a half mirror 103 is provided on one imaging unit 11B.
Is provided. The optical axis of the distance detection unit 31B extends forward via the half mirror 103, and is configured to be very close to or coincide with the optical axis of the imaging unit 11B.

The positional relationship between the distance detecting section 31B and the image pickup section 11B is fixed, and they are fixed so as not to move. That is, in the first embodiment, the postures of both the imaging units 11.21 are controllable, but in the second embodiment, the posture of one imaging unit 11B is fixed, and the posture of the other imaging unit 11B is fixed. Only the imaging unit 21 is attitude-controlled.

That is, based on the distance LB0 measured by the distance detection unit 31B, the attitude of the imaging unit 21 is controlled, and the optical axis of the imaging unit 21 passes through the intersection of the optical axis of the imaging unit 11B and the surface of the object Q. Is controlled. The focal length LB2, which is the control target of the imaging unit 21, is determined corresponding to the distance LB0. Note that the focal length LB1 of the imaging unit 11B
Is equal to the distance LB0.

Since the focal lengths LB1 and LB2 are different from each other, that is, the focus positions are different from each other,
Image data D1 obtained from the imaging unit 11B and the imaging unit 21
Is different from the image data D2 obtained from. Therefore, the magnification correction value is calculated by the magnification correction value calculation unit 33, and based on this, the magnification of the image data D1 and D2 is corrected by the magnification correction unit.

In the image input device 1B of the second embodiment,
It is sufficient to control the attitude of only one of the imaging units 21. Therefore, the configuration for control is simplified. As shown in FIG. 8, when the distance LB0 measured by the distance detection unit 31 is set to Zk in order to effectively use the measured depth, the control of the convergence angle α and the focus is represented by the following equation. Distance Zs
May be performed at the position.

Zs = Zk + Zt Here, Zt is a value that can be set by the user, and can be either positive or negative.

In FIG. 8, Zt is set to a positive value, and the control is performed so that the focal point is located at a position that is deeper than the surface of the object Q by the distance Zt. When obtaining a panoramic image by arranging a plurality of images, for example, the image input device 1 according to the first embodiment may be used to control the imaging units 11 and 21 so as to be in the posture shown in FIG. .

That is, the light receiving ranges are aligned at the position of the distance LB0 measured by the distance detecting unit 31.
In addition, the postures of the imaging units 11 and 21 are controlled such that a part of the imaging ranges of the imaging units 11 and 21 overlap with each other.

FIG. 10 shows three imaging units 11, 21, 21C.
FIG. 3 is a diagram showing an example of an image input device 1C using the image input device 1C. FIG.
1 shows a view of the image input apparatus 1C as viewed from the direction of the optical axis.

In FIG. 10, the image pickup unit 11 is fixed, the image pickup unit 21 can perform attitude control so that the convergence angle α changes in the direction of the arrow M1 in the figure, and the image pickup unit 21E is the arrow in the figure. Posture control is possible so that the convergence angle α changes in the M2 direction.

For example, three-dimensional data is generated based on the image data obtained from the imaging unit 11 and the imaging unit 21.
Further, three-dimensional data is separately generated based on the image data obtained from the imaging unit 11 and the imaging unit 21E. By comparing the three-dimensional data with each other, it is possible to check whether or not the generated three-dimensional data is correct. If there is insufficient three-dimensional data, the other three-dimensional data can be supplemented to complete three-dimensional data.

Therefore, by providing three or more imaging units, the accuracy and reliability in reconstructing a three-dimensional image can be further improved, and the sense of discomfort in a panoramic image can be further reduced.

In the embodiment described above, the configuration, arrangement, control method, data processing content, etc. of each part or the whole of the image input devices 1, 1B, 1C can be appropriately changed in accordance with the gist of the present invention. it can.

[0066]

According to the present invention, 1 obtained by measurement is obtained.
By controlling the focus of each of the plurality of imaging units using one distance, it is possible to improve the accuracy in reconstructing the three-dimensional image or reduce the sense of discomfort in the panoramic image.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating an image input device according to a first embodiment of the present invention.

FIG. 2 is a diagram illustrating a state of imaging by an image input device.

FIG. 3 is a diagram for describing depth in imaging.

FIG. 4 is a diagram for describing depth in imaging.

FIG. 5 is a diagram illustrating an example of an aperture value table.

FIG. 6 is a flowchart illustrating the flow of aperture control processing.

FIG. 7 is a diagram illustrating a state of imaging by an image input device according to a second embodiment of the present invention.

FIG. 8 is a diagram showing another example of control of the convergence angle and the focus.

FIG. 9 is a diagram illustrating an example of attitude control when obtaining a panoramic image.

FIG. 10 is a diagram illustrating an example of an image input device using three imaging units.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Image input device 11, 21 Imaging part 12, 22 Focus control part (control part) 13, 23 Zoom control part (control part) 14, 24 Aperture control part (control part) 15, 25 Attitude control part (control part) 30 CPU 31 Distance detection unit (distance measurement means) 32 Memory 33 Magnification correction value calculation unit 34 Magnification correction unit 101 Light receiving lens 102 Photoelectric conversion element Q Object

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) H04N 5/232 H04N 5/238 Z G02B 7/11 A 5/238 G03B 3/00 A (72) Inventor Takayuki Hamaguchi 2-3-1, Azuchicho, Chuo-ku, Osaka City, Osaka Prefecture F-term in Osaka International Building Minolta Co., Ltd. 2H011 AA06 BA01 CA21 DA01 2H051 AA00 BB01 EB04 FA47 GB12 2H059 AA08 AA12 BA11 5C022 AA01 AB12 AB21 AB61 AB66 AC42 AC78 5C061 AA29 AB02 AB04

Claims (6)

[Claims] 1. An image input apparatus for inputting an image by capturing an object by using a plurality of image pickup units including a light receiving lens and a photoelectric conversion element so that at least a part of the image has a portion overlapping each other. Distance measuring means for measuring the distance to the object, and control means for controlling the focus of each of the plurality of imaging units using one distance based on the measurement by the distance measuring means. An image input device, characterized in that: 2. The image input device according to claim 1, wherein said control means is capable of independently controlling each of said plurality of imaging units. 3. The image input device according to claim 1, wherein a magnification of an image to be picked up is corrected when a focus position controlled by said control means differs between said image pickup units. 4. The image input device according to claim 1, wherein the control means controls the attitude of each of the plurality of imaging units using the one distance. 5. The image input device according to claim 1, wherein the control unit controls the aperture of each of the plurality of imaging units using the one distance. 6. A means for obtaining an aperture value Aa based on a measurement result by the distance measuring means, a means for obtaining an aperture value Ar based on an exposure amount by photometry, and a control based on the aperture value Aa and the aperture value Ar. The image input apparatus according to claim 5, further comprising: means for selecting an aperture value A for the following.