JP2974500B2 - Compound eye imaging device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image pickup apparatus, and provides one high-definition image by electrically correcting and synthesizing at least two images obtained by using at least two sets of image pickup optical systems. To a device that

[0002]

2. Description of the Related Art Conventionally, as a principle method of obtaining one high-definition image by combining two images obtained by using two sets of imaging optical systems, for example,
-03-04 (pp. 23-28) and the like are known.

[0003] When the sampling points of the image sensors used in the two sets of image pickup optical systems are projected onto a virtually common subject, the phase difference is spatially different, and each image signal is used. Are combined to obtain one high-definition image. FIG. 9 shows a conceptual diagram of this principle. In FIG. 9, 811 and 821 are imaging optical systems, and 812 and 822 are images output by the respective imaging optical systems. In step 801, each image signal is synthesized to obtain a high-definition image 802.

However, in the conventional example, since the optical arrangement has a convergence angle in principle, it is conceivable that a registration shift occurs in an image obtained from the imaging optical system. FIG. 10A shows an outline of the registration deviation in the compound-eye imaging system, in which 910 and 920 are imaging optical systems (lenses), and 911 and 921 are image sensors. The optical axes of the imaging optical systems 910 and 920 are 912 and 922, respectively. Here, the central axis O-
When an image of a subject is taken with each of the optical axes 912 and 922 inclined by θ in the xz plane with respect to O ′, an arbitrary object point on the subject plane is set to P. At this time, assuming that the image points on the image sensors 911 and 921 with respect to P are R 'and L', respectively, registration deviation occurs because of R '≠ L', as shown in FIG. In the composite image 930 obtained by simply adding to the above, the image for the object point P is doubled,
As a result, a high-precision image cannot be provided.

[0005]

SUMMARY OF THE INVENTION It is an object of the present invention to improve the definition of an image synthesized from a plurality of original images.

[0006]

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and according to the first aspect of the present invention, a plurality of convergence angles variably arranged. In an apparatus for imaging a common subject using an imaging system, first and second memories for storing image signals output from the respective imaging systems, a unit for detecting imaging conditions of the imaging system, The first output from each imaging system
Means for detecting the position coordinate information of the subject by performing a correlation operation between the image signals stored in the first and second memories, and capturing the images stored in the first and second memories, respectively. Based on the conditions and the position coordinate information, each is converted into the corresponding position coordinates of the subject image in pixel units, and then written into the third image memory to be synthesized, thereby generating a high-definition image, and A compound-eye imaging device comprising: means for correcting registration deviation between images.
According to a second aspect of the present invention, in the first aspect, the combining means forms a combined image based on a predetermined viewpoint position.

[0007]

1 shows a basic arrangement of a compound-eye photographing system according to the present invention. In the figure, 1 is a common object plane, 102 and 202 are first and second imaging optical systems having equivalent specifications,
Generally, a zoom lens is used as described later. 10
Similarly, reference numerals 3 and 203 denote image sensors having equivalent specifications, and use an image pickup tube such as a saticon or a solid-state image pickup device such as a CCD. Here, for the sake of simplicity, a single-plate type (or single-tube type) is schematically shown, but a two-plate type (two-tube type) or a three-plate type (three-tube type) via a color separation optical system is also general. Do not lose.

The optical axes 101 and 201 intersect at a point O on the object plane 1, pass through the point O, and pass through a normal line O of the object plane 1.
It is arranged to be symmetrically inclined with respect to −O ′ by θ. 2
θ is defined as a convergence angle, and an image is taken by changing the convergence angle according to a change in the subject distance S.

FIG. 2 shows a specific configuration of the image pickup optical systems 102 and 202, and FIG. 3 is a block diagram showing the function of each member constituting the image pickup optical system. 102a, 1
02b, 102c, 102d and 202a, 202
b, 202c and 202d are the first and second imaging optical systems 1
2 shows a lens group constituting the lens group 02 and 202,
b and 202b denote a zooming group, and 102d and 202d denote focusing groups. Reference numerals 106 and 206 denote the zooming groups 102b and 2
A drive system (zoom motor) for driving 02b, and 107 and 207 indicate drive systems (focus motors) for driving the focusing groups 102d and 202d. Further, 102 and 103, 202 and 203 are integrally provided with a mechanical system (not shown) that rotates in a plane including the optical axes 101 and 201, and drive systems (convergence angle motors) 104 and 204.

Reference numerals 105 and 205 denote angle encoders.
The rotation angles of the imaging optical systems 102 and 202 are measured. 108
And 208 are zoom encoders, and the zoom groups 102b and 20
The zoom ratio is determined by measuring the movement of 2b. 109 and 209
Is a focus encoder that measures the position of the focusing group.
Reference numerals 110 and 210 are video signals, and are image sensors 103.
And 203, and stored in the image memories 111 and 211.

The operations of the operation control unit 12, the correlation operation unit 13, and the correction operation unit 14 will be described later.

Next, detection of position information of a subject when taking an image with the arrangement shown in FIG. 4 will be described.

As shown in FIG. 4, the point O
, The x-axis, the y-axis, and the z-axis are defined.

[0014] Each Q _R the intersection of the optical axis 101 and 201 of the imaging optical system 102 and 202 with each of the imaging optical system, and Q _L, a point from the front side principal point of the imaging optical system (lens) 102 and 202 described above The distance to O is S ₀ , and the distance from the rear principal point to each of the image sensors 103 and 203 is S ₀ ′. Here, the derivation of the coordinates P ₂ (x ₀ , z ₀ ) on the object plane 1 in the xz plane shown in FIG. 5 will be briefly described.

[0015] Each Q _R (x ₁ the intersection of the optical axis 101 and 201 of the imaging optical system 102 and 202 with each of the imaging optical system,
_{-Z 1), Q L (-x} 1, when the -z _1), the coordinates x _1, for z ₁ by using the convergence angle θ and the photographing condition S ₀ geometrically x ₁ = S ₀ sinθ - (1-a) z ₁ = S ₀ cos θ− (1-b)

Each of the image sensors 103 and 2
03 and the intersection between the optical axis 101 and _{201 O R '(x 2,} -z
_{2), O L '(-x} 2, likewise x ₂ = true for _{-z 2) (S 0 + S} 0') sinθ - (2-
a) It can be expressed as z ₂ = (S ₀ + S ₀ ′) cos θ − (2-b).

Here, as shown in FIG. 5, each image sensor 10 with respect to an object point P ₂ (x ₀ , z ₀ ) on the subject.
3,203 to at image point _{P R '(x R, -z} R) and _{P L' (-x L, -z} L), also the in each image height on the image sensor 103 and 203 x _R ′, X _L ′,
Geometrically _{x R = (S 0 + S} 0 ') sinθ-x R' cosθ - (3-a) z R = (S 0 + S 0 ') cosθ + x R' sinθ - (3-b) x L = ( _{S 0 + S 0 ') sinθ} + x L' cosθ - (3-c) z L = (S 0 + S 0 ') cosθ-x L' sinθ - ( can be expressed 3-d) and.

[0018] a straight line passing through this time point P _R 'and the point Q _R f _{R
(S 0, S 0 ',} θ, x R') and 'a straight line passing through the point _{Q L f L (S 0,} S 0' point _{P L, θ, x L '} ) is expressed as, the object plane The upper point P ₂ (x ₀ , z ₀ ) is, by definition, the coordinates of the intersection of these two straight lines.

As shown in FIG. 4, y ₀ can also be obtained for the image pickup optical system 102 as y ₀ = y _R 'S _R ' / S _R- (4). Here, S _R is the distance from the at object point P ₂ in FIG. 5 (x _0, z ₀₎ to a front principal point of the imaging optical system 102, S _R 'are side principal point of the imaging optical system 102 From the image point P _R ′ in the image sensor 103.

At this time, the point P (x ₀ , y ₀ , z ₀ ) in FIG.
Can be expressed by P = f (S ₀ , S ₀ ′, θ, x _R ′, x _L ′, y _{R (L)} ′) (5), and the imaging conditions (S ₀ , S ₀ ′, θ) ,)
And the output image of each image sensor (x _R ′, x _L ′, y
_{R (L)} ), and the position information of the object plane can be obtained by detecting each parameter.

A method of obtaining the above process will be described with reference to FIGS. First, the convergence angle 2θ is detected by rotation angle information detecting means 105 and 205 such as a rotary encoder. Encoders (zoom encoders) 108 and 208 for obtaining position information in the optical axis direction of the respective lens groups provided in the zooming groups 102b and 202b of the imaging optical system are used. Find the focal length f. Similarly, reference numerals 109 and 209 denote focusing groups 10 of the imaging optical system.
Encoders (focus encoders) for obtaining position information in the optical axis direction of each lens group provided in the 2d and 202bd may be external members such as a potentiometer, or may be, for example, a pulse motor. The drive system itself may be a system that knows the position information of the lens in the optical axis direction by the drive method. Then, the arithmetic control unit 12 obtains a subject distance S ₀ with respect to the imaging optical systems 102 and 202 based on signals from the focus encoders 109 and 209, and further combines the focal length f of the imaging optical systems 102 and 202 with the imaging distance. The lens back S ₀ ′ of the optical systems 102 and 202 is obtained. The encoders 108, 1
09, 208, and 209, the driving system 106,
By separately controlling 107, 206, and 207,
It is assumed that the focal length f of the two sets of imaging optical systems 102 and 202 always coincides with the lens back S ₀ ′.

As described above, based on the signals of the encoders 105, 108, 109, 205, 208, and 209 provided in each mechanism system, the arithmetic control unit 12 sets S ₀ (the front main unit of the imaging optical system (lens)). Distance from the point to the intersection of each optical axis),
S ₀ ′ (distance from the rear principal point of the imaging optical system (lens) to the image plane) and θ (convergence angle) are obtained. 111 and 2 on the other hand
An image memory 11 temporarily stores video signals 110 and 210.

Reference numeral 13 denotes a correlation operation processing unit which is an image memory 1
Correlation calculations are performed on the image data 11 and 211. FIGS. 6A and 6B are conceptual diagrams of the correlation calculation processing. At the time of processing, as shown in FIG. 6B, first, a pixel data group 112 centered on the pixel data R _ij at the point (x _iR ′, y _jR ′) in the image memory 111 is formed as one block. And a correlation operation between the image and the image in the image memory 211. FIG. 7 schematically shows the relationship between the correlation value and the x '(y') axis coordinate in the correlation operation in the horizontal and vertical directions. Here, x _L ′ (y _L ′) at which the correlation value becomes maximum is obtained by approximating the relationship in the vicinity of the correlation peak by a function or the like with an accuracy equal to or less than the sampling pitch of the image sensor. A block 112 is provided for each pixel data R _{ij in} the image memory 111, and the same arithmetic processing is performed to obtain x _L ′ and y _L ′ corresponding to each pixel data.

The image information (x
_{iR ', x L', y} jR ') and the imaging conditions (S _0, S _0',
Using θ), the coordinates P _R (x ₀ , y ₀ , z ₀ ) of the subject surface corresponding to each pixel data R _ij of the image sensor 103 can be obtained from the relationship shown in the above equation (5). .

Each pixel data L _ij of the image sensor 203
(X _iL ′, y _jL ′) is subjected to the same arithmetic processing.
X _R ′ and y _R ′ corresponding to each pixel are obtained, and coordinates P _L (x ₀ , y ₀ , z ₀ ) of the object plane corresponding to each pixel data L _ij of the image sensor 203 are obtained. The coordinates P of the object plane
_R and P _L do not always match.

The correction processing section 14 performs processing based on the coordinates P _R and P _L obtained by the above processing. Correction processing unit 1
In step 4, first, a desired viewpoint position is input. Entering this time viewpoint position as shown in FIG. 8, the coordinates P _{R (L) (x} photographing condition S _0, S ₀ 'and the object plane of the _{aforementioned,}
y ₀ , z ₀ ) and the image point (x ′,
x ′ = x ₀ S ₀ ′ / (S ₀ + z ₀ ) − (6-a) y ′ = y ₀ S ₀ ′ / (S ₀ + z ₀ ) − ( 6-b) Each coordinate (x _{iR (L)} ′, y _{jR (L)} ′) of each image memory is subjected to coordinate conversion by arithmetic processing based on the above equation (6-a, b) and written to the image memory 15. . The registration in the image in the image memory 15 is corrected, and the image data is ideally doubled with respect to the image data output from the respective image sensors 103 and 203.
As a result, the output image has a high definition.

[0027]

As described above, according to the present invention, since a high-definition image can be obtained, there is an effect that high-definition information which is increasingly required from now on can be provided.

[Brief description of the drawings]

FIG. 1 is a diagram showing a basic arrangement of an embodiment of the present invention.

FIG. 2 is a perspective view showing a specific configuration of the embodiment.

FIG. 3 is a block diagram showing a relation between functions of the embodiment.

FIG. 4 is an explanatory diagram related to deriving three-dimensional position information of a subject.

FIG. 5 is an explanatory diagram relating to derivation of two-dimensional position information of a subject.

FIG. 6 is a schematic diagram of a correlation operation process.

FIG. 7 is a characteristic diagram showing a correlation value.

FIG. 8 is an explanatory diagram of an image correction process.

FIG. 9 is a schematic diagram for explaining the principle of high definition.

FIG. 10 is an explanatory diagram of a registration shift occurring in a conventional example.

[Explanation of symbols]

1 Object plane 102, 202 Imaging optical system 102a, 102b, 102c, 102d Lens group 202a, 202b, 202c, 202d of imaging optical system 102 Lens group 101, 201 of imaging optical system 202 Optical axis 103 of imaging optical system 102, 201 , 203 Image sensor 104, 204 Convergence angle motor 105, 205 Angle encoder 106, 206 Zoom motor 107, 207 Focus motor 108, 208 Zoom encoder 109, 209 Focus encoder 110, 210 Image signal 111, 211, 15 Image memory 12, 13 , 14 Operation control unit

Continuation of the front page (56) References JP-A-3-175886 (JP, A) JP-A-3-29472 (JP, A) JP-A-60-18067 (JP, A) JP-A-57-93788 (JP) JP-A-55-60381 (JP, A) JP-A-50-74815 (JP, U) (58) Fields investigated (Int. Cl. ⁶ , DB name) H04N 5/262-5/28 H04N 5/222-5/232

Claims (2)

(57) [Claims] 1. An apparatus for imaging a common subject using a plurality of imaging systems variably arranged with a convergence angle, wherein first and second memories for storing image signals respectively output from the respective imaging systems. Means for detecting an imaging condition of the imaging system; and performing a correlation operation between the image signals output from each of the imaging systems and stored in the first and second memories, thereby obtaining position coordinate information of the subject. And converting the images respectively stored in the first and second memories into corresponding position coordinates of the subject image on a pixel-by-pixel basis based on the imaging conditions and the position coordinate information. Means for generating a high-definition image by compiling by writing to an image memory of the image memory and correcting registration deviation between the images at the time of composition. Compound-eye imaging apparatus according to claim and. 2. The apparatus according to claim 1, wherein said combining means forms a combined image based on a predetermined viewpoint position.
Compound eye imaging device.