JPH05344422A - Compound eye image pickup device - Google Patents

Abstract

PURPOSE:To make a picture subjected to synthesis processing from plural original pictures high precise by providing a means synthesizing a picture outputted from each image pickup system based on an image pickup condition and position information to the device. CONSTITUTION:A common object plane 1, 1st and 2nd image pickup optical systems 102, 202 having equivalent specification (a zoom lens is employed in general), image sensors 103, 203 having an equivalent specification (image pickup tube such as Saticon or solid-state image pickup element such as CCD) are arranged. Optical axes 101, 201 passes through a point 0 while being crossed at the point 0 on the object plane 1 with a tilt angle of theta symmetrical with respect to a normal 0-0' on the object plane 1. Furthermore, 2theta is referred to as a congestion angle and an image is picked up by changing the congestion angle in response to a change in an object distance S. In this case, pictures being outputs from the 1st and 2nd image pickup optical systems 102, 202 are synthesized based on an image pickup condition and position information.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image pickup apparatus, and provides one high-definition image by electrically correcting and synthesizing at least two images obtained by using at least two sets of image pickup optical systems. Related to the device.

[0002]

2. Description of the Related Art Conventionally, as a principle method for obtaining one high-definition image by synthesizing two images obtained by using two sets of image pickup optical systems, for example, the IEICE Preliminary Report 90
A method disclosed in -03-04 (p23 to 28) and the like is known.

This is because when the sampling points of the image sensors used in the two sets of image pickup optical systems are projected onto a virtually common subject, they are spatially different in phase from each other. Is intended to obtain one high-definition image. FIG. 9 shows a conceptual diagram of this principle. In FIG. 9, 811 and 821 are image pickup optical systems, and 812 and 822 are images output by the respective image pickup optical systems. In 801 the respective image signals are combined and processed to obtain a high definition image 802.

However, in the conventional example, because of the optical arrangement having the angle of convergence in principle, it is conceivable that the registration shift occurs in the image obtained from the image pickup optical system. FIG. 10A shows an outline of registration deviation in the compound-eye image pickup system. Reference numerals 910 and 920 are image pickup optical systems (lenses), and 911 and 921 are image sensors. The optical axes of the image pickup optical systems 910 and 920 are 912 and 922, respectively. Here, the central axis O-
When the subject is imaged with the optical axes 912 and 922 inclined by θ in the xz plane with respect to O ′, an arbitrary object point on the subject surface is designated as P. At this time, assuming that the image points on the image sensors 911 and 921 with respect to P are R ′ and L ′, respectively, a registration shift occurs because R ′ ≠ L ′, and as shown in FIG. In the composite image 930 obtained by simply adding to, the image for the object point P becomes double,
As a result, it has become impossible to provide highly accurate images.

[0005]

SUMMARY OF THE INVENTION An object of the present invention is to improve the definition of an image obtained by combining a plurality of original images.

[0006]

According to the present invention, in an apparatus for imaging a common subject using a plurality of image pickup systems, means for detecting the image pickup conditions of the image pickup systems and an image output from each image pickup system. It comprises means for detecting the position information of the subject from the signal, and means for synthesizing an image based on the output from each imaging system based on the imaging conditions and the position information.

[0007]

DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 1 shows the basic arrangement of a compound eye photographing system according to the present invention. In the figure, 1 is a common object plane, 102 and 202 are first and second imaging optical systems having equivalent specifications,
Generally, a zoom lens is used as described later. 10
Image sensors 3 and 203 similarly have equivalent specifications, and an image pickup tube such as a SATICON or a solid-state image pickup device such as a CCD is used. Here, for simplicity, a single plate type (or a single tube type) is schematically shown, but a two plate type (two tube type) or a three plate type (three tube type) via a color separation optical system is also general. Do not lose.

These optical axes 101 and 201 intersect at a point O on the object plane 1, pass through the point O, and pass through a normal line O of the object plane 1.
It is arranged in a state of being inclined by θ symmetrically with respect to −O ′. 2
θ is defined as the vergence angle, and the vergence angle is changed according to the change in the subject distance S, and an image is taken.

FIG. 2 shows a concrete structure of the image pickup optical systems 102 and 202, and FIG. 3 shows a function of each member constituting the image pickup optical system as a block diagram. 102a, 1
02b, 102c, 102d and 202a, 202
Reference numerals b, 202c and 202d denote the first and second imaging optical systems 1
02 and 202 show the lens groups that form the lens group, particularly 102
Reference numerals b and 202b represent a zooming group, and 102d and 202d represent a focusing group. Reference numerals 106 and 206 denote zoom groups 102b and 2
A drive system (zoom motor) for driving 02b, and 107 and 207 are drive systems (focus motors) for driving the focusing groups 102d and 202d at the same time. Further, 102 and 103, 202 and 203 are integrally provided with a mechanical system (not shown) that rotates in a plane including the optical axes 101 and 201, and drive systems (convergence angle motors) 104 and 204.

Further, 105 and 205 are angle encoders,
The rotation angles of the image pickup optical systems 102 and 202 are measured. 108
And 208 are zoom encoders, which are variable power groups 102b, 20
The movement of 2b is measured to obtain the zoom ratio. 109 and 209
Is a focus encoder that measures the position of the focusing group.
Further, 110 and 210 are video signals, and the image sensor 103
And 203 and are stored in the image memories 111 and 211.

The operation of the arithmetic control unit 12, the correlation arithmetic unit 13, and the correction arithmetic unit 14 will be described later.

Next, the detection of the position information of the subject when the image is picked up in the arrangement shown in FIG. 4 will be described.

As shown in FIG. 4, the above-mentioned point O on the object plane
The origin is defined as x-axis, y-axis, and z-axis.

[0014] Each Q _R the intersection of the optical axis 101 and 201 of the imaging optical system 102 and 202 with each of the imaging optical system, and Q _L, a point from the front side principal point of the imaging optical system (lens) 102 and 202 described above The distance to O is S ₀ , and the distance from the rear principal point to each of the image sensors 103 and 203 is S ₀ ′. Here, the derivation of the coordinates P ₂ (x ₀ , z ₀ ) on the object plane 1 in the xz plane shown in FIG. 5 will be briefly described.

[0015] Each Q _R (x ₁ the intersection of the optical axis 101 and 201 of the imaging optical system 102 and 202 with each of the imaging optical system,
_{-Z 1), Q L (-x} 1, when the -z _1), the coordinates x _1, for z ₁ by using the convergence angle θ and the photographing condition S ₀ geometrically x ₁ = S ₀ sinθ - It is expressed as (1-a) z ₁ = S ₀ cos θ − (1-b).

Further, each of the image sensors 103 and 2
03 and the intersection between the optical axis 101 and _{201 O R '(x 2,} -z
₂ ) and O _L ′ (−x ₂ , −z ₂ ) similarly, x ₂ = (S ₀ + S ₀ ′) sin θ − (2-
a) It can be expressed as z ₂ = (S ₀ + S ₀ ′) cos θ − (2-b).

Here, as shown in FIG. 5, each image sensor 10 for the object point P ₂ (x ₀ , z ₀ ) on the object.
3,203 to at image point _{P R '(x R, -z} R) and _{P L' (-x L, -z} L), also the in each image height on the image sensor 103 and 203 x _R ′, X _L ′,
Geometrically x _R = (S ₀ + S ₀ ′) sin θ−x _R ′ cos θ − (3-a) z _R = (S ₀ + S ₀ ′) cos θ + x _R ′ sin θ − (3-b) x _L = ( It can be expressed as S ₀ + S ₀ ′) sin θ + x _L ′ cos θ − (3-c) z _L = (S ₀ + S ₀ ′) cos θ−x _L ′ sin θ − (3-d).

At this time, a straight line passing through the points P _R ′ and Q _R is f _R
Let (S ₀ , S ₀ ′, θ, x _R ′) be f _L (S ₀ , S ₀ ′, θ, x _L ′) be the straight line passing through the point P _L ′ and the point Q _L. By definition, the upper point P ₂ (x ₀ , z ₀ ) becomes the coordinates of the intersection of these two straight lines.

Further, as shown in FIG. 4, y ₀ can also be calculated for the image pickup optical system 102 as y ₀ = y _R ′ S _R ′ / S _R − (4). Here, S _R is the distance from the object point P ₂ (x ₀ , z ₀ ) in FIG. 5 to the front principal point of the imaging optical system 102, and S _R ′ is the rear principal point of the imaging optical system 102. To the image point P _R ′ at the image sensor 103.

At this time, point P (x ₀ , y ₀ , z ₀ ) in FIG.
Can be expressed by P = f (S ₀ , S ₀ ′, θ, x _R ′, x _L ′, y _{R (L)} ′) (5), and the shooting conditions (S ₀ , S ₀ ′, θ 、)
And the output image of each image sensor (x _R ′, x _L ′, y
It becomes a function of _{R (L)} ), and the position information of the object surface can be obtained by detecting each parameter.

A method of obtaining the above process will be described with reference to FIGS. First, the convergence angle 2θ is detected by the rotation angle information detecting means 105 and 205 such as a rotary encoder. Encoders (zoom encoders) 108 and 208 for obtaining position information in the optical axis direction of the respective lens groups provided in the variable power groups 102b and 202b of the image pickup optical system are used. Obtain the focal length f. Similarly, 109 and 209 are focusing groups 10 of the imaging optical system.
The encoders (focus encoders) provided on the lens groups 2d and 202bd for obtaining the positional information of the respective lens groups in the optical axis direction may be external members such as potentiometers, or pulse motors such as pulse motors. A system in which the driving system itself knows position information in the optical axis direction of the lens by the driving method may be used. Then, the arithmetic and control unit 12 obtains the subject distance S ₀ with respect to the image pickup optical systems 102 and 202 from the signals from the focus encoders 109 and 209, and further combines with the focal length f of the image pickup optical systems 102 and 202 described above to obtain an image The lens back S ₀ ′ of the optical systems 102 and 202 is obtained. The encoders 108, 1
09, 208, 209 drive system 106,
By controlling 107, 206 and 207 separately,
It is assumed that the focal lengths f of the two sets of imaging optical systems 102 and 202 and the lens back S ₀ ′ are always matched.

Based on the signals of the encoders 105, 108, 109, 205, 208 and 209 provided in each mechanical system as described above, S ₀ (the front side main part of the image pickup optical system (lens)) Distance from the point to the intersection of each optical axis),
S ₀ ′ (distance from the rear principal point of the image pickup optical system (lens) to the image plane) and θ (angle of convergence) are obtained. While 111 and 2
An image memory 11 temporarily stores the video signals 110 and 210.

Reference numeral 13 denotes a correlation calculation processing unit, which is the image memory 1
Correlation calculation is performed on the image data of 11 and 211, and FIGS. 6A and 6B are conceptual diagrams of the correlation calculation process. At the time of processing, as shown in FIG. 6B, the pixel data group 112 centering on the pixel data R _ij at the point (x _iR ′, y _jR ′) in the image memory 111 is made into one block, and And the image in the image memory 211 are correlated. FIG. 7 schematically shows the relationship between the correlation value and the x '(y') axis coordinate in the correlation calculation in the horizontal direction and the vertical direction. Here, x _L ′ (y _L ′) that maximizes the correlation value is obtained with an accuracy equal to or lower than the sampling pitch of the image sensor by performing a function approximation of the relationship near the correlation peak. A block 112 is provided for each pixel data R _ij of the image memory 111, and similar calculation processing is performed to obtain x _L ′ and y _L ′ corresponding to each pixel data.

The image information (x
_iR ′, x _L ′, y _jR ′) and the photographing conditions (S ₀ , S ₀ ′,
θ) can be used to determine the coordinates P _R (x ₀ , y ₀ , z ₀ ) of the object plane corresponding to each pixel data R _ij of the image sensor 103 from the relationship shown in the above equation (5). ..

Each pixel data L _ij of the image sensor 203
Similar calculation processing is performed on (x _iL ′, y _jL ′),
X _R ′ and y _R ′ corresponding to each pixel are obtained, and the coordinates P _L (x ₀ , y ₀ , z ₀ ) of the object plane corresponding to each pixel data L _ij of the image sensor 203 are obtained. Incidentally, the coordinates P of the subject plane
_R and P _L do not necessarily match.

The correction processing unit 14 performs processing based on the coordinates P _R and P _L obtained by the above processing. Correction processing unit 1
In step 4, first, a desired viewpoint position is input. Entering this time viewpoint position as shown in FIG. 8, the coordinates P _{R (L) (x} photographing condition S _0, S ₀ 'and the object plane of the _{aforementioned,}
image point (x ′, y ₀ , z ₀ ) for each coordinate value
y ′) can be obtained as follows: x ′ = x ₀ S ₀ ′ / (S ₀ + z ₀ ) − (6-a) y ′ = y ₀ S ₀ ′ / (S ₀ + z ₀ ) − ( 6-b) The coordinates (x _{iR (L)} ′, y _{jR (L)} ′) of each image memory are coordinate-converted and written in the image memory 15 by arithmetic processing based on the equation (6-a, b). .. The registration error of the image in the image memory 15 is corrected, and the image data is ideally doubled with respect to the image data output from the image sensors 103 and 203,
As a result, the output image has a high definition.

[0027]

As described above, according to the present invention, since a high-definition image can be obtained, there is an effect that high-definition information which is required more and more can be provided.

[Brief description of drawings]

FIG. 1 is a diagram showing a basic arrangement of an embodiment of the present invention.

FIG. 2 is a perspective view showing a specific configuration of the embodiment.

FIG. 3 is a block diagram showing the relationship of functions of the embodiment.

FIG. 4 is an explanatory diagram regarding derivation of three-dimensional position information of a subject.

FIG. 5 is an explanatory diagram for deriving two-dimensional position information of a subject.

FIG. 6 is a schematic diagram of correlation calculation processing.

FIG. 7 is a characteristic diagram showing a correlation value.

FIG. 8 is an explanatory diagram of image correction processing.

FIG. 9 is a schematic diagram for explaining the principle of high definition.

FIG. 10 is an explanatory diagram of registration deviation that occurs in a conventional example.

[Explanation of symbols]

1 Object plane 102, 202 Imaging optical system 102a, 102b, 102c, 102d Lens group 202a, 202b, 202c, 202d Lens of imaging optical system 202 Optical axis 103 of imaging optical system 102, 201 , 203 image sensor 104, 204 angle of convergence motor 105, 205 angle encoder 106, 206 zoom motor 107, 207 focus motor 108, 208 zoom encoder 109, 209 focus encoder 110, 210 image signal 111, 211, 15 image memory 12, 13 , 14 Arithmetic control unit

Claims (2)

[Claims] 1. A device for imaging a common subject using a plurality of imaging systems, and means for detecting an imaging condition of the imaging system and position information of the subject from an image signal output from each imaging system. And a means for synthesizing an image depending on an output from each image pickup system based on the image pickup condition and the position information. 2. The synthesizing means forms a synthetic image based on a predetermined viewpoint position.
Compound eye imaging device.