JPH06225187A - Image pickup device - Google Patents

Abstract

PURPOSE:To correct a position deviation of plural image pickup elements through signal processing. CONSTITUTION:Position deviation information of image pickup elements 3R, 3B with respect to an image pickup element 3G are stored in a ROM 9. Data of each picture element address (i, j) are read from a memory 7G and written in an address corresponding to the address (i, j) of a card 15. A control circuit 8 calculates picture element address (k, m) of the elements 3R, 3B corresponding to the picture element address (i, j) of the element 3G by using information from the ROM 9. When k, m are integral numbers, data of picture element address (k, m) are read from memories 7R, 7B and written in the address corresponding to the address (i, j) of the card 15: When the k, m are not integers, interpolation data corresponding to the address (k, m) are formed from surrounding picture element data and written in the address corresponding to the address (i, j) of the card 15. Even when position deviation is in existence, the picture element data of each color in the address corresponding to the address (i, j) of the card 15 are related to the same picture position.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image pickup apparatus in which a plurality of image pickup elements are combined and fixed, and output signals of the image pickup elements are combined to generate an image.

[0002]

2. Description of the Related Art Conventionally, since the resolution of an image pickup device using an image pickup device such as a CCD solid-state image pickup device is governed by the number of pixels of the image pickup device used, the pixel size of the image pickup device is increased in order to perform high-resolution image pickup. Higher density has been achieved. However, increasing the number of pixels of the image pickup device is limited in yield and technology, and the cost becomes high.

In recent years, image pickup devices have been manufactured in which the pixels are arranged in the same number of pixels in the highest possible density to reduce the area, increase the yield, and reduce the cost. This also enables downsizing and cost reduction of the imaging lens used. Due to such high density of pixels, the pixel pitch of the image sensor is narrowed, and for example, one pitch is 6.4 μm (when 1/3 inch size and the number of pixels in the horizontal direction is 760).
Are manufactured.

On the other hand, as a method for high-resolution image pickup using an image pickup device having the same number of pixels as in the conventional case, a prism is used to divide incident light, and the divided light flux forms an image. There is a technique in which a plurality of image pickup devices provided corresponding to the obtained image are used for image pickup, and the image pickup signals from the plurality of image pickup devices are superimposed to increase the resolution.

Further, as described in Japanese Patent Publication No. 56-40546, the spatial arrangement of n image pickup devices is arranged so as to be displaced from each other by 1 / n of the pixel pitch, and a resolution higher than the pixel pitch is obtained. A method of obtaining it has also been proposed.

[0006]

However, when arranging the image pickup devices as described above, the positional precision of each image pickup device is required to be about one digit higher than the minimum pixel pitch. Has become difficult. Also,
After the alignment, each image sensor is fixed by adhesion, screwing, or the like, but the position may shift due to the stress generated at the stage of this adhesion, screwing, or the like. Further, the fixing accuracy must be increased in accordance with the reduction of the pixel pitch due to the high density of the image pickup device, and there are problems that a sophisticated manufacturing apparatus is required and the manufacturing time is long.

Therefore, the present invention provides an image pickup apparatus capable of obtaining a good image without highly precise spatial alignment of a plurality of image pickup elements.

[0008]

According to the present invention, in an image pickup apparatus constructed by using a plurality of image pickup elements, a deviation of the plurality of image pickup elements from a reference position is corrected by signal processing by calculation.

[0009]

According to the present invention, since the deviations of the plurality of image pickup elements from the reference position are corrected by the signal processing by calculation, it is not necessary to make the spatial alignment of the plurality of image pickup elements highly accurate, and the manufacturing time can be reduced. It can be shortened, and sophisticated manufacturing equipment is not required. Further, even if a spatial displacement of a plurality of image pickup devices occurs due to changes in environmental conditions or changes with time, it can be easily corrected.

[0010]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to FIG. In this example, red obtained by imaging,
This is an example of application to an electronic still camera in which green and blue color signals are recorded in a memory card.

In the figure, 1 is an imaging lens and 2 is a color separation prism. The red, green, and blue color lights LR, LG, and LB emitted from the prism 2 are applied to the image pickup surfaces of the CCD solid-state image pickup devices 3R, 3G, and 3B, respectively. Image sensor 3
The output signals of R, 3G, and 3B are sampled and held by the sample and hold circuit 4, converted into digital signals by the A / D converter 5, and then supplied to the signal processing circuit 6.

The signal processing circuit 6 carries out signal processing such as conventionally known white balance and gamma correction. The red, green and blue color signals R, G and B for one frame output from the signal processing circuit 6 are respectively stored in the frame memories 7R and 7R.
It is supplied to G and 7B and written.

Reference numeral 8 is a control circuit for controlling the entire system, and reference numeral 9 is a writable ROM in which shift information is written during manufacturing. In the ROM 9, after the image pickup devices 3R, 3G, 3B are fixed relative to the prism 2, the shifts of the image pickup devices 3R, 3G, 3B are measured. For example, the position of the image sensor 3G is set as a reference position, and the displacement of the position of the image sensors 3R and 3B with respect to this reference position is measured.

That is, if the Cartesian coordinates set on the image sensor 3G are (x, y) and the Cartesian coordinates set on the image sensors 3R, 3B are (ur, vr), (ub, vb), the Cartesian coordinates are orthogonal. Cartesian coordinates (ur, vr), (u) with respect to coordinates (x, y)
b, vb) origin deviation (Δxr, Δyr), (Δxb, Δ
yb) and tilt angles θr, θb are measured (FIGS. 2A, B).
reference).

The displacement is measured by a measuring device (analyzing means) (not shown) after the image pickup devices 3R, 3G and 3B are fixed relatively to the prism 2 as described above. It should be noted that a test chart or the like in which a gradation pattern of a repeating pitch having a specific relationship with the pixel pitch, which is disclosed in Japanese Patent Laid-Open No. 61-288688, is provided in the horizontal and vertical directions, is imaged, and the imaging signal thereof is used by the control circuit 8. The deviation information may be written in the ROM 9 by processing the deviation amount by processing with a microcomputer or an external analysis means. In this case, a RAM may be used instead of the writable ROM 9.

Here, the Cartesian coordinates (ur, vr), (ub,
The coordinate conversion formula for converting vb) into Cartesian coordinates (x, y) is
These are represented by the equations 2 and 3, respectively.

[0017]

[Equation 2]

[0018]

[Equation 3]

Equations 2 and 3 show the positional relationship between the image pickup device 3G and the image pickup devices 3R and 3B which receive image light from the same image position. Here, for example, the address of the upper left pixel of each of the image pickup devices 3R, 3G, and 3B is set to (0, 0), and the Cartesian coordinates (x, y), (ur, vr), (u
Set the origin of b, vb) to the upper left position. As a result, when the pixel pitch in the horizontal direction of the image pickup devices 3R, 3G, 3B is PH and the pixel pitch in the vertical direction is PV, the relationship between the pixel address (i, j) of the image pickup device 3G and the coordinates (x, Y). , The relationship between the pixel address (kr, mr) of the image sensor 3R and the coordinates (ur, vr), and the relationship between the pixel address (kb, mb) of the image sensor 3B and the coordinates (ub, vb) are expressed by equations 4 to several. It is represented by 6.

[0020]

[Equation 4]

[0021]

[Equation 5]

[0022]

[Equation 6]

Reference numeral 10 is a memory control unit for reading out necessary pixel data from the frame memories 7R, 7G, 7B.

First, the pixel data of each pixel address (i, j) is sequentially read from the frame memory 7G under the control of the memory control unit 10, and the calculation unit 11 and the buffer memory 1 are operated.
3 and the memory card interface 14 to supply the data to the memory card 15, and the pixel data of the green signal G is written to the address corresponding to the pixel address (i, j) of the memory card 15.

Next, in the control circuit 8, the pixel address (kr, mr) of the image pickup device 3R corresponding to the pixel address (i, j) of the image pickup device 3G is sequentially calculated using the equations (2), (4) and (5). Calculate

When kr and mr are integers, the pixel data of the pixel address (kr, mr) is read from the frame memory 7R and supplied to the memory card 15 via the arithmetic unit 11, the buffer memory 13 and the interface 14. Then, the data is written in the address corresponding to the pixel address (i, j) of the memory card 15 as red signal pixel data.

On the other hand, when kr and mr are not integers, FIG.
As shown in FIG. 5, the interpolation data corresponding to the pixel address (kr, mr) is formed by the cubic convolution interpolation method from the pixel data of 16 pixel addresses P11 to P44 around the pixel address (kr, mr). It Equation 7 shows a cubic convolution function, D is interpolation data, D11 to D44 are pixel data of pixel addresses P11 to P44, f (y1) to f (y4) and f (x1) to f (x4). Is a coefficient.

[0028]

[Equation 7]

The coefficients f (y1) to f (y4) and f (x
1) to f (x4) are shown by the equation 8, and after this coefficient is calculated by the microcomputer in the control circuit 8, the RAM
Held at 12. In addition, in Expression 8, the symbol [] is a Gaussian symbol, which means that fractions below the decimal point are truncated, as is well known.

[0030]

[Equation 8]

Under the control of the memory controller 10, the pixel data D11 to D44 are read from the frame memory 7R and the coefficients f (y1) to f (y are read from the RAM 12.
4), f (x1) to f (x4) are read out, and the operation unit 1
In 1, the sum of products operation of Equation 7 is performed and the pixel address (k
The interpolated data D corresponding to r, mr) is calculated.

The interpolated data D is transferred to the memory card 1 via the buffer memory 13 and the interface 14.
5 and the pixel address (i,
The pixel data of the red signal is written in the address corresponding to j).

As a result, even if the image pickup device 3R is misaligned with respect to the image pickup device 3G, the pixel data of the green signal and the red signal written to the addresses corresponding to the pixel address (i, j) of the memory card 15 respectively. Are related to the same image position, and the positional deviation is corrected.

Next, in the control circuit 8, the pixel address (kb, mb) of the image pickup device 3B corresponding to the pixel address (i, j) of the image pickup device 3G is sequentially used by using equations (2), (4) and (6). Calculate

When kb and mb are integers, the pixel data of the pixel address (kb, mb) is read from the frame memory 7B and supplied to the memory card 15 via the arithmetic unit 11, the buffer memory 13 and the interface 14. Then, the pixel data of the blue signal is written in the address corresponding to the pixel address (i, j) of the memory card 15.

On the other hand, when kr and mr are not integers, the interpolated data D is calculated as in the case of the above-mentioned red signal,
This interpolated data D is supplied to the memory card 15 via the buffer memory 13 and the interface 14, and is written in the address corresponding to the pixel address (i, j) of the memory card 15 as blue signal pixel data.

As a result, even if the image pickup device 3B is misaligned with respect to the image pickup device 3G, the pixel data of the green signal and the blue signal written to the addresses corresponding to the pixel address (i, j) of the memory card 15 respectively. Are related to the same image position, and the positional deviation is corrected.

Thus, in this example, the image pickup device 3G
The positional deviations of the image pickup devices 3R and 3G with respect to the image pickup devices 3R and 3G are corrected by signal processing by calculation.
3B spatial alignment does not need to be highly accurate,
Manufacturing time can be shortened, and sophisticated manufacturing equipment is not required.

Even if the image pickup devices 3R, 3G, and 3B are displaced due to changes in environmental conditions such as temperature, RO
Just write the deviation information after the change of environmental conditions to M9,
The positional deviation can be easily corrected and the environment resistance can be improved. In this case, the deviation information under a plurality of environmental conditions may be stored in the ROM 9 in advance, and the deviation information may be switched and used according to changes in the environmental conditions. Further, the shift information can be stored in a mathematical expression using the temperature as a parameter, for example.

Further, even when the positional shifts of the image pickup devices 3R, 3G, 3B change with time, optimum correction can be performed by writing the shift information after the change with time in the ROM 9.

In the above embodiment, the interpolation data D is calculated by the dedicated calculation unit 11, but it may be calculated by the microcomputer in the control circuit 8.

Further, in the above embodiment, the image pickup device 3
Although G is used as the reference position, it is also possible to separately assume the reference position and perform correction so that all of the image pickup devices 3R, 3G, and 3B are adjusted to the separately assumed reference positions.

In the above embodiment, the image pickup device 3
Although the one having R, 3G, and 3B is shown, the present invention fixes the position shift of the image pickup device in an image pickup device which combines and fixes a plurality of image pickup devices and synthesizes output signals of the respective image pickup devices to generate an image. It can be applied when correcting.

In the above embodiment, when kr, mr and kb, mb are not integers, the interpolation data corresponding to the pixel addresses (kr, mr), (kb, mb) by the cubic convolution interpolation method. Although D is obtained, pixel data having pixel addresses close to the pixel addresses (kr, mr) and (kb, mb) may be used.

By the way, the pixel pitch of the image sensor is often not equal in the vertical and horizontal directions, but in still image data for image processing, etc., this pitch is often treated as being equal. It is necessary to interpolate the data sampled by the anisotropic image pickup device as still image data of such a processing system and convert it into the data of a square lattice with equal vertical and horizontal pitches. When this function is built into the video camera, it can be calculated at once by combining it with the positional deviation correction calculation in the above-described embodiment.

Now, the horizontal pitch of the image pickup element is m, which is the vertical pitch.
If it is doubled, making it a square lattice means expanding the image laterally by m times. Specifically, ur and ub in Equations 2 and 3 are m × ur, m ×
You can replace it with ub.

[0047]

According to the present invention, the displacement of a plurality of image pickup elements with respect to a predetermined position is corrected by signal processing by calculation, so that the spatial alignment of the plurality of image pickup elements does not need to be highly accurate, The manufacturing time can be shortened, a sophisticated manufacturing apparatus is not required, and the cost can be reduced. Further, there is an effect that even if a spatial displacement of a plurality of image pickup elements occurs due to a change in environmental conditions or a change with time, it can be easily corrected.

[Brief description of drawings]

FIG. 1 is a configuration diagram showing an embodiment of an image pickup apparatus according to the present invention.

FIG. 2 is a diagram for explaining shift information.

FIG. 3 is a diagram for explaining interpolation data by a cubic convolution interpolation method.

[Explanation of symbols]

 1 Imaging Lens 2 Color Separation Prism 3R, 3G, 3B CCD Solid State Imaging Device 4 Sample and Hold Circuit 5 A / D Converter 6 Signal Processing Circuit 7R, 7G, 7B Frame Memory 8 Control Circuit 9 Writable ROM 10 Memory Control Section 11 Arithmetic unit 12 RAM 13 Buffer memory 14 Memory card interface 15 Memory card

Claims (5)

[Claims] 1. An image pickup apparatus comprising a plurality of image pickup elements, wherein the deviation of the plurality of image pickup elements from a reference position is corrected by signal processing by calculation. 2. A first orthogonal coordinate set on the reference position is (x, y), a second orthogonal coordinate set on the image sensor is (u, v), and the second orthogonal coordinate is set. The first of the coordinates
When the deviation of the origin with respect to the Cartesian coordinate of (Δx, Δy) and the inclination angle of the second Cartesian coordinate with respect to the first Cartesian coordinate are θ, the relational expression of Formula 1 is used for the signal processing by the above calculation. The image pickup apparatus according to claim 1, wherein [Equation 1] 3. The image pickup apparatus according to claim 1, further comprising a shift detection unit that detects a shift of the plurality of image pickup elements from the reference position. 4. The image pickup apparatus according to claim 1, further comprising a storage unit that stores information about a shift of the plurality of image pickup elements from the reference position. 5. The displacement of the plurality of image pickup devices with respect to the reference position is the shift of the origin of the second rectangular coordinate set on the image pickup device with respect to the first rectangular coordinate set on the reference position. And an inclination angle of the second rectangular coordinate with respect to the first rectangular coordinate.