JP2003348610A - Imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To change a combination of a plurality of image signals used for pixel shift, so as to enhance the signal to noise ratio of a high resolution component of an image produced thereby and prevent the image quality of the entire image from being deteriorated. <P>SOLUTION: The multi-board type imaging apparatus with high performance using pixel shift to enhance the resolution switches signals from a plurality of imaging devices depending on photographing conditions for the pixel shift, so as to obtain the highest image quality at all times under every photographing condition. <P>COPYRIGHT: (C)2004,JPO

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image pickup apparatus having a plurality of image pickup devices.

[0002]

2. Description of the Related Art FIG. 5 is a block diagram showing a configuration of a conventional multi-plate camera. Since each component will be described in an embodiment to be described later, a description will be given focusing on a problem.

Reference numeral 23 denotes a block for generating a high-definition component of a luminance signal. A green signal 18 and a red signal 17 which are spatially and optically shifted from each other by a half pixel pitch are inputted and mixed. Thus, the high definition component 34 is extracted. here,
The combination of green and red is fixed and does not change. It is generally known that the total gain of the blue channel is higher than that of the red channel. In other words, the blue channel has a signal-to-noise (S / N) ratio that is half (-6 dB) if the total gain is twice. This is because, at a color temperature in a normal use state, better results in image quality can be obtained by using the red signal for pixel shift.

[0004]

However, in the above conventional example, that is, in the case where a high-definition signal is always obtained by shifting the pixels by a combination of the same signal regardless of a change in photographing conditions, the color temperature is extremely higher than the normal color temperature. When capturing an image of a displaced subject, for example, in underwater photography, blue components become extremely intense in water because long wavelength components of sunlight are absorbed in the water. Similarly, a subject against a blue sky, a subject in the shade, and the like also have a strong blue color. In such a case, the red signal component is extremely small.

[0005] Of the image signals obtained from the image pickup apparatus, high-definition components generated by pixel shift are mainly generated from red and green signals, and therefore become extremely noisy. Therefore, the image quality of the entire image is greatly reduced.

SUMMARY OF THE INVENTION The present invention has been made in view of the above-described problems, and has been made by changing a combination of a plurality of image signals used for pixel shift under a specific shooting condition as described above. It is an object of the present invention to improve the signal-to-noise ratio of a high-resolution component of an image to be processed and prevent the image quality of the entire image from being degraded.

[0007]

An image pickup apparatus according to the present invention is characterized in that a pixel shift method is used for improving resolution, and a combination of a plurality of signals based on which a signal is synthesized is changed according to a shooting situation. .

Another feature of the image pickup apparatus according to the present invention resides in that the combination of signals used for synthesizing signals by shifting pixels is changed in a shooting state where white balance is not obtained or cannot be obtained. . Furthermore, changing the combination of signals used for signal synthesis by pixel shifting
The point is that the composition ratio is continuously changed as needed according to the photographing conditions.

Another feature of the image pickup apparatus of the present invention is that the image pickup apparatus is of a multi-view type in which a plurality of optical systems are arranged in corresponding image pickup elements.

[0010]

Embodiments of the present invention will be described below with reference to the drawings. First, a multi-lens, multi-plate imaging apparatus to which the present invention is applied will be described. FIG. 2 is a conceptual diagram of a multi-lens, multi-plate image pickup apparatus. It is basically assumed that the light incident on the camera from the subject is parallel light, that is, the subject is at infinite distance.

Light 1 from a subject enters respective lenses of red (R) 2, green (G) 3, and blue (B) 4, and each of them passes through a red filter 5, a green filter 6, and a blue filter 7, respectively. The band light is selected, and after that, it is converted into an electric signal by each of the imaging elements 8, 9, and 10, and each signal of red 37, green 38, and blue 39 is obtained.

In this case, an optical system is required for each color. However, since only the primary color light beams need to be transmitted, simplification is possible without considering chromatic aberration.

Referring to the drawings, a block diagram of a basic video camera employing this multi-lens, multi-plate system will be described as an embodiment of the present invention. In FIG. 1, reference numeral 1 denotes light from a subject to be imaged, and the light enters respective optical systems (lenses) 2, 3, and 4, and thereafter, each of the three primary colors of light, red,
Only each component is extracted by the green and blue color filters 5, 6, and 7.

Next, each of the three primary colors is an image sensor 8, 9,
10 and is converted into electric signals 37, 38 and 39. The red signal 37 and the blue signal 39 are respectively VCA11,
13 is input. The VCA is an amplifier whose gain is controlled by control lines 15 and 16 from the microprocessor 14. The VCA controls the level of the red channel and the level of the blue channel by the microprocessor 14 to perform a white balance operation. Numeral 12 enlarges the signal amplitude at a predetermined amplification rate fixed by a green channel amplifier.

Next, among the outputs 17, 18, and 19 of the respective colors, signals of all the colors are input to the luminance signal matrix circuit 24. In the case of the NTSC signal, as is well known, the following equation ( The luminance signal 35 is synthesized at the ratio shown in 1),
Output Y = 0.30R + 0.59G + 0.11B (1) Y: luminance signal, R: R (red) signal, G: G (green) signal,
B: B (blue) signal

The high-definition signal generating circuit 23 receives two signals that are spatially shifted by half a pixel from each other, and combines them to generate a signal 34 that constitutes a fine detail of the image.

Incidentally, one green signal is input to the high-definition signal generating circuit 23 as a pixel-shifted signal. This is because green has the highest composition ratio of the luminance signal in the light spectrum in the natural world and has little noise.

However, the other signals that can be used are the red and blue signals which are pixel shifted with respect to green.

In this embodiment, each of the signals of red 17 and blue 19 is input to a changeover switch 21 which is a feature of the present invention, and an output 22 of which is output to a control line 20 from a microprocessor 14. Is to be selected. This operation will be described later.

Next, the luminance signal 35 synthesized by the luminance signal matrix circuit 24 is added and output by the addition circuit 25 with the high resolution signal 34 generated by the high definition signal generation circuit 23, and the signal 36 is output by the luminance signal processing circuit 26. , And undergoes processing such as gamma, knee, and white clip, and is output as a Y (luminance) signal 27.

On the other hand, the red 17, green 18, and blue 19 signals are input to a color difference signal matrix 28, and the color difference signals (R-
A Y color difference signal 29 and a BY color difference signal 28) are generated.

These color difference signals are input to the color encoder 31 together with the subcarrier 32 and output as a chroma (C) signal 33. The color difference signals 29 and 30 are also input to the microprocessor 14 and used for white balance control described later.

It should be noted that what is shown in the above embodiment is a basic block diagram for explaining the present invention,
In an actual product, an aperture is inserted between the lens and each color filter.
Many parts such as S circuits are omitted.

In the above embodiment, the signals are output separately from Y (luminance) and C (chroma). This is based on the premise that the S terminal is output. It can be obtained by adding an adder circuit.

Here, the main point of the present invention will be described in this embodiment. The above high definition signal 34 is supplied with the red signal 17 through the switching circuit 22 in addition to the green signal 18. Is controlled by a microprocessor, and switches to a blue signal 26 in the case of a specific shooting condition set in advance.

For example, underwater photography, a subject against a blue sky, a subject in the shade, and the like have an extremely strong blue component. Naturally, if the signal level of blue is much larger than the red signal, the red signal is higher. When a high-resolution signal is combined with a blue signal and a green signal instead of the above, a good image quality with improved definition and little noise can be obtained.

Therefore, by switching the above-mentioned switches and using an optimal signal under such a photographing condition, it is possible to always realize the highest image quality under all photographing conditions.

FIG. 3 is an example of a flowchart of software of the microprocessor 14 for performing the above-described control operation. As described above, in the block shown in FIG. 1, the RY color difference signal 29 and the BY color difference signal 30 are input to the microprocessor 14. The output control lines 15 and 16 are connected to the red VCA and the blue VCA, respectively, to change the levels of the red channel and the blue channel, and to take a white balance.

Step A is a subroutine for operating the white balance, that is, extracting a white color in the imaged picture and making the color difference signal thereof zero, which will be described later with reference to FIG.

Next, in step B, it is determined whether or not the white balance has converged, and if it has converged, it is determined that the imaging screen is not an unusual color such as extreme blue,
The operation here ends.

If the convergence has not occurred, the process proceeds to step C. If the blue gain controlled by the white balance operation in step A has become maximum, the process proceeds to step D.
Switches the signal for generating high definition components from the red channel to the blue channel.

FIG. 4 is a flow chart of the white balance operation subroutine shown in the above-mentioned step A. As a basic principle, the color difference signal of BY of the white area in the screen is zero, that is, red level 17 = green level 18 = Convergence operation is performed to achieve blue level 19.

In step E, a white area in the photographing screen is extracted. In step F, it is determined whether or not the RY color difference signal is not zero. , And returns to F again, and repeats until it becomes zero.

If the red gain is not maximized, that is, does not increase, the subroutine is terminated. If it is zero, the process proceeds to step H.

Here, it is determined whether or not the BY color difference signal is not zero, as in step F. If not, the blue gain is increased or decreased at I, and the process returns to step H and returns to zero. Repeat until If the blue gain does not reach the maximum, that is, does not increase, this subroutine is terminated as it is.

If zero, then RY, B
The two color difference signals of −Y become zero, the white balance converges, and the process ends.

(Other Embodiments) In order to operate various devices to realize the functions of the above-described embodiments, an apparatus connected to the various devices or a computer in a system is provided with the above-described embodiments. Supply the software program code to realize the functions of
The present invention also includes those implemented by operating the various devices according to programs stored in a computer (CPU or MPU) of the system or apparatus.

In this case, the software program code itself implements the functions of the above-described embodiment, and the program code itself constitutes the present invention. As a transmission medium of the program code, a communication medium (a wired line or a wireless line such as an optical fiber) in a computer network (WAN such as a LAN or the Internet, a wireless communication network, or the like) system for propagating and supplying the program information as a carrier wave. Etc.) can be used.

Further, means for supplying the program code to a computer, for example, a recording medium storing the program code constitutes the present invention. As a recording medium for storing such a program code, for example, a flexible disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a magnetic tape, a nonvolatile memory card, a ROM, and the like can be used.

When the computer executes the supplied program code, not only the functions of the above-described embodiment are realized, but also the OS (Operating System) or other operating system in which the program code runs on the computer. It goes without saying that such program codes are also included in the embodiments of the present invention when the functions of the above-described embodiments are realized in cooperation with the application software or the like.

Further, after the supplied program code is stored in a memory provided in a function expansion board of a computer or a function expansion unit connected to the computer, the function expansion board or the function expansion unit is specified based on an instruction of the program code. It is needless to say that the present invention also includes a case where the CPU or the like provided for performs part or all of the actual processing, and the processing realizes the functions of the above-described embodiments.

It should be noted that the shapes and structures of the respective parts shown in the above-described embodiments are merely examples of the embodiment for carrying out the present invention, and the technical scope of the present invention is thereby reduced. It should not be interpreted restrictively. That is, the present invention can be implemented in various forms without departing from the spirit or main features thereof.

[0043]

As described above, according to the present invention, in a multi-chip type image pickup apparatus which improves resolution by using pixel shift, signals from a plurality of image sensors for pixel shift are switched according to shooting conditions. Accordingly, it is possible to realize a high-performance imaging apparatus capable of always obtaining the highest image quality under all shooting conditions.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of an embodiment of an imaging apparatus according to the present invention.

FIG. 2 is a conceptual diagram of a multi-view multi-panel imaging apparatus.

FIG. 3 is a flowchart illustrating an example of an execution operation.

FIG. 4 is a flowchart illustrating an example of a white balance operation subroutine.

FIG. 5 is a block diagram illustrating a configuration of a conventional imaging apparatus.

[Explanation of symbols]

1 Light from the subject 2 Red (R) lens 3 Green (G) lens 4 Blue (B) lens 5 Red (R) color filter 6 Green (G) color filter 7 Blue (B) color filter 8 Red (R) image sensor 9 Green (G) image sensor 10 Blue (B) image sensor 11 Red VCA (Voltage Controlled Amplifier) 12 green amplifier 13. Blue VCA (Voltage Controlled Amplifier) 14 Microprocessor 15 Red white balance control line 16 Blue white balance control line 17 Red signal output 18 Green signal output 19 Green signal output 20 Changeover switch control line 21 Changeover switch 22 Changeover switch output signal 23 High Definition Component Generation Circuit 24 Luminance signal matrix circuit 25 Addition circuit 26 Luminance signal processing circuit 30 BY color difference signal 31 color encoder 32 subcarriers 33 Chroma (C) output signal 34 High-definition component signal 35 Luminance signal matrix output signal 36 Adder circuit output signal 37 Red image sensor output signal 38 Green image sensor output signal 39 Blue image sensor output signal

Claims (4)

[Claims] 1. A method of pixel shift for improving resolution,
A multi-plate type imaging apparatus characterized in that a combination of a plurality of signals based on which signals are synthesized is changed according to a shooting situation. 2. The imaging apparatus according to claim 1, wherein a combination of signals used for signal synthesis by pixel shift is changed in a shooting state in which white balance is not taken or cannot be taken out. 3. The imaging apparatus according to claim 2, wherein the change of the combination of the signals used for the signal synthesis due to the pixel shift is performed by continuously changing the composition ratio according to the photographing condition. 4. The image pickup apparatus according to claim 1, wherein the image pickup apparatus is a multi-view type in which a plurality of optical systems are arranged in corresponding image pickup devices.