JP2006222537A - Image processing apparatus and its image processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing apparatus capable of realizing high speed reading and obtaining an image with high image quality, without making the cost increase. <P>SOLUTION: The image processing apparatus is provided, includes a correction means for correcting variations in the signals outputted from a plurality of imaging elements among output terminals of the plurality of imaging elements; and a composite means for composing the signals outputted from the plurality of imaging elements and corrected by the correction means. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image processing apparatus and an image processing method using an image sensor.

In recent years, imaging devices with a large number of pixels have become available at a lower cost in response to advances in semiconductor technology and market expansion. In addition, image sensors used in video cameras and the like are also becoming smaller and more pixels. Under such circumstances, a multi-plate type image processing apparatus has been proposed as a technique for making an inexpensive image processing apparatus with high resolution and good color reproduction. In this image processing apparatus, each output signal obtained from a plurality of image sensors is processed by a signal processing circuit, thereby obtaining a colored video signal and realizing high color reproducibility. (For example, reference 1)
JP 08-051635 A

  However, when reading the information of all pixels read by the image sensor with multiple pixels at the video rate, or when improving the capability of continuous shooting, etc., the signal from the pixel was read out with a single readout circuit In this case, it is necessary to increase the driving frequency, which raises the problem that the circuit component cost for increasing the speed is increased and the current consumption of the circuit is increased.

  In order to solve the above problem, an image processing apparatus according to the present invention includes a correction unit that corrects variation in output signals between output terminals in a plurality of imaging devices having a plurality of output terminals, and the correction unit. And a synthesizing unit that synthesizes the corrected signals from the plurality of image sensors.

  The image processing method of the present invention is an image processing method of an image processing apparatus having a plurality of image pickup devices having a plurality of output terminals, and corrects variations in output signals between the output terminals of the image pickup device. And a synthesis step of synthesizing signals from the plurality of image sensors corrected in the correction step.

  A method equivalent to high-speed reading can be realized and high-quality images can be obtained without increasing costs.

Example 1
FIG. 1 is a block diagram illustrating the configuration of the first embodiment. In the figure, reference numerals 100, 200 and 300 denote R, G and B CCD area sensors, respectively, which are divided into two. Reference numerals 101, 102, 201, 202, 301, and 302 denote photoelectric conversion units, and 103, 104, 203, 204, 303, and 304 denote horizontal transfer units. Correspondingly, they are provided so that separate video signals are processed on the left and right sides of the approximate center of the screen.

  Reference numerals 106, 107, 206, 207, 306, and 307 denote output amplifiers that amplify and output the left and right signal charges of the CCD area sensors 100, 200, and 300, respectively. 109, 110, 209, 210, 309, 310 are analog front ends that perform correlated double sampling and A / D conversion, and 111, 112, 211, 212, 311, 312 perform black level detection and black level correction. Black level correction circuits 113, 114, 213, 214, 313, and 314 are level correction circuits that adjust levels, and 116, 216, and 316 are step evaluation value generation circuits for detecting non-uniformity between the left and right channels, Reference numerals 117, 217, and 317 denote screen synthesis circuits that synthesize two image signals and generate one image. Reference numerals 115, 215, and 315 denote control circuits that control the operation of the entire video camera. The control circuits 115, 215, and 315 include a CPU such as a microcomputer, and a program memory that stores programs, data, and the like. A signal processing unit 118 outputs a signal processed here from an output terminal (not shown).

  Next, the operation of the image processing apparatus of this embodiment will be described. The subject images formed on the CCD area sensors 100, 200, 300 are converted into electric signals by the photoelectric conversion units 101, 102, 201, 202, 301, 302, and then transferred to the horizontal transfer units 103, 104, 203, The signals are divided into two systems by 204, 303, and 304 and supplied to output amplifiers 106, 107, 206, 207, 306, and 307. These output amplifiers 106, 107, 206, 207, 306, 307 convert the signal charges corresponding to the left and right imaging regions of the CCD area sensor into voltages and output the voltages. Here, the imaging signals obtained from the output amplifiers 106, 206, and 306 are referred to as R, G, and B left channel signals, and the imaging signals output from 107, 207, and 307 are referred to as R, G, and B right channel signals. I will decide.

  These left and right channel signals are subjected to correlated double sampling processing and A / D conversion by analog front ends 109, 110, 209, 210, 309, 310, respectively, and then black level correction units 111, 112, 211 for each CCD channel signal. , 212, 311, 312. The black level correction circuit for each channel uses a dummy signal portion or an optical black signal portion of each channel signal so that the black level of the imaging signal of the left and right channels respectively matches “0” of the digital data. Black level correction is performed. Thereby, the error of the offset component between the left and right channel signals is removed. After the black level is corrected, level correction units 113, 114, 213, 214, 313, and 314 perform level correction with a gain corresponding to the input signal for each of the left and right channel signals. As the gain value applied at the time of level correction, the level difference evaluation value generators 116, 216, and 316 detect the left and right level differences from the outputs from the image compositing units 117, 217, and 317, and the results are used as control units. 115 and calculated by digital calculation. Based on the calculated correction data, level correction is performed in the level correction units 113, 114, 213, 214, 313, and 314. As a result, the variation in circuit characteristics corresponding to the left and right channels, the influence of fluctuation over time, and temperature fluctuation are eliminated, and correction is performed so that the output levels for the input signals match.

  2 to 4 are diagrams showing an example of correction of a gain difference between two systems.

2A shows the signal input / output level of the left channel of the CCD 100, and FIG. 2B similarly shows the signal input / output level of the right channel of the CCD 100. The output signal level Rb ′ of the right channel is smaller than the output signal level Rb of the left channel with respect to the input light b,
Rb ′ = Rb × 0.8
Suppose that This gain difference is discriminated by the control unit 115, and as a result, a gain of 1.25 times is given to the level correction unit 113 and a gain of 1 time is given to the level correction unit 114.
Rb ″ = Rb
Thus, the difference between the left and right output is corrected, the luminance level difference due to the left and right output difference of the CCD 100 is corrected, and the difference is output from the image combining unit 117 (FIG. 2C).

3A shows the signal input / output level of the left channel of the CCD 200, and FIG. 3B similarly shows the signal input / output level of the right channel of the CCD 200. It is assumed that the output signal level Ga of the left channel with respect to the input light a is the same as the output signal level Ga of the right channel. In this case, the control unit 215 determines that there is no gain difference, and as a result, a one-time gain is given to the level correction unit 213 and a one-time gain is given to the level correction unit 214.
Ga ″ = Ba
Thus, the left and right outputs 217 of the CCD 200 are output from the image composition unit (FIG. 3C).

Next, FIGS. 4A, 4B, and 4C will be described. 4A shows the signal input / output level of the left channel of the CCD 300, and FIG. 4B similarly shows the signal input / output level of the right channel of the CCD 300. The output signal level Ba ′ of the left channel with respect to the input light a is smaller than the output signal level Ba of the right channel,
Ba ′ = Ba × 0.8
Suppose that This gain difference is discriminated by the control unit 315. As a result, a gain of 1 time is given to the level correction unit 313 and a gain of 1.25 times is given to the level correction unit 314. In the output of the gain adjustment unit,
Ba ″ = Ba
Thus, the difference between the left and right output is adjusted, and the luminance step due to the left and right output difference of the CCD 300 is corrected and output from the image combining unit 317 (FIG. 4C).

  As a result, the generated luminance signal is a signal in which the left and right level differences and the black level difference are corrected.

  As described above, it is possible to speed up reading by dividing the area of the image sensor into a plurality of parts and outputting an image signal. Further, by correcting the signals between the output units before combining the signals read from the plurality of output units, it is possible to obtain a high-quality image without requiring a complicated circuit configuration.

(Example 2)
Next, Example 2 will be described. Note that a description of the same parts as those in the first embodiment will be omitted.

  FIG. 5 is a block diagram showing the configuration of the second embodiment. The CCD 100 and the CCD 300 and the CCD 200 are spatially shifted by 0.5 pixels.

  FIG. 6 is an enlarged view of the central portion of FIG. As can be seen from FIG. 6, if the CCD 100 and the CCD 300 and the CCD 200 are spatially shifted by 0.5 pixels, and the image is output and synthesized as it is, the central divided portion is spatially equivalent to 0.5 pixels. It will shift to. That is, as shown in FIG. 9, a level difference occurs at two locations, the central portion of R and B, and the central portion of G, and image quality deterioration occurs. Therefore, in this embodiment, in order not to generate such a double step, first, the left and right levels of the signal divided into two systems and output are adjusted. Thereafter, information between pixels in the video signal of each color is supplemented by the video signal of another color. Thus, by first performing level adjustment on the two systems of outputs, it is possible to prevent image quality deterioration.

  Next, detailed operation of the image processing apparatus according to the present embodiment will be described below. The subject images formed on the CCD area sensors 100, 200, 300 are converted into electric signals by the photoelectric conversion units 101, 102, 201, 202, 301, 302, and then transferred to the horizontal transfer units 103, 104, 203, The signals are divided into two systems by 204, 303, and 304 and supplied to output amplifiers 106, 107, 206, 207, 306, and 307. These output amplifiers 106, 107, 206, 207, 306, 307 convert the signal charges corresponding to the left and right imaging regions of the CCD area sensor into voltages and output the voltages.

  The left channel signal, which is an imaging signal obtained from the output amplifiers 106, 206, 306, and the right channel signal, which is an imaging signal output from the 107, 207, 307, are analog front ends 109, 110, 209, 210, 309, respectively. After being subjected to correlated double sampling processing and A / D conversion by 310, each CCD channel signal is input to black level correction units 111, 112, 211, 212, 311 and 312. The black level correction circuit for each channel uses a dummy signal portion or an optical black signal portion of each channel signal so that the black level of the imaging signal of the left and right channels respectively matches “0” of the digital data. Black level correction is performed. Thereby, the error of the offset component between the left and right channel signals is removed. After the black level is corrected, level correction units 113, 114, 213, 214, 313, and 314 perform level correction with a gain corresponding to the input signal for each of the left and right channel signals.

  As the gain value applied at the time of level correction, the level difference evaluation value generators 116, 216, and 316 detect the left and right level differences from the outputs from the image compositing units 117, 217, and 317, and the results are used as control units. 115 and calculated by digital calculation. Based on the calculated correction data, level correction is performed in the level correction units 113, 114, 213, 214, 313, and 314. As a result, the variation in circuit characteristics corresponding to the left and right channels, the influence of fluctuation over time, and temperature fluctuation are eliminated, and correction is performed so that the output levels for the input signals match.

  For example, as shown in FIG. 7, when the horizontal direction of the pixel pitch of the CCD image sensor for each of the three primary colors of RGB is dx and the vertical direction is dy, for each pixel of the green (G) CCD image sensor. Thus, the pixels of the red (R) and blue (B) CCD image sensors are arranged so as to be shifted by dx / 2 in the horizontal direction and dy / 2 in the vertical direction. The reason why the pixels of the CCD image sensor for green are shifted with respect to those for red and blue is that when the luminance signal is generated, the contribution ratios of the green video signal and the red and blue video signals are comparable. is there. Then, interpolation processing is performed to double the number of pixels of the video signal of each color in the horizontal and vertical directions so that the number of pixels is four times as a whole. As shown in FIG. 8, this interpolation processing interpolates five pixels Sa to Se between four adjacent real pixels S (1, 1) to S (2, 2). It is processing. The sampling value of each pixel Sa to Se is calculated by performing each calculation of Formula 1 based on the sampling values of the four real pixels (1, 1) to S (2, 2).

  When the video signal of each color in which the number of pixels is quadrupled as described above is a red and blue video signal, the four actual pixels S (1,1) to S (2,2) shown in FIG. The actual pixel of the green video signal is arranged so as to overlap the position of the pixel Sc interpolated in between, and the real pixels of the red and blue video signals are the same for the pixel interpolated with the green video signal. Are placed on top of each other. Therefore, since this spatial diagonal pixel shifting method can supplement the information between pixels in each color video signal with the video signal of other colors, it is more than the case where the number of pixels is increased by a factor of 4 simply by interpolation processing. The horizontal and vertical resolution can be improved.

  As described above, the readout speed can be reduced by dividing the image sensor region into a plurality of regions and outputting image signals. In the present embodiment, the left and right pixel divisions are corrected in the blocks 111 to 117, 211 to 217, and 311 to 317 for each CCD, as shown in the figure. A step in the area is corrected by each CCD. For this reason, a deviation of 0.5 pixels does not occur in the generated luminance signal, and an image with good image quality can be obtained.

It is a block diagram which shows the structure of a 1st Example. A, B, and C are explanatory diagrams showing correction between the left and right channels. A, B, and C are explanatory diagrams showing correction between the left and right channels. A, B, and C are explanatory diagrams showing correction between the left and right channels. It is a block diagram which shows a 2nd Example. It is a figure which shows the pixel shift | offset | difference of the center part of a screen. It is a figure which shows the concept color-shifted by shifting 1/2 pixel in the horizontal / vertical direction. It is a figure explaining supplementing the information between the pixels in the video signal of each color with the video signal of another color. It is a conceptual diagram when a signal level difference occurs twice when shifting pixels.

Explanation of symbols

100, 200, 300 R, G, B CCD area sensors 101, 102, 201, 202, 301, 302 Photoelectric conversion units 103, 104, 203, 204, 303, 304 Horizontal transfer units 106, 107, 206, 207, 306, 307 Output amplifier 109, 110, 209, 210, 309, 310 Analog front end 111, 112, 211, 212, 311, 312 Black level correction circuit 113, 114, 213, 214, 313, 314 Level correction circuit 116, 216, 316 Step evaluation value generation circuit 117, 217, 317 Screen composition circuit 115, 215, 315 Control circuit 118 Signal processing unit

Claims (5)

Correction means for correcting variations in the output signal between the output terminals in a plurality of imaging devices having a plurality of output terminals;
An image processing apparatus comprising: combining means for combining signals from the plurality of image sensors corrected by the correcting means. Amplifying means for amplifying a signal from each imaging region corresponding to each output terminal;
The image processing apparatus according to claim 1, wherein the amplifying unit amplifies a plurality of signals output from each output terminal of the image sensor independently.   The image processing apparatus according to claim 1, wherein a spatial position between at least two of the plurality of image sensors has an offset. An image sensor that receives light from a lens that forms an image;
An A / D converter for A / D converting a signal from the image sensor;
The image processing apparatus according to claim 1, further comprising: a recording control unit that controls the recording unit to record the signal output from the correcting unit. An image processing method of an image processing apparatus having a plurality of image sensors having a plurality of output terminals,
A correction step of correcting variation in the output signal between the output terminals of the image pickup device, and a combining step of combining signals from the plurality of image pickup devices corrected in the correction step. An image processing apparatus.

Images