JP2003244710A - Color imaging unit - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a color imaging unit having a extended dynamic range and providing a color image with excellent image quality. <P>SOLUTION: The color imaging unit includes: a timing generator 31 for setting an exposure time T1 and exposure times T2, T3 resulting from dividing the exposure time T1; a CCD 2 that generates an image signal for the exposure time T1 of a first light receiving section comprising a light receiving element group for two consecutive lines and at an interval of two lines and second and third image signals for the exposure times T2, T3 of the light receiving section other than the first light receiving section; and an image data composition section 52 for composing the three image signals to generate an image signal of an object. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a light receiving section comprising a plurality of light receiving elements arranged in a matrix and a color filter having a predetermined color arrangement on its front surface, and the light receiving section for a predetermined exposure time. The present invention relates to a color imaging device including a control unit that generates a color image of a subject by exposing.

[0002]

2. Description of the Related Art Instead of a conventional silver salt type camera, a CCD
Digital cameras using solid-state imaging devices such as (charge-coupled devices) have become popular in recent years. Since the dynamic range of the CCD used in this digital camera is narrower than the dynamic range of a silver salt film, various measures have been taken to expand the dynamic range of the digital camera.

For example, a method is known in which the dynamic range is substantially expanded by synthesizing a plurality of images taken continuously while changing the sensitivity of the CCD. However, in this method, the image capturing time of each image is different, and a time delay occurs between the images. Therefore, it is necessary to take an image of only a stationary subject, and it is necessary to fix the digital camera to a fixed stand such as a tripod to take an image.

Further, in the electronic camera device disclosed in Japanese Patent Laid-Open No. 7-79372, the exposure time is made different between the odd field and the even field to make the image signal of the odd field highly sensitive and at the even field. When the image signal of No. 2 is made low in sensitivity and whiteout occurs in the image signal of the odd field, the image signal of the even field adjacent to the odd field is used to interpolate the image signal in which the whiteout occurs. As a result, the dynamic range is apparently expanded to obtain an image with whiteout correction.

In the image pickup apparatus disclosed in Japanese Patent Laid-Open No. 11-220659, image signals for a plurality of fields are read from the CCD and recorded in one field period, and image signals for a plurality of fields recorded during reproduction are read and added. By doing so, the dynamic range is expanded while each CCD is kept below the saturation level.

Further, even in the image pickup apparatus disclosed in Japanese Patent Laid-Open No. 11-298801, image signals for a plurality of fields are read from the CCD and recorded in one field period or one frame period, and at the time of reproduction, a plurality of fields recorded are reproduced. By adding the image signals of, the dynamic range is expanded while each CCD is suppressed below the saturation level.

[0007]

However, in the conventional electronic camera device, since the image signal of low sensitivity is created by simply shortening the exposure time, the S / N ratio of the image signal is lowered. Therefore, even if an image signal in which overexposure has occurred is interpolated using an image signal with a low S / N ratio,
There is a limit to improving the image quality of the interpolated portion, and it becomes difficult to obtain a good image.

Further, since each conventional image pickup device adds the image signals of the respective fields in the exposure period which is deviated in time, the image is blurred and it is not easy to obtain a good image. In particular, when a moving object is photographed or a digital camera is handheld for photographing, the blurring of the image becomes more remarkable, and the image quality of the image deteriorates.

Therefore, the applicant of the present invention utilizes the first exposure time, which is the proper exposure time, and the second and third exposure times created by dividing the appropriate exposure time, and makes an odd number in the first exposure time. The first image signal is obtained from the line, the second and third image signals are obtained from the even-numbered lines at the second and third exposure times, respectively, and the fourth image signal obtained by adding the second and third image signals is obtained. An image pickup apparatus has been proposed that uses the overexposure part of the first image signal to detect the overexposure part (Japanese Patent Application No. 2001-091861).

This image pickup apparatus uses the fourth image signal obtained by adding the image signals obtained within the overlapping exposure times, and when the whiteout portion of the first image signal is detected, the whiteout portion is detected. Is interpolated, the dynamic range can be expanded, and an image having good image quality can be obtained.

On the other hand, when capturing a color image, C
On the front side (subject side) of the CD, color filters (primary color filters or complementary color filters) arranged in color based on the Bayer method or the like are arranged. For example, when a color image is picked up by the above-mentioned image pickup apparatus using primary color filters arranged in color based on the Bayer method, the second and third image signals do not include all three primary colors. (Specifically, the R color or B color signal is not included). Therefore, since the first image signal is interpolated using the fourth image signal obtained by adding the second and third image signals, it cannot be said that the interpolation accuracy is sufficient for a color image, and the image quality is low. There was a problem of decrease.

The present invention has been made in view of the above problems, and an object of the present invention is to provide an image pickup apparatus capable of expanding a dynamic range and obtaining a color image having good image quality.

[0013]

According to a first aspect of the present invention, there is provided an image pickup device comprising: a plurality of light receiving elements arranged in a matrix; and a light receiving portion provided on the front surface thereof with a color filter having a predetermined color arrangement, and a predetermined light receiving element. And a control unit configured to generate a color image of a subject by exposing the light receiving unit for an exposure time of, the control unit sets the first exposure time and the first exposure time to each other. Split second and third
And an exposure time setting means for setting the exposure time of the light receiving element, and the first light receiving portion in the first light receiving section which is a light receiving element group for a plurality of consecutive lines of the light receiving elements and is formed by a plurality of light receiving element groups arranged in the same plurality of lines. The first image signal is generated from the accumulated charges of each light receiving element during one exposure time, and the second image is generated at the second light receiving portion of the light receiving element that is formed of a light receiving element group other than the first light receiving portion. Second image signal is generated from the charge accumulated in each light receiving element during the exposure time of, and the third image signal is generated from the charge accumulated in each light receiving element during the third exposure time in the second light receiving unit.
Image generating means for generating the image signal of, and the first and second
And an image synthesizing unit that synthesizes the third image signal to generate an image signal of the subject.

According to the above arrangement, the exposure time setting means sets the first exposure time and the second and third exposure times obtained by dividing the first exposure time. Accumulation of each light receiving element in the first exposure time in the first light receiving unit, which is a light receiving element group for a plurality of consecutive lines of the light receiving element and is formed by a group of light receiving elements arranged on the same plurality of lines, by the image generating means. A first image signal is generated from the electric charge, and a second image is generated from the accumulated electric charge of each light receiving element at the second exposure time in the second light receiving section including the light receiving element group other than the first light receiving section in the light receiving element. Image signal is generated, and the third image signal is generated from the charge accumulated in each light receiving element during the third exposure time in the second light receiving unit. Then, the image synthesizing unit synthesizes the first, second and third image signals to generate an image signal of the subject.

At this time, the second and third exposure times are the first
Since the second and third exposure times are shorter than the first exposure time, the second and third image signals are compared with the first image signal. It is difficult to saturate.

Furthermore, since the second and third exposure times are created by dividing the first exposure time, the second and third exposure times overlap in time with the first exposure time. Therefore, the first image signal and the second and third image signals become image signals which are photographed almost at the same time, and the second and third image signals are image signals without blurring with respect to the first image signal. Become.

Further, the first image signal is generated from the accumulated charge of each light receiving element in the first light receiving section, which is a light receiving element group for a plurality of continuous lines and is formed by a plurality of light receiving element groups arranged in the same plurality of lines. , And the second and third image signals are generated from the accumulated charges of each light receiving element in the second light receiving portion of the light receiving element, which is a group of light receiving elements other than the first light receiving portion, and therefore, for example, Bayer Even when a color image is captured using the primary color filters that are color-aligned based on the method, each of the first, second, and third image signals includes all three primary colors.

Therefore, the image signal of the subject is generated by synthesizing the first, second and third image signals having different exposure times and including all primary colors (or complementary colors) An image signal having a wide range and good quality can be obtained.

An image pickup device according to a second aspect of the present invention is the color image pickup device according to the first aspect, in which the image generating means includes:
The signal charges photoelectrically converted by the first light receiving unit during the first exposure time are read out as a first image signal, and the signal charges photoelectrically converted by the second light receiving unit during the second exposure time are read out. It is characterized in that it is provided with a signal reading means for reading out as a second image signal and reading out, as a third image signal, a signal charge photoelectrically converted by the second light receiving section during the third exposure time.

According to the above arrangement, the signal read-out means reads out the signal charges photoelectrically converted by the first light receiving portion as the first image signal by the signal reading means, and reads out the signal charges by the signal exposure means during the second exposure time. The signal charges photoelectrically converted by the second light receiving unit are read out as the second image signal,
The signal charges photoelectrically converted by the second light receiving unit during the third exposure time are read out as the third image signal. Therefore, the first, second, and third image signals are generated by reading out the signal charges, so that the first, second, and third image signals are easily generated in a short time.

An image pickup device according to a third aspect of the present invention is the color image pickup device according to the first or second aspect, wherein the synthesizing means corresponds to the second light receiving section by using the first image signal. A fifth interpolating means for interpolating an image signal corresponding to the first light receiving section using the second interpolating means for interpolating an image signal to generate a fourth image signal. And a third interpolating means for generating a sixth image signal by interpolating an image signal corresponding to the first light receiving portion using the third image signal, Fourth,
It is characterized in that the image signal of the subject is generated by synthesizing the fifth and sixth image signals.

With the above arrangement, the first interpolating means interpolates the image signal corresponding to the second light receiving portion using the first image signal to generate the fourth image signal. Further, the second interpolating means interpolates the image signal corresponding to the first light receiving section using the second image signal to generate the fifth image signal. Further, the third interpolating means interpolates the image signal corresponding to the first light receiving portion using the third image signal to generate the sixth image signal. Then, the synthesizing unit synthesizes the fourth, fifth, and sixth image signals to generate an image signal of the subject.

Therefore, since the fourth, fifth and sixth image signals are respectively interpolated and generated from the first, second and third image signals containing all the primary colors (or complementary colors), Interpolation accuracy becomes good, and an image signal with good image quality can be obtained.

Further, as described above, the second and third image signals are image signals which have no blurring with respect to the first image signal, and therefore are generated by being interpolated from the second and third image signals, respectively. The fifth and sixth image signals that are generated are image signals that have no blur with respect to the fourth image signal that is generated by being interpolated from the first image signal.

Therefore, the image signal of the subject, which is generated by combining the fourth, fifth and sixth image signals, is obtained by combining three image signals with good image quality and different exposure times. The image signal has a wide dynamic range and good image quality.

An image pickup device according to a fourth aspect is the image pickup device according to any one of the first to third aspects.
4. The color image pickup apparatus according to any one of 3 to 3, wherein the second exposure time is 0.3 to 0.9 times the appropriate exposure time, and the third exposure time is approximately 1. It is characterized by being 0 times.

According to the above arrangement, the second exposure time is 0.3 to 0.9 times the proper exposure time, and the third exposure time is about 1.0 times the proper exposure time. The first exposure time, the second exposure time, and the third exposure time are each 1.3 to the appropriate exposure time.
1.9 times, 0.3 to 0.9 times and 1.0 times, the first
First image signal generated from the accumulated charge of the light receiving element during the exposure time of, the second image signal generated from the accumulated charge of the light receiving element during the second exposure time, and the light receiving element during the third exposure time The third image signal generated from the accumulated electric charge of is an image signal having an appropriate exposure time.
Therefore, the image signal of the subject generated by combining the first, second, and third image signals becomes an image signal with a wide dynamic range.

An image pickup device according to claim 5 is the color image pickup device according to claim 1 or 2, wherein the second exposure time is shorter than the third exposure time, and the synthesizing means is configured to First interpolation means for generating a fourth image signal by interpolating an image signal corresponding to the second light receiving section using a first image signal; and the first interpolation means using the second image signal. Second interpolating means for interpolating an image signal corresponding to the light receiving portion to generate a fifth image signal, and synthesizing the fourth and fifth image signals to generate an image signal of a subject. I am trying.

According to the above arrangement, the first interpolating means interpolates the image signal corresponding to the second light receiving section using the first image signal to generate the fourth image signal. Further, the second interpolating means interpolates the image signal corresponding to the first light receiving section using the second image signal to generate the fifth image signal. Then, the synthesizing unit synthesizes the fourth and fifth image signals to generate an image signal of the subject.

Therefore, since the fourth and fifth image signals are generated by being interpolated from the first and second image signals containing all the primary colors (or complementary colors), the interpolation accuracy becomes good. The obtained image signal also becomes an image signal with good image quality.

Further, as described above, since the second image signal is an image signal which has no blur with respect to the first image signal,
The fifth image signal generated by being interpolated from the image signal of 1 becomes an image signal having no blur with respect to the fourth image signal generated by being interpolated from the first image signal.

Therefore, since the image signal of the object generated by combining the fourth and fifth image signals is obtained by combining two image signals having good image quality and different exposure times, the dynamic range is increased. The resulting image signal has a wide image quality.

[0033]

DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 1 is a block diagram showing the configuration of the main part of a digital camera which is an embodiment of the present invention.

In the digital camera shown in FIG. 1, a lens 1 for collecting light from a subject and a plurality of light receiving elements are arrayed, light from the subject is photoelectrically converted by each light receiving element, and the converted charges are converted. CCD (charge coupled device) for storage
2 (corresponding to image generating means), a CCD driving unit 3 that drives the CCD 2, a memory 4 that temporarily stores an image signal captured by the CCD 2, and a predetermined process for the image signal stored in the memory 4. An image data processing unit 5 for performing the operation, an overall control unit 6 that outputs a control signal to each unit that configures the camera, an operation unit 71 that receives an operation from the outside, a liquid crystal display (LCD), and the like. The display unit 72 for displaying the information of 1., the memory 8 for temporarily storing the image data, and the recording unit 9 configured by a memory card recorder or the like for storing the compressed image data.

The image pickup lens 1 is, for example, an electric zoom lens, and is provided with a lens driving section 11 for performing a focusing operation and a zooming operation of the image pickup lens 1. At the image forming position of the light flux on the optical axis L of the image pickup lens 1, a CCD 2 is provided via a diaphragm and a mechanical shutter 12 for exposure control. Although not shown, an optical low-pass filter, an infrared cut filter, an ND filter for adjusting the amount of light, or the like is provided as necessary. A color filter 13 having a predetermined color arrangement is arranged on the front surface (subject side) of the CCD 2.

The CCD 2 is an interline transfer type CCD in which light receiving elements PE _{i, j} (i = 1 to M, j = 1 to N) formed of photodiodes are arranged in a matrix. Further, the CCD 2 performs a predetermined operation according to a control signal SC1 described later output from the timing generator 31 (corresponding to a part of the signal reading unit and the exposure time setting unit) to photoelectrically convert incident light, Further, the accumulated charges are output as an image signal to the signal processing unit 32.

The color filter 13 is a Bayer (B
It is a primary color filter having a color array of an ayer system.
FIG. 2 is an explanatory diagram of the color arrangement of the color filter 13. Figure 2
First, as shown in,
The G (green) color filter, which has a large contribution to
R (red) and B (blue) color filters lined up and the rest
Are arranged in a checkerboard pattern. In addition, each color filter
Is the light receiving element PE of CCD2_{i, j}Array at the position corresponding to
(Or the light receiving element PE of CCD2)_{i , j}When
It is configured as one).

Here, the CCD 2 will be described in detail. FIG. 3 is a block diagram showing the configuration of the CCD 2.
The CCD 2 includes a light receiving element PE _{i, j} and vertical transfer units 211 to 2
1N (corresponding to a part of the signal reading unit), a horizontal transfer unit 22 (corresponding to a part of the signal reading unit), and an output unit 2
3 (corresponding to a part of the signal reading means).

The light-receiving element PE _{i, j} is a light-receiving element group for two consecutive lines (two rows), and the light-receiving element group PE _{i, j} every two lines (two rows) (a multiple of i = 4 + ( 1 or 2), a first light receiving portion PEA composed of j = 1 to N), and a light receiving element group PE _{i, j} other than the first light receiving portion PEA (a multiple of i = 4 + (3 or 0), j = 1 to N) and a second light receiving portion PEB.

Each light receiving element PE _{i, j} is composed of a photodiode and photoelectrically converts the light incident during the exposure time,
The vertical transfer unit 211 stores the signal charge accumulated according to the amount of incident light.
Transfer to 21N at once.

The vertical transfer units 211 to 21N are shown in FIG.
The signal for one light receiving element is shown in the shaded area and the blank area, respectively.
It has the ability to transfer charges. That is, the vertical transfer unit 2
11 to 21N are for each light receiving element (M / 2)
(Where M is the light receiving element PE_{i, j}Number of lines)
It can be transferred to the horizontal transfer unit 22.

The vertical transfer units 211 to 21N serially transfer the transferred signal charges to the horizontal transfer unit 22. The horizontal transfer unit 22 serially transfers the transferred signal charges to the output unit 23. The output unit 23 outputs an image signal according to the transferred signal charge.

Next, the CCD 2 constructed as described above
The operation of will be described. The signal charges accumulated in each light receiving element PE _{i, j} (i = 4 multiples + (1 or 2), j = 1 to N) arranged in the first light receiving unit PEA are transferred to the vertical transfer units 211 to 21N. To the vertical transfer unit 211.
21N, the signal charges for two lines are transferred to the horizontal transfer unit 22.
Are sequentially transferred to. Next, the transferred signal charges of two lines are sequentially transferred to the output unit 23 by the horizontal transfer unit 22, and one pixel is output from the output unit 23 as an image signal of A field (corresponding to the first image signal) described later. Are output one by one.

Here, the method of transferring the signal charges from the respective light receiving elements PE _{i, j} to the vertical transfer portions 211 to 21N will be described in more detail. Here, for convenience, the vertical transfer unit 211
Will be described.

First, in response to the shift pulse SG1a from the timing generator 31, the first light receiving portion PEA is received.
Of the light receiving elements PR _{i, 1} (i =
A signal charge of a multiple of 4 + 1) is transferred to the vertical transfer unit 211 (hatched portion in FIG. 3). Next, in response to the vertical transfer pulse SG1b from the timing generator 31, the signal charges transferred to the vertical transfer unit 211 are moved downward by one light-receiving element in the downward direction of FIG. 3 (direction of the horizontal transfer unit 22) (see FIG. 3). Shift from the shaded area to the white area).

Then, in response to the shift pulse SG1c from the timing generator 31, the first light receiving portion PE
Of A, the light receiving element PR _{i, 1} (i
= A multiple of 4 + 2) signal charges are transferred to the vertical transfer unit 211 (FIG. 3).
(Shaded area). Thus, in the vertical transfer unit 211, the light receiving elements PR _{i, 1} (multiples of i = 4 + 1, j = 1 to 1) corresponding to the R color filter of the first light receiving unit PEA are provided.
N) signal light and the light receiving element PR corresponding to the G color filter
The signal charges of _{i, 1} (multiple of i = 4 + 2, j = 1 to N) are arranged alternately.

Next, the two vertical lines (light receiving element PR arranged by the vertical transfer units 211 to 21N are arranged as described above.
_The signal charges _{i, j} (corresponding to i = 4 multiples + 1) and the light receiving elements PR _{i, j} (i = 4 multiples + 2) are sequentially transferred to the horizontal transfer unit 22.

The shift pulse SG1 (see FIG.
(See) is a shift pulse SG1a and a vertical transfer pulse S
G1b and shift pulse SG1c are collectively described.

Each light receiving element PE _{i, j} (i = 4 multiples + (1 or 2), j) arranged in the first light receiving portion PEA.
= 1 to N), each light receiving element PE _{i, j} (i = 4 multiples + (3 or 0)) arranged in the second light receiving unit PEB at a timing different from the transfer timing of the signal charge accumulated in , J
= 1 to N), the signal charges stored in the vertical transfer units 211 to 21N and the horizontal transfer unit 22 are transferred in the same manner as described above, and the image signals of the B or C field (second and third images) to be described later. (Corresponding to a signal) is output from the output unit 23 pixel by pixel.

As described above, the image signals of the A, B, and C fields are a group of light receiving elements for two consecutive lines (two rows), and a group of light receiving elements every two lines (two rows) (first line). The image signal corresponds to the signal charge from the light receiving portion or the second light receiving portion). As the CCD 2, another type of CCD can be used as long as the above-described operation is possible.

Returning to FIG. 1, the signal processing unit 32 is the CCD.
The signal output from 2 is processed by correlated double sampling processing or A
Signal processing such as / D conversion processing is performed and the result is output to the memory 4 as a digitized image data signal.

The memory 4 comprises an A field memory 41 for storing an image of the A field, a B field memory 42 for storing an image of the B field, and a C field memory 43 for storing an image of the C field.

The image data processing section 5 reads out the image data stored in the A, B and C fields and performs a predetermined interpolation process on the image data interpolation section 51 (corresponding to the first, second and third interpolation means). Image data combining unit 52 (corresponding to an image combining unit) that combines the interpolated three image data to generate one image data, and the image data obtained by the combination is R (red). , G (green) and B (blue) 3
A white balance processing unit (WB) 53 that adjusts the white balance for each primary color; and a γ correction unit 54 that performs a predetermined γ correction process on the image data whose white balance has been adjusted and stores it in the memory 8 for each color. The color difference matrix processing unit 55, which converts the signals of the respective colors of R, G, and B included in the γ-corrected image data (subject image data) into predetermined luminance signals and color difference signals and outputs the signals to the display unit 72 and the like, JPEG (Joint Photographic Experts Group) for recording the brightness signal and the color difference signal on the recording medium 91.
An image data compression unit 56 that performs compression processing such as
2 is provided with an image data decompression unit 57 which reads out the compressed luminance signal and color difference signal from the recording medium 91 for decompressing the image data and displays the decompressed signal.

Interpolation processing performed by the image data interpolating section 51 will be described with reference to FIG. Here, the case of interpolating the image signal of the A field will be described. That is, by using the image signal corresponding to the signal charge from the first light receiving unit PEA, which is a light receiving element group for two consecutive lines (two rows) and a light receiving element group every two lines (two rows), A case of interpolating an image signal corresponding to the second light receiving unit PEB which is another light receiving element group will be described. Here, the image signal QE _{i, j} (i = 1 to M, j = 1
Up to N), the light receiving elements PE _{i, j} (i = 1 to M, j = 1 to
The image signal corresponds to the signal charge of N).

Image signal Q corresponding to the second light receiving portion PEB
E_{i, j}(I = multiple of 4+ (3 or 0), j = 1 to N)
Corresponding to the R and B color filters of the color filter 13
Position image signal QE_{i, j}Is close to the position of interest.
Six surrounding image signals QE_{i , j}To interpolate. Ingredient
Physically, the interpolation is performed using the following equation (1). QE_{i, j}= (QE_{i-2, j-2}+ QE_{i-2, j}+ QE_{i-2, j + 2}+ QE_{i + 2, j-2}+ QE_{i + 2, j}+ QE_{i + 2, j + 2}) / 6 (1)

Image signal Q corresponding to the second light receiving portion PEB
E _{i, j} (i = multiple of 4 + (3 or 0), j = 1 to N)
Among these, the image signal QE _{i, j at} the position corresponding to the G color filter of the color filter 13 is interpolated using the four image signals QE _{i, j} around which the distance is closer to the position of interest. Specifically, the interpolation is performed using the following formula (2) or (3). QE _{i, j} = (QE _{i-1, j-1} + QE _{i-1, j + 1} + QE _{i-2, j} + QE _{i + 2, j} ) / 4 (2) QE _{i, j} = (QE _{i + 1 , j-1} + QE _{i + 1, j + 1} + QE _{i-2, j} + QE _{i + 2, j} ) / 4 (3)

Interpolation processing is applied to the image signals of the B and C fields in the same manner. Note that here, the interpolation is performed by simply averaging the image signals QE _{i, j} , but the interpolation may be performed using other methods.
For example, interpolation may be performed by performing a weighted average using preset weights. Further, the image signal QE _{i, j at} the position corresponding to the R and B color filters of the color filter 13 is
Six surrounding image signals QE that are close to the target position
The image signal QE _{i, j at} the position corresponding to the G color filter of the color filter 13 is interpolated using _{i, j,} and four surrounding image signals QE _{i, j} close to the target position are used. Although the interpolation is performed, the number of pixel signals used for the interpolation is not limited to the above and can be appropriately selected.

Next, the image data synthesizing unit 5 will be described with reference to FIG.
2 describes the image data composition process
Reveal Here, the image data interpolating unit 51 interpolates
If the generated image data is 10 bits for each pixel,
The case will be described. Corresponding to the image signal of A field
10-bit image data QEA after interpolation_{i, j}And B
10-bit image after interpolation corresponding to the field image signal
Image data QEB_{i, j}And C field image signals
10-bit image data QEC after interpolation_{i, j}And add
Calculated, 12-bit image data QED_{i, j}To get Was
However, i = 1 to M, j = 1 to N, and the above calculation is paired.
This is performed for each pixel at the corresponding position.

In the present embodiment, a case has been described in which the image data after interpolation corresponding to the image signals of the A, B, and C fields is added to be combined, but another method may be used for combining. Good. For example, A, B
Alternatively, the image data after interpolation corresponding to the image signal of the C field may be weighted and averaged using a predetermined weight set in advance.

Returning to FIG. 1 again, the overall control unit 6 responds to a signal from the operation unit 71 operated from the outside, and the lens drive unit 11, the mechanical shutter 12, the CCD drive unit 3, the image data processing unit. 5, the recording unit 9, the photometric unit 10 and the like are controlled.

The operation unit 71 includes a power switch, a release button, a mode switching switch for recording / reproduction, a frame advance switch for image reproduction, a zoom switch, etc., and is provided in the digital camera and operated from the outside. Switch group.

The memory 8 stores the image data of each color which has been γ corrected by the γ correction unit 54, and the stored image data is output to the color difference matrix processing unit 55. Further, although not shown, related information such as the image pickup date and time is also recorded in the recording medium 91 by the recording unit 9 as the header information of the image file together with the image data.

The photometry unit 10 performs photometry (other distance measurement) as a shooting preparation operation by pressing the release button halfway, and sets an appropriate aperture value and an appropriate exposure time from the photometric value. In this embodiment, the exposure times T1 and T3 are set based on the proper exposure time T2 as shown in FIG. 6 described later.

Power supplies for supplying electric power to the respective switch groups and respective parts are not shown. Further, each unit constituting the above digital camera includes a CPU,
It is assumed to be composed of ROM, RAM and the like.

Next, the operation of the digital camera configured as described above will be described. FIG. 6 is a timing chart for explaining the operations of the CCD 2 and the timing generator 31 of the digital camera shown in FIG. The control pulse SUB and shift pulses SG1 and SG2 shown in FIG.
This is a signal output as a control signal SC1 to CD2.

First, when a release operation is performed by pressing a release button (not shown) from the outside, the mechanical shutter 12 is opened at an opening degree corresponding to the set aperture value. In FIG. 6, a signal for opening and closing the mechanical shutter 12 is shown as a mechanical shutter signal MS. It indicates that the mechanical shutter 12 is in a closed state while the mechanical shutter signal MS is at a low level, and the mechanical shutter signal MS is at a high level. This shows that the shutter 12 is in the open state.

[0067] In the above state, the timing generator 31, to precisely control the set exposure time, generates a control pulse SUB at time t _a.
In response to the generated control pulse SUB, the first light receiving portion PEA and the second light receiving portion PEB of the CCD 2, that is,
The residual charges accumulated in all the light receiving elements PE _{i, j} are discharged (initialized). The mechanical shutter signal MS is changed from low level to high level in synchronization with the control pulse SUB. Therefore, the time t _a the exposure time T1, T
As the starting point of 2, the exposure of the first light receiving portion PEA and the second light receiving portion PEB is started, and each light receiving element PE _{i, j} (i = 4 multiples + (1 or 2), each of the signal charges AC stored in j = 1 to N) and each light receiving element PE _{i, j} (i
= Multiple of 4+ (3 or 0), j = 1 to N), the amount of the signal charge BC accumulated increases with time (charges are accumulated).

Next, the timing generator 31 shifts the shift pulse S at time t _b when the exposure time T2 has elapsed.
G1 is generated. In response to the generated shift pulse SG1, the signal charge of each light receiving element PE _{i, j} forming the second light receiving portion PEB of the CCD 2 is transferred to the vertical transfer portions 211 to 21N.
To the vertical transfer unit 211 every two lines.
21N is transferred to the horizontal transfer unit 22, and the first image signal BFD (B-field image signal) of the second light receiving unit PEB is read. As a result, as the image signal RD output from the output unit 23, the first image signal BFD of the second light receiving unit PEB is output. At the same time, new exposure is started for the second light receiving unit PEB from time t _b .

Next, when the mechanical shutter signal MS changes from the high level (open state) to the low level (closed state) at time t _c when the exposure time T3 has elapsed, and the mechanical shutter 12 is closed, After the exposure of the first and second light receiving portions is completed, the signal charges of the respective light receiving elements PE _{i, j} of the second light receiving portion PEB become the signal charges accumulated during the exposure time T3, and the first light receiving portion PEA Each light receiving element P of
The signal charge of E _{i, j} becomes the signal charge accumulated during the exposure time T1.

Then, the timing generator 31
The shift pulse SG1 is generated at time t _d when the reading of the first image signal BFD from the second light receiving unit PEB is completed. In response to the generated shift pulse SG1, C
The signal charge BC of each light receiving element PE _{i, j} of the second light receiving unit PEB of CD2 is transferred to the vertical transfer units 211 to 21N, and is also transferred from the vertical transfer units 211 to 21N to the horizontal transfer unit 22 every two lines. , The second image signal CFD (image signal of C field) of the second light receiving unit PEB is read. As a result, the image signal R output from the output unit 23
As D, the second image signal CFD of the second light receiving unit PEB
Is output.

Next, the timing generator 31
The shift pulse SG2 is generated at the time t _e when the reading of the second image signal CFD from the second light receiving unit PEB is completed. C in response to the generated shift pulse SG2
The signal charge AC of each light-receiving element PE _{i, j} of the first light-receiving portion PEA of CD2 is transferred to the vertical transfer portions 211 to 21N, and is also transferred from the vertical transfer portions 211 to 21N to the horizontal transfer portion 22 every two lines. , The image signal AFD (image signal of the A field) of the first light receiving unit PEA is read.
As a result, the image signal AFD of the first light receiving unit PEA is output as the image signal RD output from the output unit 23.

In this embodiment, the exposure time T2 is 0.5 times the proper exposure time, and the exposure time T3 is 1.
It is assumed that it is preset to 0 times (as a result, the exposure time T1 is set to about 1.5 times the proper exposure time, as described later). Therefore, the first image signal B of the second light receiving portion PEB formed at the exposure time T2
It is possible to make the FD less saturated than the image signal AFD of the first light receiving unit PEA formed at the exposure time T1 by about three times.

The exposure times T2 and T3 are the exposure times T
1 is divided and created, and the shift pulse SG1
The pulse width of is sufficiently short, and if this pulse width is ignored,
Since T1 = T2 + T3, the exposure time T1 of the first light receiving portion PEA and the exposure times T2 and T3 of the second light receiving portion PEB overlap in time. Therefore, the first and second image signals BFD and CFD of the second light receiving unit PEB,
The image signal PEA of the first light receiving portion PEA becomes an image signal captured almost at the same time, and the first image signal PEA of the second light receiving portion PEB is obtained.
And the second image signals BFD and CFD to the first light receiving portion P.
An image signal having no blur can be obtained with respect to the image signal PEA of EA.

Furthermore, after the first image signal BFD of the second light receiving portion PEB is read out, the second image signal CFD of the second light receiving portion PEB is read out, and the second light receiving portion PEB is read out.
Since the image signal AFD of the first light receiving unit PEA is read after the second image signal CFD of
Each image signal is read in the order of shorter exposure time, and the influence of noise generated during the period from the end of exposure to the reading can be reduced.

Since the exposure time T2 is controlled by the CCD 3 by the electronic shutter operation, the shortest exposure time T
2 can be controlled accurately and the exposure time T
Since the end points of T1 and T3 are controlled by the mechanical shutter operation of the mechanical shutter, the exposure times T1 and T3
Can be terminated at the same time to block the light incident on the first light receiving portion PEA and the second light receiving portion PEB, and a sufficient read time can be secured to secure the first and second light receiving portions PEB. Image signals BFD, CFD and first light receiving portion PE
The image signal AFD of A can be sequentially read.

The exposure times T1, T2, T3 are not particularly limited to the above example, and various times can be used, but the combined image (subject image) signal is regarded as an image signal having a wide dynamic range. For this reason, the exposure time T2 is preferably 0.3 to 0.9 times the proper exposure time, and the exposure time T3 is preferably about 1.0 times the proper exposure time. This is because the exposure times T1, T2 and T3 are appropriately different from each other with the appropriate exposure time as the center.
Further, the exposure times T2 and T3 may be set so that the exposure time T3 is shorter than the exposure time T2. In this case, since the image signal of the proper exposure time (the first and second image signals BFD of the second light receiving portion PEB) becomes an image signal close to the timing of the release operation, the subject image in which the shutter timing is prioritized The signal is obtained.

The first and second image signals BFD and CFD of the second light receiving portion PEB and the image signal AFD of the first light receiving portion PEA read out as described above are correlated by the signal processing portion 32. After being subjected to the double sampling processing, it is converted into, for example, 10-bit digital data, and the first and second image data BFD and CF of the second light receiving unit PEB are processed.
D and the image data AFD of the first light receiving unit PEA are output.

Next, the 10-bit image data AFD of the first light receiving portion PEA is stored in the A field memory 4 of the memory 4.
1 of the 10-bit second light receiving unit PEB
Image data BFD is stored in the B field memory 42, and the second image data CFD of the 10-bit second light receiving unit PEB is stored in the C field memory 42.

Then, by the image data interpolating section 51,
Image data AF from A field memory 41, B field memory 42 and C field memory 43, respectively.
D, BFD, and CFD are read out and subjected to interpolation processing to generate three image data corresponding to the entire screen. Then, the image data synthesizing unit 52 performs a synthesizing process on the three generated image data to generate one image data.

Next, the white balance processing section (WB)
The white balance of the image data is adjusted by 53, and the γ correction unit 54 adjusts the image data by a predetermined γ.
The image data is corrected, converted into image data having a desired γ characteristic, and stored in the memory 3 as subject image data.

When the subject image is output to the display unit 72, the color difference matrix processing unit 55 converts the R, G, and B color signals included in the subject image data into predetermined luminance signals and color difference signals. The display 72
Is output to. The converted luminance signal and color difference signal are compressed by the image data compression unit 56 and stored in the recording medium 91. When the image data stored in the recording medium 91 is displayed on the display unit 72 as an image, the image data decompression unit 57 reads and decompresses the image data and outputs it to the display unit 72.

As described above, the first image data BFD and the normal sensitivity of the second light-receiving portion PEB, which have low sensitivity (low S / N ratio) but are difficult to saturate, which are photographed at the same time with temporal overlap. Second image data C of the second light receiving portion PEB of
Using the FD and the image signal AFD of the high-sensitivity (high S / N ratio) first light-receiving unit PEA, a single color still image having a wide dynamic range and good image quality (subject image) Can be obtained.

That is, in the present embodiment, the exposure time T
2 is set to 0.5 times the appropriate exposure time, and the exposure time T1
Is set to about 1.5 times the appropriate exposure time, and therefore the first image signal BFD of the second light receiving unit PEB is less likely to be saturated about three times as much as the image signal AFD of the first light receiving unit PEA.

Further, the exposure time T1 and the exposure times T2 and T3
Since and are temporally overlapped, the first and second image signals BFD and CFD of the second light receiving unit PEB and the image signal AFD of the first light receiving unit PEA are image signals captured almost at the same time. Therefore, the image signal AFD of the first light receiving unit PEA
First and second of the second light receiving portion PEB without blurring with respect to
The image signals BFD and CFD can be obtained.

Further, the exposure times T1, T2 and T3 are respectively about 1.5 times, 0.5 times and 1.0 times the proper exposure time, and the first image signal BFD of the second light receiving portion PEB,
The second image signal CFD of the second light receiving portion PEB and the image signal AFD of the first light receiving portion PEA are image signals having appropriately different exposure times. Therefore, the image signal of the subject, which is generated by interpolating these three image signals and then combining, is an image signal having a wide dynamic range.

The present invention can take the following forms.

(A) In the present embodiment, the case has been described in which the first and second light receiving portions are light receiving element groups for two consecutive lines and light receiving element groups for every two lines, but they are continuous. The light receiving element group for a plurality of lines and the light receiving element groups for the same plurality of lines may be used.

(B) In this embodiment, A, B, C
The case where the image signal of the field is a group of light receiving elements for two consecutive lines and the image signal corresponds to the signal charges from the group of light receiving elements every two lines has been described, but at least the image signal of the B field is continuous. The light receiving element group for two lines may be any image signal corresponding to the signal charge from the light receiving element group every two lines. That is, the A and C field image signals may be, for example, image signals obtained by general interlaced reading. In this case, for example, the A'field may be an odd line and the C'field may be an even line. However, in this case, a functional unit that distributes the image signal is required between the memory 4 and the image data interpolating unit 51 or between the signal control unit 32 and the memory 4.

The function of the signal distribution unit will be specifically described in the case where the signal distribution unit for distributing the image signal is provided between the memory 4 and the image data interpolation unit 51. Here, image data REA _{i, j} (i = 1 to M odd numbers, j = 1 to 1) corresponding to the image signals of the A ′, B ′, and C ′ fields.
N), REB _{i, j} (a multiple of i = 4 + (3 or 0), j
= 1 to N) and REC _{i, j} (i = 1 to M even number, j = 1)
To N) are stored in the A field memory 41, the B field memory 42, and the C field memory 43, respectively.

The signal distribution unit displays the image of the A'field.
Image data REA corresponding to the signal_{i, j}Of the first light received
Image data REA from the light receiving element included in the PEA section
_{i, j}(I = multiple of 4 + 1, j = 1 to N), and C'feed
Image data REC corresponding to the image signal of the field_{i, j}Of
Image data from the light receiving element included in the first light receiving unit PEA.
TAREC_{i, j}(I = multiple of 4 + 2, j = 1 to N)
It is read from the memory 4 and output to the image data interpolation unit 51.
It The data interpolator 51 uses these image data
An image data synthesizing unit that performs the same interpolation processing as this embodiment
To the exposure time T1 at the first light receiving portion PEA supplied to 52
Image data based on the image signal (first image signal) in
To generate.

Further, the signal distribution unit reads out the image data REB _{i, j} corresponding to the image signal of the B ′ (= B) field from the memory 4 and outputs it to the image data interpolation unit 51. The data interpolating unit 51 performs the same interpolation processing as in the present embodiment using these image data, and supplies the image data synthesizing unit 52 with the exposure time T2 at the second light receiving unit PEB.
Image data is generated based on the image signal (second image signal) in.

Further, the signal distribution unit, among the image data REA _{i, j} corresponding to the image signal of the A ′ field, the image data REA _{i, j} (i = i = _j A multiple of 4 +3, j = 1 to N), and C ′
Image data REC _{i, j} corresponding to the field image signal
Of the image data REC _{i, j} from the light receiving element included in the second light receiving unit PEB (i = 4 multiples + 0, j = 1 to N)
And are read from the memory 4 and output to the image data interpolation unit 51. The data interpolating unit 51 performs the same interpolation processing as in the present embodiment using these image data, and supplies the image signal to the image data synthesizing unit 52 at the image signal (third image) at the exposure time T3 in the second light receiving unit PEB. Image data based on the signal).

(C) In the present embodiment, A, B, C
The case where three pieces of full-face image data obtained by interpolating the image signals of each field are combined has been described.
It is also possible to combine two full-screen image data obtained by interpolating the image signals of the A and B fields, respectively. In this case, in order to widen the dynamic range of the combined image signal, the exposure time T1 is set to 1.5 of the proper exposure time.
.About.2.5 times, and the exposure time T2 is about 0.
It is preferably 5 times. The exposure times T1 and T2 are
This is because the exposure times are appropriately different from each other around the appropriate exposure time.

(D) In this embodiment, A, B, C
The case where three pieces of full-face image data obtained by interpolating the image signals of each field are combined has been described.
A form in which the image signals of the B and C fields are tone-synthesized and the image signals of the A field are interpolated and synthesized may be used.

An example of this case will be described more specifically. Image signals of B and C fields, that is, first image data BFD (for example, 10-bit data) of the second light receiving unit PEB and second image data CFD (for example, 10-bit data) of the second light receiving unit PEB. Image data DFD (11-bit data) of the second light receiving unit PEB is generated by performing gradation combination with (data). Next, the image signal of the A field, that is, the image data A of the first light receiving unit PEA.
FD (for example, 10-bit data) is doubled to generate 11-bit image data AFD2. Then, the image data DFD of the second light receiving unit PEB (11-bit data)
And the image data AFD2 (11-bit data) of the first light receiving unit PEA are interpolated and combined to obtain full-face image data.

[0096]

According to the first aspect of the present invention, the first, second and third images in which the image signals of the subject have different exposure times and include all primary colors (or complementary colors) Since the signals are generated by being combined, it is possible to obtain an image signal of good quality with a wide dynamic range.

According to the second aspect of the invention, the first, second and third image signals are generated by reading out the signal charges, so that the first, second and third image signals are easily and quickly obtained. 3 image signals can be generated.

According to the third aspect of the present invention, the image signal of the object generated by synthesizing the fourth, fifth and sixth image signals is three image signals with good image quality and different exposure times. Is obtained by synthesizing, the image signal having a wide dynamic range and good image quality can be obtained.

According to the fourth aspect of the invention, since the first, second and third image signals are image signals having appropriately different exposure times, the image signal of the subject generated by combining them is generated. Can be an image signal with a wide dynamic range.

According to the fifth aspect of the present invention, since the image signal of the subject is generated by synthesizing the fourth and fifth image signals having good image quality and different exposure times, the image signal having a wide dynamic range is obtained. It is possible to obtain a good image signal of.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a main part of a digital camera according to an embodiment of the present invention.

FIG. 2 is an explanatory diagram of a color array of color filters.

FIG. 3 is a block diagram showing a configuration of a CCD.

FIG. 4 is an explanatory diagram of interpolation processing performed by an image data interpolation unit.

FIG. 5 is an explanatory diagram of image data combining processing.

6 is a timing chart for explaining the operations of the CCD and the timing generator of the digital camera shown in FIG.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 lens 10 photometry part 11 lens drive part 12 mechanical shutter 13 color filter 2 CCD (image generation means) 211 to 21N vertical transfer part (a part of signal reading means) 22 horizontal transfer part (a part of signal reading means) 23 output Part (part of signal reading means) 3 CCD driving part 31 Timing generator (part of signal reading means and exposure time setting means) 32 Signal processing part 4 Memory 5 Image data processing part 51 Image data interpolating part (first, first) 2 and third interpolation means) 52 image data composition section (image composition section) 53 white balance processing section 54 γ correction section 55 color difference matrix processing section 56 image data compression section 57 image data decompression section 6 overall control section 71 operation section 72 display Section 8 memory 9 recording section

   ─────────────────────────────────────────────────── ─── Continued front page    (72) Inventor Manji Takano             2-3-3 Azuchi-cho, Chuo-ku, Osaka             Kokusai Building Minolta Co., Ltd. F-term (reference) 5C024 BX01 CX03 CX47 CX53 DX01                       DX04 EX31 GY04 HX14 HX28                 5C065 BB22 BB48 CC01 DD07 EE05                       GG13 GG21 GG30

Claims (5)

[Claims] 1. A light receiving section comprising a plurality of light receiving elements arranged in a matrix and a color filter having a predetermined color array on the front surface thereof, and a light receiving section of a subject by exposing the light receiving section for a predetermined exposure time. In a color imaging apparatus including a control unit that generates a color image, the control unit sets an exposure time that sets a first exposure time and a second and a third exposure time obtained by dividing the first exposure time. A setting means; and a light-receiving element group for a plurality of continuous lines of the light-receiving elements, the light-receiving element group having the same plurality of lines
Of the light receiving elements of the second light receiving element group other than the first light receiving element in the light receiving element A second image signal is generated from the charge accumulated in each light receiving element at the second exposure time in the light receiving unit, and the second image signal is generated from the charge accumulated in each light receiving element at the third exposure time in the second light receiving unit. 3. A color image pickup apparatus comprising: an image generating unit that generates a third image signal; and an image synthesizing unit that synthesizes the first, second, and third image signals to generate a subject image signal. 2. The image generating means reads out, as a first image signal, signal charges photoelectrically converted by the first light receiving section during the first exposure time, and reads out the second electric charge during the second exposure time. Signal reading means for reading out the signal charge photoelectrically converted by the light receiving section as a second image signal, and reading the signal charge photoelectrically converted by the second light receiving section as a third image signal during the third exposure time. The color image pickup apparatus according to claim 1, further comprising: 3. The first synthesizing means for interpolating an image signal corresponding to the second light receiving section using the first image signal to generate a fourth image signal; Second interpolating means for interpolating an image signal corresponding to the first light receiving section using the second image signal to generate a fifth image signal;
Third interpolating means for interpolating an image signal corresponding to the first light receiving portion using the third image signal to generate a sixth image signal, and the fourth, fifth and sixth The color image pickup apparatus according to claim 1, wherein the image signals are combined to generate an image signal of a subject. 4. The second exposure time is 0.3 to 0.9 times the proper exposure time, and the third exposure time is about 1.0 times the proper exposure time. Claim 1
3. The color imaging device according to any one of 3 above. 5. The second exposure time is shorter than the third exposure time, and the synthesizing unit interpolates an image signal corresponding to the second light receiving unit using the first image signal. First interpolating means for generating a fourth image signal, and second interpolating means for interpolating an image signal corresponding to the first light receiving portion using the second image signal to generate a fifth image signal. 3. The color image pickup apparatus according to claim 1, further comprising: and generating the image signal of the subject by combining the fourth and fifth image signals.