JP4105556B2 - Digital camera and recorded image data recording method - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a digital camera that captures both a low-sensitivity image and a high-sensitivity image and a captured image data recording method thereof. In particular, the captured image data is converted to RAW data (raw image data output from a solid-state image sensor). The present invention relates to a digital camera and captured image data recording method for recording in a memory with raw data.
[0002]
[Prior art]
In digital cameras such as digital still cameras and digital video cameras, the saturation amount of charge accumulated in the photodiodes that make up each pixel of the solid-state image sensor decreases as the number of pixels increases, that is, the photodiodes become smaller. Has a drawback that the dynamic range becomes narrow.
[0003]
In order to overcome this drawback, for example, in the conventional technique described in Japanese Patent Application Laid-Open No. 2001-8104, a solid-state image sensor is provided with two types of pixels, a high-sensitivity pixel and a low-sensitivity pixel, and a captured image obtained from the high-sensitivity pixel. The dynamic range of the captured image is expanded by combining the data and the captured image data obtained from the low sensitivity pixels.
[0004]
On the other hand, when the number of pixels of a solid-state image sensor increases as in recent years, an image equivalent to that of a silver salt camera is picked up. The image data is saved in raw memory as raw data without white balance correction, gamma correction, JPEG compression, etc., and this image data is read by a personal computer etc., and white balance correction, gamma correction, color tone correction, etc. Users' desire to do according to their own preference is increasing. For this reason, digital cameras record image data as RAW data as described in JP-A-11-261933 (Patent Document 2) and JP-A-2001-2231979 (Patent Document 3). Is increasing.
[0005]
[Patent Document 1]
Japanese Patent Laid-Open No. 2001-8104 [Patent Document 2]
JP-A-11-261933 [Patent Document 3]
[Patent Document 1] Japanese Patent Application Laid-Open No. 2001-223979
[Problems to be solved by the invention]
As described above, in the digital camera that captures both the low-sensitivity image and the high-sensitivity image, when the captured image data is recorded in the memory as RAW data, both the captured image data of the low-sensitivity image and the captured image data of the high-sensitivity image are recorded. Will be recorded. However, even if the user reads low-sensitivity image data and RAW data of high-sensitivity image data from the memory and composes the image using application software on a personal computer, the composition process does not go well, and a composite image with the desired image quality is obtained. There is a problem that can not be.
[0007]
SUMMARY OF THE INVENTION An object of the present invention is to provide a digital camera capable of easily and quickly obtaining a synthesized image of desired image quality even when synthesizing RAW data of low sensitivity image data and high sensitivity image data outside the camera, and its captured image data. It is to provide a recording method.
[0008]
[Means for Solving the Problems]
The digital camera of the present invention to achieve the above object, a solid-state imaging device each of the individual pixels provided on a semiconductor substrate is divided asymmetrically and the low-sensitivity pixels of the high-sensitivity pixel and the small area of the large area, RAW data includes an optical system placed in front of the solid-state imaging device, a driving mechanism of the optical system, high-sensitivity image data obtained from the high-sensitivity pixels, and low-sensitivity image data obtained from the low-sensitivity pixels. And recording on the recording medium as image correction parameter data the sensitivity ratio between the high-sensitivity pixels and the low-sensitivity pixels according to the optical system at the time of shooting and the driving state of the optical system. Means.
[0009]
With this configuration, when the user performs image composition processing by reading high-sensitivity image data and low-sensitivity image data RAW data from a recording medium, the camera itself refers to the value of an image correction parameter that is automatically determined to be optimal for image composition. Therefore, it is possible to easily and quickly generate a composite image with good image quality.
[0010]
The driving state of the optical system and the optical system of the digital camera of the present invention, the zoom position of the lens and the lens used in shooting, and wherein the focus position, an aperture position.
[0011]
With this configuration, an image low-sensitivity pixels and the high-sensitivity pixels are imaged by changing the incident angle of changing the focus position and aperture or zoom the lens in solid-state imaging device provided on the asymmetric to each pixel of the it is possible to product the R AW data and easily synthesized high-quality image quickly.
[0014]
Image data recording method for a digital camera of the present invention, a solid-state imaging device each of the individual pixels provided on a semiconductor substrate is divided asymmetrically and the low-sensitivity pixels of the high-sensitivity pixel and the small area of the large area An image data recording method for a digital camera comprising an optical system placed in front of the solid-state image sensor and a drive mechanism for the optical system, wherein the high-sensitivity image data obtained from the high-sensitivity pixels and the low-sensitivity image data The low sensitivity image data obtained from the sensitivity pixel is recorded on the recording medium as RAW data, and the sensitivity ratio between the high sensitivity pixel and the low sensitivity pixel according to the optical system at the time of photographing and the driving state of the optical system Is recorded on the recording medium as image correction parameter data.
[0015]
With this configuration, when the user performs image composition processing by reading high-sensitivity image data and low-sensitivity image data RAW data from a recording medium, the camera itself refers to the value of an image correction parameter that is automatically determined to be optimal for image composition. Therefore, it is possible to easily and quickly generate a composite image with good image quality.
[0016]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
[0017]
FIG. 1 is a configuration diagram of a digital still camera according to an embodiment of the present invention. In this embodiment, a digital still camera will be described as an example. However, the present invention can also be applied to other types of imaging devices such as a camera mounted on a small electronic device such as a digital video camera or a mobile phone. Further, the image synthesis processing of the present embodiment is executed by software by the digital signal processing unit 26 described later, but this can also be realized by a hardware circuit.
[0018]
The digital still camera shown in FIG. 1 includes a photographic lens 10, a solid-state image sensor 11, a diaphragm 12 provided between them, an infrared cut filter 13, and an optical low-pass filter 14. The CPU 15 that controls the entire digital still camera controls the light emitting unit 16 and the light receiving unit 17 for flash, controls the lens driving unit 18 to adjust the position of the photographing lens 10 to the focus position, and stops the aperture driving unit. The opening amount of the diaphragm 12 is controlled via 19 to adjust the exposure amount to an appropriate exposure amount.
[0019]
Further, the CPU 15 drives the solid-state image sensor 11 via the image sensor driving unit 20 and outputs the subject image captured through the photographing lens 10 as a color signal. In addition, a user instruction signal is input to the CPU 15 through the operation unit 21, and the CPU 15 performs various controls according to the instruction.
[0020]
The electric control system of the digital still camera includes an analog signal processing unit 22 connected to the output of the solid-state imaging device 11, and an A / D conversion that converts RGB color signals output from the analog signal processing unit 22 into digital signals. The circuit 23 is provided and these are controlled by the CPU 15.
[0021]
Further, the electric control system of this digital still camera includes a memory control unit 25 connected to the main memory 24, a digital signal processing unit 26 described later in detail, and compresses the captured image into a JPEG image and decompresses the compressed image. Mounted on the back side of the camera, the compression / decompression processing unit 27, the integration unit 28 for integrating the photometric data and adjusting the white balance gain, the external memory control unit 30 to which the removable recording medium 29 is connected. And a display control unit 32 to which the liquid crystal display unit 31 is connected. These are connected to each other by a control bus 33 and a data bus 34, and are controlled by a command from the CPU 15.
[0022]
The digital signal processing unit 26, the analog signal processing unit 22, the A / D conversion circuit 23, and the like shown in FIG. 1 can be mounted on the digital still camera as separate circuits. It is preferable to manufacture on the same semiconductor substrate using LSI manufacturing technology to form one solid-state imaging device.
[0023]
FIG. 2 is a pixel arrangement diagram of the solid-state imaging device 11 used in the present embodiment. The pixel 1 in the CCD portion that captures an image with a wide dynamic range has a pixel arrangement described in, for example, Japanese Patent Laid-Open No. 10-136391, and each pixel in the odd row is horizontally aligned with each pixel in the even row. A vertical transfer path (not shown) that is arranged with a ½ pitch shift and transfers signal charges read from each pixel in the vertical direction is meandered so as to avoid each pixel in the vertical direction. ing.
[0024]
In the illustrated example, each pixel 1 according to the present embodiment includes a low-sensitivity pixel (sub-pixel) 2 that occupies about 1/5 of the area of the pixel 1 and a high-sensitivity pixel (subpixel) that occupies the remaining about 4/5. Main signal) 3 so that the signal charge of each low-sensitivity pixel 2 and the signal charge of each high-sensitivity pixel 3 can be distinguished and transferred to the vertical transfer path. . It should be noted that the ratio and the position at which the pixel 1 is divided are determined by design, and FIG. 2 is merely an example.
[0025]
In the imaging apparatus according to the present embodiment, a low-sensitivity image (an image obtained by the low-sensitivity pixel 2) and a high-sensitivity image (an image obtained by the high-sensitivity pixel 3) are simultaneously acquired by one imaging, and each image is acquired. Are sequentially read out from the pixels 2 and 3, and the details are combined as described later, or are recorded as they are on the recording medium 29 as RAW data.
[0026]
The solid-state imaging device 11 has been described by taking a CCD having a honeycomb pixel arrangement as shown in FIG. 2 as an example, but may be a Bayer CCD or a CMOS sensor.
[0027]
FIG. 3 is a diagram for explaining the operation of the digital still camera described above. An analog high-sensitivity image signal H output from the high-sensitivity pixel 3 of the solid-state image sensor 11 is converted into, for example, 10-bit digital data by the A / D converter 23 and output from the low-sensitivity pixel 2 of the solid-state image sensor 11. The analog low-sensitivity image signal L is converted into, for example, 8-bit digital data by the A / D converter 23.
[0028]
When the user sets the completed image recording mode with the operation unit 21, the digital signal processing unit 26 captures the 10-bit high sensitivity image data and the 8-bit low sensitivity image data output from the A / D converter 23. The external memory control unit (recording circuit) 30 performs, for example, JPEG received from the digital signal processing unit 26 via the compression / decompression processing unit 27 (not shown in FIG. 3). The compressed completed image data is recorded on the recording medium 29.
[0029]
When the user sets the RAW data recording mode on the operation unit 21, the external memory control unit 30 outputs the 10-bit high sensitivity image data and the 8-bit low sensitivity image data output from the A / D converter 23. The data is recorded on the recording medium 29 as it is. At this time, the external memory control unit 30 records the value of an image correction parameter described later together with the RAW data.
[0030]
FIG. 4 is a processing configuration diagram of the digital signal processing unit 26 that operates in the completed image recording mode. The digital signal processing unit 26 employs a logarithmic addition method in which high-sensitivity image signals and low-sensitivity image signals are subjected to gamma correction and then subjected to addition processing, and the high-sensitivity image output from the A / D conversion circuit 23 is used. An offset correction circuit 41a that takes an RGB color signal that is a digital signal and performs an offset process, a gain correction circuit 42a that white balances an output signal of the offset correction circuit 41a, and a gamma correction for the color signal after gain correction. A gamma correction circuit 43a for performing, an offset correction circuit 41b for performing an offset process by taking in RGB color signals which are digital signals of low-sensitivity images output from the A / D conversion circuit 23 shown in FIG. 1, and an offset correction circuit 41b The gain correction circuit 42b for white balance of the output signal and the color signal after gain correction And a gamma correction circuit 43b which performs Ma correction. When linear matrix processing or the like is performed on the signal after offset correction, it is performed between the gain correction circuits 42a and 42b and the gamma correction circuits 43a and 43b.
[0031]
The digital signal processing unit 26 further interpolates the RGB color signal after the image composition by performing an image composition process by taking both output signals of each of the gamma correction circuits 43a and 43b and performing image composition processing. Reduces noise from the RGB interpolation calculation unit 45 for obtaining RGB three-color signals, the RGB / YC conversion circuit 46 for obtaining the luminance signal Y and the color difference signals Cr and Cb from the RGB signal, and the luminance signal Y and the color difference signals Cr and Cb. A noise filter 47, a contour correction circuit 48 that performs contour correction on the luminance signal Y after noise reduction, and a color difference matrix circuit 49 that performs color correction by multiplying the color difference signals Cr and Cb by the color difference matrix. Prepare.
[0032]
When the high-sensitivity image data and the low-sensitivity image data are combined in units of pixels by the digital signal processing unit 26, the image processing is performed based on the sensitivity ratio between the high-sensitivity pixels 3 and the low-sensitivity pixels 2. However, the solid-state imaging device 11 used in this embodiment, since as can be seen from Figure 2, in which the high-sensitivity pixels (main pixel) 3 and 2 low-sensitivity pixels (sub-pixels) are arranged in asymmetrical lenses Depending on the zoom position, the focus position, and the aperture position, the incident angle of light to the pixels 2 and 3 changes, and the sensitivity ratio between the high-sensitivity pixel 3 and the low-sensitivity pixel 2 changes. In addition, the multi-group lens 10 also has individual differences and has different characteristics for each camera, which affects the change in the sensitivity ratio.
[0033]
In order to absorb the change in the sensitivity ratio, an appropriate gain may be applied to each detection signal of the high-sensitivity pixel 3 and the low-sensitivity pixel 2 in the signal processing performed in units of pixels. However, the gain depends on the zoom position, focus position, and aperture position at the time of shooting. Therefore, the CPU 15 reads information on the zoom position, focus position, and aperture position at the time of shooting, and table data of gain values (main memory 24 and other memory not shown) set in advance for each color of R, G, and B. The R, G, and B gain values corresponding to the zoom position, gain position, and aperture position are read out and passed to the digital signal processing unit 26.
[0034]
As a result, the digital signal processing unit 26 multiplies these gain values by the detection signals for R, G, and B of the high sensitivity pixel (main pixel) 3 and the low sensitivity pixel (sub pixel) 2, and then performs image synthesis. And a composite image with good image quality can be created.
[0035]
In the completed image recording mode, as described above, the camera itself generates a composite image with an optimum image quality by the digital signal processing unit 26. In the RAW data recording mode, the high sensitivity output from the A / D converter 23 is used. Image data and low-sensitivity image data are recorded on the recording medium 29 as they are, so that the user reads RAW data into a personal computer and adjusts various image composition parameters to obtain a composite image of a desired image quality. Repeat trial and error.
[0036]
When the user composes an image, if the data recorded on the recording medium 29 is only the RAW data of the high-sensitivity image data and the low-sensitivity image data, the sensitivity ratio described above is unknown. If it is not repeated many times, an image with good image quality cannot be created.
[0037]
Therefore, in the present embodiment, the above-described gain values for R, G, and B of the sensitivity ratio of the high sensitivity pixel 3 and the low sensitivity pixel 2 that are used when the digital still camera generates a complete image are written in the recording medium 29. , So that the user can refer to it when performing the image composition work.
[0038]
FIG. 5 is a format diagram of an image file when RAW data is recorded on the recording medium 29. The image file includes a header portion and an image data portion. The header portion includes file information 51 such as a file number, model information 52 of a digital still camera and a lens used for photographing, an image correction parameter 53, And shooting ancillary information 54 such as an exposure amount and a shutter speed when the image is taken. The image data portion includes RAW data 55 for high-sensitivity images and RAW data 56 for low-sensitivity images.
[0039]
FIG. 6 is a detailed configuration diagram of the image correction parameter 53. In this example, the image correction parameter includes an R gain value 61 of a high-sensitivity pixel (referred to as “main pixel” in FIG. 6) 3 corresponding to the zoom position, focus position, and aperture position of the lens used during shooting. The same G gain value 62 of the main pixel 3, the B gain value 63 of the same main pixel 3, the R gain value 64 of the same low sensitivity pixel (referred to as “sub-pixel” in FIG. 6) 2, and the like The G gain value 65 of the sub-pixel 2 and the B gain value 66 of the similar sub-pixel 2 are configured.
[0040]
When a user reads out the RAW data 55 and 56 and performs image composition, referring to these gain values 61 to 66 can reduce the number of trial and error in image composition work and create an image with good image quality. It becomes.
[0041]
In the present embodiment, as shown in FIG. 6, the R, G, and B gain values 64, 65, and 66 of the sub-pixel 2 are used as the image correction parameters, but these gain values 64, 65, and 66 are used. 7, ratios 64 ′, 65 ′, and 66 ′ of R, G, and B gain values of the main pixel and the sub-pixel may be used as the image correction parameters, as shown in FIG.
[0042]
In the above-described embodiment, the digital still camera having the two modes of the RAW data recording mode and the completed image recording mode has been described. However, the present invention can be applied to a digital camera not equipped with the completed image recording mode. Applicable.
[0043]
【The invention's effect】
According to the present invention, an image correction parameter automatically determined by a camera when recording image data of a digital still camera equipped with a solid-state image sensor that outputs high-sensitivity image data and low-sensitivity image data on a recording medium as RAW data. Since the image is recorded together, when the user performs image composition work, the composition processing can be performed with reference to the value of the parameter for image correction, and a composite image with good image quality can be obtained easily and quickly. It becomes.
[Brief description of the drawings]
FIG. 1 is a block diagram of a digital still camera according to a first embodiment of the present invention.
FIG. 2 is a schematic surface view of the solid-state imaging device shown in FIG.
3 is an operation explanatory diagram of the digital still camera shown in FIG. 1. FIG.
4 is a processing configuration diagram of a digital signal processing unit shown in FIG. 3. FIG.
5 is a format diagram of an image file when captured image data is recorded on a recording medium as RAW data by the digital still camera shown in FIG. 1. FIG.
6 is a detailed view of the image correction parameters shown in FIG.
7 is a detailed view of image correction parameters in place of FIG. 6. FIG.
[Explanation of symbols]
1 pixel 2 low sensitivity pixel (sub pixel)
3 High sensitivity pixels (main pixels)
DESCRIPTION OF SYMBOLS 10 Lens 11 Solid-state image sensor 12 Aperture 18 Lens drive part 19 Aperture drive part 23 A / D converter 26 Digital signal processing part 29 Recording medium 30 External memory control part 44 Image composition processing circuit 53 Image correction parameter

Claims (3)

A solid-state imaging device each of the individual pixels provided on the semiconductor substrate is divided asymmetrically and the low-sensitivity pixels of the high-sensitivity pixel and the small area of the large area, an optical system placed in front of the solid-state image capturing device And a drive mechanism of the optical system, high-sensitivity image data obtained from the high-sensitivity pixels, and low-sensitivity image data obtained from the low-sensitivity pixels are recorded on a recording medium as raw data and the optical at the time of shooting A digital camera comprising: a recording unit configured to record a sensitivity ratio between the high-sensitivity pixel and the low-sensitivity pixel according to a driving state of the system and the optical system on the recording medium as image correction parameter data.   2. The digital camera according to claim 1, wherein the optical system and the driving state of the optical system are a lens used for photographing, a zoom position, a focus position, and a diaphragm position of the lens. A solid-state imaging device each of the individual pixels provided on the semiconductor substrate is divided asymmetrically and the low-sensitivity pixels of the high-sensitivity pixel and the small area of the large area, an optical system placed in front of the solid-state image capturing device And a captured image data recording method of a digital camera comprising the optical system drive mechanism, wherein the high sensitivity image data obtained from the high sensitivity pixel and the low sensitivity image data obtained from the low sensitivity pixel are RAW Data is recorded on a recording medium, and the sensitivity ratio between the high-sensitivity pixel and the low-sensitivity pixel corresponding to the optical system at the time of shooting and the driving state of the optical system is recorded on the recording medium as image correction parameter data. A method for recording captured image data of a digital camera.

Images