JP3977260B2 - Digital camera and recorded image data recording method - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a digital camera that captures both a low-sensitivity image and a high-sensitivity image and a captured image data recording method thereof. In particular, the captured image data is converted to RAW data (raw image data output from a solid-state image sensor). The present invention relates to a digital camera and captured image data recording method for recording in a memory with raw data.
[0002]
[Prior art]
In digital cameras such as digital still cameras and digital video cameras, the saturation amount of charge accumulated in the photodiodes that make up each pixel of the solid-state image sensor decreases as the number of pixels increases, that is, the photodiodes become smaller. Has a drawback that the dynamic range becomes narrow.
[0003]
In order to overcome this drawback, for example, in the conventional technique described in Japanese Patent Application Laid-Open No. 2001-8104, a solid-state image sensor is provided with two types of pixels, a high-sensitivity pixel and a low-sensitivity pixel, and a captured image obtained from the high-sensitivity pixel. The dynamic range of the captured image is expanded by combining the data and the captured image data obtained from the low sensitivity pixels.
[0004]
On the other hand, when the number of pixels of a solid-state image sensor increases as in recent years, an image equivalent to that of a silver salt camera is picked up. The image data is saved in raw memory as raw data without white balance correction, gamma correction, JPEG compression, etc., and this image data is read by a personal computer etc., and white balance correction, gamma correction, color tone correction, etc. Users' desire to do according to their own preference is increasing. For this reason, digital cameras record image data as RAW data as described in JP-A-11-261933 (Patent Document 2) and JP-A-2001-2231979 (Patent Document 3). Is increasing.
[0005]
[Patent Document 1]
Japanese Patent Laid-Open No. 2001-8104 [Patent Document 2]
JP-A-11-261933 [Patent Document 3]
[Patent Document 1] Japanese Patent Application Laid-Open No. 2001-223979
[Problems to be solved by the invention]
As described above, in the digital camera that captures both the low-sensitivity image and the high-sensitivity image, when the captured image data is recorded in the memory as RAW data, both the captured image data of the low-sensitivity image and the captured image data of the high-sensitivity image are recorded. Will be recorded. However, even if the user reads low-sensitivity image data and RAW data of high-sensitivity image data from the memory and composes the image using application software on a personal computer, the composition process does not go well, and a composite image with the desired image quality is obtained. There is a problem that can not be.
[0007]
SUMMARY OF THE INVENTION An object of the present invention is to provide a digital camera capable of easily and quickly obtaining a synthesized image of desired image quality even when synthesizing RAW data of low sensitivity image data and high sensitivity image data outside the camera, and its captured image data. It is to provide a recording method.
[0008]
[Means for Solving the Problems]
A digital camera that achieves the above object and a method for recording captured image data thereof are obtained from the high-sensitivity image data obtained from the high-sensitivity pixel and the low-sensitivity pixel of a solid-state imaging device having high-sensitivity pixels and low-sensitivity pixels. The low-sensitivity image data is recorded on the recording medium as RAW data, and each data d in units of pixels when the high-sensitivity image data and the low-sensitivity image data are combined is expressed as d = [high + MIN (high / th, 1) × low] × MAX [(− k × high / th) + α, p]
Where: high: Data after gamma correction of high sensitivity (high output) image signal
low: Data after gamma correction of low sensitivity (low output) image signal p: total gain
k: coefficient
th: threshold α: value determined by the scene (≈1)
The above-described values of p and α automatically set according to the shooting scene by the camera and the k value obtained from the signal charge saturation amount ratio of the high-sensitivity pixel and the low-sensitivity pixel are used as the image correction parameters. This is achieved by recording on a medium.
[0009]
With this configuration, the range of parameter values for image composition can be adjusted with reference to the values of k, p, and α recorded on the recording medium, and a composite image with the image quality desired by the user can be generated easily and quickly. It becomes.
[0010]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
[0011]
FIG. 1 is a configuration diagram of a digital still camera according to an embodiment of the present invention. In this embodiment, a digital still camera will be described as an example. However, the present invention can also be applied to other types of digital cameras such as a camera mounted on a small electronic device such as a digital video camera or a mobile phone. Further, the image synthesis processing of the present embodiment is executed by software by the digital signal processing unit 26 described later, but this can also be realized by a hardware circuit.
[0012]
The digital still camera shown in FIG. 1 includes a photographic lens 10, a solid-state image sensor 11, a diaphragm 12 provided between them, an infrared cut filter 13, and an optical low-pass filter 14. The CPU 15 that controls the entire digital still camera controls the light emitting unit 16 and the light receiving unit 17 for flash, controls the lens driving unit 18 to adjust the position of the photographing lens 10 to the focus position, and stops the aperture driving unit. The opening amount of the diaphragm 12 is controlled via 19 to adjust the exposure amount to an appropriate exposure amount.
[0013]
Further, the CPU 15 drives the solid-state image sensor 11 via the image sensor driving unit 20 and outputs the subject image captured through the photographing lens 10 as a color signal. In addition, a user instruction signal is input to the CPU 15 through the operation unit 21, and the CPU 15 performs various controls according to the instruction.
[0014]
The electric control system of the digital still camera includes an analog signal processing unit 22 connected to the output of the solid-state imaging device 11, and an A / D conversion that converts RGB color signals output from the analog signal processing unit 22 into digital signals. The circuit 23 is provided and these are controlled by the CPU 15.
[0015]
Further, the electric control system of this digital still camera includes a memory control unit 25 connected to the main memory 24, a digital signal processing unit 26 described later in detail, and compresses the captured image into a JPEG image and decompresses the compressed image. Mounted on the back side of the camera, the compression / decompression processing unit 27, the integration unit 28 for integrating the photometric data and adjusting the white balance gain, the external memory control unit 30 to which the removable recording medium 29 is connected. And a display control unit 32 to which the liquid crystal display unit 31 is connected. These are connected to each other by a control bus 33 and a data bus 34, and are controlled by a command from the CPU 15.
[0016]
The digital signal processing unit 26, the analog signal processing unit 22, the A / D conversion circuit 23, and the like shown in FIG. 1 can be mounted on the digital still camera as separate circuits. It is preferable to manufacture on the same semiconductor substrate using LSI manufacturing technology to form one solid-state imaging device.
[0017]
FIG. 2 is a pixel arrangement diagram of the solid-state imaging device 11 used in the present embodiment. The pixel 1 in the CCD portion that captures an image with a wide dynamic range has a pixel arrangement described in, for example, Japanese Patent Laid-Open No. 10-136391, and each pixel in the odd row is horizontally aligned with each pixel in the even row. A vertical transfer path (not shown) that is arranged with a ½ pitch shift and transfers signal charges read from each pixel in the vertical direction is meandered so as to avoid each pixel in the vertical direction. ing.
[0018]
In the illustrated example, each pixel 1 according to the present embodiment includes a low-sensitivity pixel (sub-pixel) 2 that occupies about 1/5 of the area of the pixel 1 and a high-sensitivity pixel (subpixel) that occupies the remaining about 4/5. Main signal) 3 so that the signal charge of each low-sensitivity pixel 2 and the signal charge of each high-sensitivity pixel 3 can be distinguished and transferred to the vertical transfer path. . It should be noted that the ratio and the position at which the pixel 1 is divided are determined by design, and FIG. 2 is merely an example.
[0019]
In the imaging apparatus according to the present embodiment, a low-sensitivity image (an image obtained by the low-sensitivity pixel 2) and a high-sensitivity image (an image obtained by the high-sensitivity pixel 3) are simultaneously acquired by one imaging, and each image is acquired. Are sequentially read out from the pixels 2 and 3, and the details are combined as described later, or are recorded as they are on the recording medium 29 as RAW data.
[0020]
The solid-state imaging device 11 has been described by taking a CCD having a honeycomb pixel arrangement as shown in FIG. 2 as an example, but may be a Bayer CCD or a CMOS sensor.
[0021]
FIG. 3 is a diagram for explaining the operation of the digital still camera described above. An analog high-sensitivity image signal H output from the high-sensitivity pixel 3 of the solid-state image sensor 11 is converted into, for example, 10-bit digital data by the A / D converter 23 and output from the low-sensitivity pixel 2 of the solid-state image sensor 11. The analog low-sensitivity image signal L is converted into, for example, 8-bit digital data by the A / D converter 23.
[0022]
When the user sets the completed image recording mode with the operation unit 21, the digital signal processing unit 26 captures the 10-bit high sensitivity image data and the 8-bit low sensitivity image data output from the A / D converter 23. The external memory control unit (recording circuit) 30 performs, for example, JPEG received from the digital signal processing unit 26 via the compression / decompression processing unit 27 (not shown in FIG. 3). The compressed completed image data is recorded on the recording medium 29.
[0023]
When the user sets the RAW data recording mode on the operation unit 21, the external memory control unit 30 outputs the 10-bit high sensitivity image data and the 8-bit low sensitivity image data output from the A / D converter 23. The data is recorded on the recording medium 29 as it is. At this time, the external memory control unit 30 records the value of an image correction parameter described later together with the RAW data.
[0024]
FIG. 4 is a processing configuration diagram of the digital signal processing unit 26 that operates in the completed image recording mode. The digital signal processing unit 26 employs a logarithmic addition method in which high-sensitivity image signals and low-sensitivity image signals are subjected to gamma correction and then subjected to addition processing, and the high-sensitivity image output from the A / D conversion circuit 23 is used. An offset correction circuit 41a that takes an RGB color signal that is a digital signal and performs an offset process, a gain correction circuit 42a that white balances an output signal of the offset correction circuit 41a, and a gamma correction for the color signal after gain correction. A gamma correction circuit 43a for performing, an offset correction circuit 41b for performing an offset process by taking in RGB color signals which are digital signals of low-sensitivity images output from the A / D conversion circuit 23 shown in FIG. 1, and an offset correction circuit 41b The gain correction circuit 42b for white balance of the output signal and the color signal after gain correction And a gamma correction circuit 43b which performs Ma correction. When linear matrix processing or the like is performed on the signal after offset correction, it is performed between the gain correction circuits 42a and 42b and the gamma correction circuits 43a and 43b.
[0025]
The digital signal processing unit 26 further interpolates the RGB color signal after the image composition by performing an image composition process by taking both output signals of each of the gamma correction circuits 43a and 43b and performing image composition processing. Reduces noise from the RGB interpolation calculation unit 45 for obtaining RGB three-color signals, the RGB / YC conversion circuit 46 for obtaining the luminance signal Y and the color difference signals Cr and Cb from the RGB signal, and the luminance signal Y and the color difference signals Cr and Cb. A noise filter 47, a contour correction circuit 48 that performs contour correction on the luminance signal Y after noise reduction, and a color difference matrix circuit 49 that performs color correction by multiplying the color difference signals Cr and Cb by the color difference matrix. Prepare.
[0026]
The above-described image synthesis processing circuit 44 synthesizes the high sensitivity image signal output from the gamma correction circuit 43a and the low sensitivity image signal output from the gamma correction circuit 43b in units of pixels based on the following equation (1). Output.
[Expression 1]
data = [high + MIN (high / th, 1) × low] × MAX [(− k × high / th) + α, p]
Where: high: Data after gamma correction of high sensitivity (high output) image signal
low: Data after gamma correction of low sensitivity (low output) image signal p: total gain
k: coefficient
th: threshold α: value determined by the scene (≈1)
It is.
[0027]
If the data after gamma correction is 8-bit data (256 gradations), the threshold value th is designated by, for example, “219” from the value 0 to 255 and the user of the digital still camera or the designer of the digital still camera. Is the value to be
[0028]
The first term of Equation 1 adds the low sensitivity image data low as it is to the high sensitivity image data high when the high sensitivity image data high exceeds the threshold th, and when the high sensitivity image data high is equal to or less than the threshold th, A value obtained by multiplying the ratio of the high sensitivity image data high to the threshold th by the low sensitivity image data low is added to the high sensitivity image data high.
[0029]
In the present embodiment, the added data obtained in the first term is not used as composite image data as it is, but the second term (MAX [(− k × high / th) + α, p]) is added to the first term. A value obtained by multiplying is used as composite image data.
[0030]
In the second term, the coefficient “k” is preferably a value “0.2” in the solid-state imaging device 11 of the embodiment shown in FIG. When the signal charge saturation ratios of the high-sensitivity pixel 3 and the low-sensitivity pixel 2 are different as in the solid-state imaging device 11 shown in FIG.
[Expression 2]
Coefficient k = 1−Sh / (Sh + Sl)
Here, Sh: signal charge saturation amount of high sensitivity pixel Sl: signal charge saturation amount of low sensitivity pixel
In the example shown in FIG. 2, the area ratio of the photodiode does not become the saturation ratio as it is, but it can be regarded as an area ratio for convenience, and when the above example is applied,
k = 1-4 / (4 + 1) = 1-0.8
= 0.2
It becomes.
[0032]
The solid-state imaging device having both high-sensitivity pixels and low-sensitivity pixels is not limited to that illustrated in FIG. 2, and for example, as shown in FIG. 5, a large number of photodiodes (not shown) formed in the same size and shape. It is conceivable to change the aperture area of the microlens provided on top of the above and provide the high-sensitivity pixel 3 and the low-sensitivity pixel 2. In this case, the saturation amount of the signal charge of the high-sensitivity pixel and the low-sensitivity pixel is the same, and thus Equation 2 cannot be applied. However, the value of the coefficient k is obtained experimentally or from the opening area of a microlens or the like. Equation 1 can be applied by obtaining the coefficient value. The value of the coefficient k is a value that is determined by the configuration of the solid-state imaging device, and is not arbitrarily changed by the user, but is set to a fixed value when the imaging apparatus is shipped.
[0033]
In Equation 1, an experimentally determined value is used as the value of the total gain p in this embodiment. p is a gain with respect to the entire composite image data. In this embodiment, the dynamic range of the image is controlled by controlling the value of p. The smaller the value of the total gain p, the wider the dynamic range, and the larger the value of p, the narrower the dynamic range. Specifically, depending on the scene, p = 0.8 for high-contrast scenes (such as midsummer sunny weather), p = 0.86 for cloudy or shaded, and p = 0.9 for indoor fluorescent lighting. To change the value of p. Thereby, when the data after gamma correction is 8-bit data, the 8-bit gradation value can be used more effectively.
[0034]
Even if the user sets the value of p by specifying the type of scene using the operation unit 21 shown in FIG. 1, the digital still camera itself can change the scene of the captured image based on the detection values of various sensors. Automatic determination and automatic setting may be used. For example, the white balance gain amount is obtained from the integrated value of the photometric data obtained by the integrating unit 28. Since the scene can be automatically determined from the white balance value, the value of p can be automatically set. .
[0035]
FIG. 6 is a diagram showing how the dynamic range changes when the value of p is changed. When the total gain p is increased, the characteristic line a has a small dynamic range, and when the total gain p is decreased, the characteristic line a changes to a characteristic line B having a large dynamic range.
[0036]
As described above, in the completed image recording mode of the present embodiment, since the high-sensitivity image data and the low-sensitivity image data are added and then multiplied by the total gain according to the scene, an image with a wide dynamic range with white balance can be obtained. Can be generated. In addition, since the logarithmic addition method is employed to combine the images after reducing the number of bits of the high-sensitivity image data and the low-sensitivity image data, the circuit scale can be reduced and the cost can be reduced.
[0037]
The above is the operation of the digital signal processing unit in the completed image recording mode. In the RAW data recording mode, the high-sensitivity image data and the low-sensitivity image data output from the A / D converter 23 are recorded on the recording medium 29 as they are. As a result, the user reads RAW data to the personal computer and adjusts various image synthesis parameters to repeat trial and error so that a synthesized image having a desired image quality can be obtained.
[0038]
When the user performs image composition work, if the user does not know under what circumstances the RAW data was taken, it does not know how to adjust the image composition parameters, and the number of trial and error increases. End up. The software processing for image composition when the user performs image composition work is not necessarily the same as the processing performed by the digital signal processing unit 26 in the completed image recording mode, but the same processing as that of the digital signal processing unit 26 is performed. When it is desired to perform the operation manually with high accuracy, the above-described coefficient k, total gain p, and value α determined by the scene are adjusted manually.
[0039]
At this time, if k, p, α values automatically determined as optimum values by the camera at the time of shooting are known, the user can narrow down the adjustment range of each value with reference to the k, p, α values. Thus, a composite image having a desired image quality can be obtained more quickly and easily.
[0040]
Therefore, in the digital still camera according to the present embodiment, the values of k, p, and α are recorded on the recording medium 29 together with the RAW data as the image correction parameters described in FIG.
[0041]
FIG. 7 is a format diagram of an image file when RAW data is recorded on the recording medium 29. The image file includes a header portion and an image data portion. The header portion includes file information 51 such as a file number, model information 52 of a digital still camera and a lens used for photographing, an image correction parameter 53, And shooting information 54 such as an exposure amount and a shutter speed when the image is taken. The image data portion includes RAW data 55 for high-sensitivity images and RAW data 56 for low-sensitivity images, and the values of k, p, and α are recorded as image correction parameters 53.
[0042]
In Equation 1, the value of α may be a value corresponding to the scene of the captured image, but may be a fixed value “1”.
[0043]
In the above-described embodiment, the digital still camera having the two modes of the RAW data recording mode and the completed image recording mode has been described. However, the present invention can be applied to a digital camera not equipped with the completed image recording mode. Applicable.
[0044]
【The invention's effect】
According to the present invention, an image correction parameter automatically determined by a camera when recording image data of a digital still camera equipped with a solid-state image sensor that outputs high-sensitivity image data and low-sensitivity image data on a recording medium as RAW data. Since the image is recorded together, when the user performs image composition work, the composition processing can be performed with reference to the value of the parameter for image correction, and a composite image with good image quality can be obtained easily and quickly. It becomes.
[Brief description of the drawings]
FIG. 1 is a block diagram of a digital still camera according to a first embodiment of the present invention.
FIG. 2 is a schematic surface view of the solid-state imaging device shown in FIG.
3 is an operation explanatory diagram of the digital still camera shown in FIG. 1. FIG.
4 is a processing configuration diagram of a digital signal processing unit shown in FIG. 3. FIG.
FIG. 5 is a pixel arrangement diagram according to another embodiment of the solid-state imaging device.
FIG. 6 is a diagram showing how the dynamic range changes.
7 is a format diagram of an image file when captured image data is recorded on a recording medium as RAW data by the digital still camera shown in FIG. 1; FIG.
[Explanation of symbols]
1 pixel 2 low sensitivity pixel (sub pixel)
3 High sensitivity pixels (main pixels)
DESCRIPTION OF SYMBOLS 10 Lens 11 Solid-state image sensor 23 A / D converter 26 Digital signal processing part 29 Recording medium 44 Image composition processing circuit 53 Image correction parameter

Claims (2)

A solid-state imaging device having a high-sensitivity pixel and a low-sensitivity pixel, high-sensitivity image data obtained from the high-sensitivity pixel, and low-sensitivity image data obtained from the low-sensitivity pixel are recorded on a recording medium as raw data. In the case of combining high-sensitivity image data and low-sensitivity image data, each data d in pixel units is expressed as d = [high + MIN (high / th, 1) × low] × MAX [(−k × high / th) + α, p]
Where: high: Data after gamma correction of high sensitivity (high output) image signal
low: Data after gamma correction of low sensitivity (low output) image signal p: total gain
k: coefficient
th: threshold α: value determined by the scene (≈1)
The above-described values of p and α automatically set according to the shooting scene by the camera and the k value obtained from the signal charge saturation amount ratio of the high-sensitivity pixel and the low-sensitivity pixel are used as the image correction parameters. A digital camera comprising recording means for recording on a medium. High-sensitivity image data obtained from the high-sensitivity pixels and low-sensitivity image data obtained from the low-sensitivity pixels in a method for recording captured image data of a digital camera equipped with a solid-state imaging device having high-sensitivity pixels and low-sensitivity pixels Are recorded as RAW data on a recording medium, and each data d in pixel units when combining high-sensitivity image data and low-sensitivity image data is d = [high + MIN (high / th, 1) × low] × MAX. [(−k × high / th) + α, p]
Where: high: Data after gamma correction of high sensitivity (high output) image signal
low: Data after gamma correction of low sensitivity (low output) image signal p: total gain
k: coefficient
th: threshold α: value determined by the scene (≈1)
The above-described values of p and α automatically set according to the shooting scene by the camera and the k value obtained from the signal charge saturation amount ratio of the high-sensitivity pixel and the low-sensitivity pixel are used as the image correction parameters. A method for recording captured image data of a digital camera, comprising: recording on a medium.

Images