JP4028396B2 - Image composition method and digital camera - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image composition method for obtaining composite image data by performing composition processing of first image data having relatively high sensitivity and second image data having relatively low sensitivity, and relatively high sensitivity. The present invention relates to a digital camera having a function of performing composite processing of first image data and relatively low-sensitivity second image data to obtain composite image data.
[0002]
[Prior art]
The dynamic range of a solid-state image sensor such as a CCD image sensor is generally narrow. When shooting a high-contrast subject, the amount of light received by the solid-state image sensor exceeds the dynamic range and the output of the solid-state image sensor saturates. Information may be missing.
[0003]
In order to solve such a problem, the first image data with relatively high sensitivity (hereinafter sometimes simply referred to as “high sensitivity image data”) and the second image data with relatively low sensitivity. (Hereinafter, it may be simply described as “low-sensitivity image data”), and a technique for expanding the dynamic range by performing a synthesis process has been proposed (see Patent Documents 1 and 2).
[0004]
The technique described in Patent Document 1 is a standard luminance video signal (high-sensitivity image) obtained by setting an object with standard brightness to an appropriate level under at least one of different exposure conditions and different gain control. Data) and a high-luminance video signal (low-sensitivity image data) obtained so that an object brighter than a predetermined value has an appropriate level is added to obtain a wide dynamic range video signal. At that time, the peak value of the high-luminance video signal is detected, and based on this peak value, a multiplication coefficient for multiplying the standard luminance video signal and the high-luminance video signal is calculated. It is.
[0005]
The technique described in Japanese Patent Laid-Open No. 2004-228561 is an image capturing unit that captures an image of a scene and generates a high-sensitivity video signal and a low-sensitivity video signal representing the scene. First signal conversion means for quantizing with a quantization resolution and outputting corresponding high-sensitivity video signal data; and quantizing a low-sensitivity video signal with a second quantization resolution lower than the first quantization resolution And a second signal converting means for outputting the corresponding low-sensitivity video signal data, and adding and combining the high-sensitivity video signal data and the corresponding low-sensitivity video signal data to produce a wide dynamic range video signal Is formed.
[0006]
However, when the luminance range of the subject is large, the optimum combined image data cannot always be obtained simply by adding and combining the high sensitivity image data and the low sensitivity image data. The same applies when the composite gain of the high-sensitivity image data and the low-sensitivity image data is changed depending on the luminance range. For example, a catchlight scene has a large luminance range of a subject, but if an image is configured with a wide dynamic range, the gradation of the highlight portion becomes soft and becomes a so-called sleepy image. On the other hand, a high-contrast scene such as a wedding is similarly wide in the luminance range of the subject. In this case, it is preferable to construct an image with a wide dynamic range because the gradation reproduction is rich.
[0007]
Further, when trying to widen the dynamic range by image composition, the apparent gain on the highlight side is lowered, and there is a problem that the saturation is lowered.
[0008]
[Patent Document 1]
JP 2001-94870 A
[Patent Document 2]
JP 2001-8104 A
[0009]
[Problems to be solved by the invention]
The present invention has been made in view of the above circumstances, and an image composition method capable of obtaining more preferable composite image data in accordance with a shooting scene, and a digital that performs composition processing using such an image composition method. The purpose is to provide a camera.
[0010]
[Means for Solving the Problems]
The image composition method of the present invention obtains composite image data by performing composition processing of relatively high-sensitivity first image data and relatively low-sensitivity second image data. The ratio x between the maximum value of the subject brightness indicated by the image data 2 and the brightness value indicated by the maximum value that can be taken by the first image data, the on / off information of the strobe, the average brightness information of the subject, and the backlight information Correcting the ratio x based on at least one of the following: determining the synthesis processing parameter based on the corrected ratio x ′; and the first image based on the synthesis processing parameter And a process of combining the data and the second image data. According to this synthesis method, the brightness range of the necessary part can be synthesized within a range that can be efficiently reproduced according to the scene of the subject and the luminance range.
[0011]
The image composition method of the present invention further includes a step of determining a gamma correction parameter of the second image data based on the corrected ratio x ′, and the second image data based on the gamma correction parameter. Correcting. According to this synthesis method, it is possible to obtain a composite image having a more preferable gradation according to the scene of the subject and the luminance range.
[0012]
The image composition method of the present invention further includes a step of determining a saturation correction parameter of the composite image data based on the corrected ratio x ′, and a saturation of the composite image data based on the saturation correction parameter. Correcting the degree. According to this synthesis method, saturation reduction due to apparent gain reduction on the highlight side can be corrected.
[0013]
The step of correcting the ratio x in the image composition method of the present invention is a step of limiting the value of the ratio x based on at least one of strobe on / off information, subject average luminance information, and backlight information. Including.
[0014]
The image composition method of the present invention obtains composite image data by performing composition processing of relatively high-sensitivity first image data and relatively low-sensitivity second image data. And determining the ratio x between the maximum value of the subject brightness indicated by the image data of 2 and the brightness value indicated by the maximum value that can be taken by the first image data, and the second image data based on the ratio x Determining the gamma correction parameter and the synthesis processing parameter; correcting the second image data based on the gamma correction parameter; and the first image data based on the synthesis processing parameter And a step of performing synthesis processing with the second image data. According to this synthesis method, it is possible to obtain a composite image having a preferable gradation according to the luminance range of the subject.
[0015]
The image composition method of the present invention obtains composite image data by performing composition processing of relatively high-sensitivity first image data and relatively low-sensitivity second image data. The ratio x between the maximum value of the subject brightness indicated by the image data of 2 and the brightness value indicated by the maximum value that can be taken by the first image data, the composition processing parameter based on the ratio x, and Determining a saturation correction parameter of the synthesized image data; performing a synthesis process of the first image data and the second image data based on the synthesis processing parameter; and based on the saturation correction parameter. And a step of correcting the saturation of the composite image data. According to this synthesis method, it is possible to correct a saturation reduction due to an apparent gain reduction on the highlight side.
[0016]
The image composition method of the present invention obtains composite image data by performing composition processing of relatively high-sensitivity first image data and relatively low-sensitivity second image data. And determining the ratio x between the maximum value of the subject brightness indicated by the image data of 2 and the brightness value indicated by the maximum value that can be taken by the first image data, and the second image data based on the ratio x Determining a gamma correction parameter, a synthesis processing parameter, and a saturation correction parameter of the composite image data; correcting the second image data based on the gamma correction parameter; and based on the composite processing parameter A step of combining the first image data and the second image data, and a step of correcting the saturation of the combined image data based on the saturation correction parameter. That.
[0017]
The step of obtaining the ratio x at the time of electronic zoom photographing in the image composition method of the present invention includes using the maximum value of the subject brightness indicated by the second image data in a different area according to the zoom magnification. According to this composition method, composition processing can be performed based on the subject luminance in a range corresponding to the reproduction region, and therefore, even in a catchlight scene illuminated from the ceiling, for example, a satisfactory composite image according to the zoom magnification Can be obtained.
[0018]
The digital camera of the present invention has a function of performing composite processing of relatively high sensitivity first image data and relatively low sensitivity second image data to obtain composite image data, The synthesizing process is performed using the image synthesizing method described above.
[0019]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 12 is a diagram showing a schematic configuration of the digital camera according to the embodiment of the present invention. The digital camera of FIG. 12 includes an imaging unit 1, an analog signal processing unit 2, an A / D conversion unit 3, a driving unit 4, a strobe 5, a digital signal processing unit 6, a compression / expansion processing unit 7, a display unit 8, and system control. A unit 9, an internal memory 10, a media interface 11, a recording medium 12, and an operation unit 13. The digital signal processing unit 6, compression / decompression processing unit 7, display unit 8, system control unit 9, internal memory 10, and media interface 11 are connected to a system bus 20.
[0020]
The imaging unit 1 includes an optical system such as a photographic lens and an imaging element such as a CCD image sensor, and shoots a subject, and outputs an analog imaging signal. An imaging signal obtained by the imaging unit 1 is sent to the analog signal processing unit 2, subjected to predetermined analog signal processing, converted into a digital signal by the A / D conversion unit 3, and then digitally converted as so-called RAW image data. It is sent to the signal processing unit 6. The RAW image data is digital image data that has been digitized in the form of the imaging signal from the imaging unit 1.
[0021]
When shooting, the imaging unit 1 is controlled via the drive unit 4. A solid-state image pickup device such as a CCD image sensor used as an image pickup device has a plurality of photoelectric conversion regions (for example, photodiodes) arranged on a semiconductor substrate surface in a row direction and a column direction orthogonal thereto, and incident light. An analog voltage signal based on the accumulated signal charge is output. The solid-state imaging device is a timing generator (not shown) included in the drive unit 4 at a predetermined timing when a release switch (not shown) is turned on by operating a release button (not shown) which is a part of the operation unit 13. 12 is written by a drive signal from TG). The drive unit 4 outputs a predetermined drive signal based on the control of the system control unit 9, and also outputs drive signals for the analog signal processing unit 2 and the A / D conversion unit 3.
[0022]
The imaging unit 1 outputs a relatively high-sensitivity imaging signal and a relatively low-sensitivity imaging signal. Both of the imaging signals pass through an analog signal processing unit 2 and an A / D conversion unit 3 to form a digital image. It is converted into data and sent to the digital signal processor 6.
[0023]
Imaging signals with different sensitivities may be obtained by changing exposure conditions or analog gains as shown in Patent Document 1, or a solid having a high-sensitivity imaging cell and a low-sensitivity imaging cell as shown in Patent Document 2. You may acquire by utilizing an image pick-up element. Further, a solid-state imaging device as shown in FIG. 13 may be used.
[0024]
FIG. 13 is a diagram illustrating a schematic configuration of a solid-state imaging device capable of outputting a high-sensitivity imaging signal and a low-sensitivity imaging signal. FIG. 13 is a partially enlarged plan view of a so-called honeycomb-structured solid-state imaging device, which is arranged on the surface of a semiconductor substrate in a row direction (direction indicated by an arrow X) and a column direction (direction indicated by an arrow Y) perpendicular thereto. The plurality of photoelectric conversion regions 111 to 157 (only some are numbered in the figure), the vertical transfer units 201 to 208, the horizontal transfer unit 300, and the output unit 400 are included. Among the plurality of photoelectric conversion regions 111 to 157, the odd-numbered columns are shifted in the column direction by about ½ of the column-direction pitch between the photoelectric conversion regions with respect to the even-numbered columns. The photoelectric conversion regions are arranged so as to be shifted in the row direction by about ½ of the row direction pitch between the photoelectric conversion regions with respect to even-numbered photoelectric conversion regions. Although FIG. 13 shows a photoelectric conversion region of 5 rows and 8 columns, in reality, more photoelectric conversion regions are provided.
[0025]
The photoelectric conversion regions 111 to 157 generate and accumulate signal charges corresponding to the amount of incident light, and are, for example, photodiodes. The photoelectric conversion regions 111 to 157 are divided into a main region m having a relatively large light receiving area and a sub region s having a relatively small light receiving area (in FIG. 13, only the photoelectric conversion region 151 is given a reference numeral. A signal charge corresponding to light having a predetermined spectral sensitivity is generated and stored. In the solid-state imaging device of FIG. 13, red (indicated by R), green (indicated by G), or blue (indicated by B) filters (not shown) have respective photoelectric conversion regions 111 to 157. The signal charge corresponding to the light of each color is generated and accumulated.
[0026]
The vertical transfer units 201 to 208 read signal charges from the photoelectric conversion regions 111 to 157 and transfer them in the column direction. The vertical transfer units 201 to 208 are provided on the sides corresponding to the columns of the photoelectric conversion regions 111 to 157. The transfer in the column direction is performed by independently reading the signal charges in the main region m and the signal charges in the sub region s to the vertical transfer units 201 to 208, respectively. The horizontal transfer unit 300 transfers signal charges from the plurality of vertical transfer units 201 to 208 and transfers the transferred signal charges in the row direction. The output unit 400 outputs a voltage signal corresponding to the transferred signal charge amount.
[0027]
The strobe 5 operates when the brightness of the subject is equal to or lower than a predetermined value, and is controlled by the system control unit 9.
[0028]
The digital signal processing unit 6 performs digital signal processing on the digital image data from the A / D conversion unit 3 in accordance with the operation mode set by the operation unit 13. The processing performed by the digital signal processing unit 6 includes black level correction processing (OB processing) and linear matrix correction processing (removing mixed components due to the photoelectric conversion characteristics of the image sensor with respect to the primary color signal from the imaging unit. Processing for performing correction (by 3 × 3 matrix operation for RBG input), white balance adjustment processing (gain adjustment), gamma correction processing, image synthesis processing, synchronization processing, Y / C conversion processing, and the like. The image composition processing of the digital signal processing unit 6 will be described later.
[0029]
The digital signal processing unit 6 is configured by a DSP, for example. The compression / decompression processing unit 7 performs compression processing on the Y / C data obtained by the digital signal processing unit 6 and also performs decompression processing on the compressed image data obtained from the recording medium 12. .
[0030]
The display unit 8 includes, for example, an LCD display device, and displays an image based on image data that has been shot and has undergone digital signal processing. An image is also displayed based on the image data obtained by decompressing the compressed image data recorded on the recording medium. It is also possible to display a through image at the time of shooting, various states of the digital camera, information on operations, and the like.
[0031]
The internal memory 10 is, for example, a DRAM and is used as a work memory for the digital signal processing unit 6 and the system processing unit 9, as well as a buffer memory that temporarily stores captured image data recorded on the recording medium 12, and a display It is also used as a buffer memory for display image data to the unit 8. The media interface 11 inputs / outputs data to / from a recording medium 12 such as a memory card.
[0032]
The system control unit 9 controls the entire digital camera including shooting operations. Specifically, the system control unit 9 is mainly configured by a processor that operates according to a predetermined program.
[0033]
The operation unit 13 performs various operations when using the digital camera, and performs an operation mode (shooting mode, playback mode, etc.) of the digital camera, a shooting method at the time of shooting, shooting conditions, settings, and the like. The operation unit 13 may be provided with an operation member corresponding to each function, but may share the operation member in conjunction with the display of the display unit 8. The operation unit 13 also includes a release button for starting a shooting operation.
[0034]
Next, functions of the digital signal processing unit 6 will be described focusing on image synthesis processing. FIG. 1 is a schematic functional block diagram of the digital signal processing unit 6. The digital signal processing unit 6 includes an OB (optical black) processing unit 61a, 61b, an LMTX (linear matrix) 62a, 62b, a WB (white balance) processing unit 63a, 63b, a gamma correction unit 64a, 64b, a combined gain LUT (look). (Uptable) 65, multiplication units 66 a and 66 b, addition unit 67, limiter 68, synchronization and Y / C processing unit 69, CMTX (color difference matrix) 70, integration processing unit 71, and synthesis range calculation unit 72.
[0035]
The OB processing units 61a and 61b correct the black level of the image data, and the LMTXs 62a and 62b adjust the hue by performing, for example, a 3 × 3 matrix operation. The WB processing units 63a and 63b multiply the image data indicating the respective colors by the gain for white balance correction, and the gamma correction units 64a and 64b make the respective image data have desired gamma characteristics. The input / output characteristics are changed.
[0036]
The composite gain LUT 65 outputs the respective gain data when the high-sensitivity image data and the low-sensitivity image data are added and combined in accordance with the high-sensitivity image data. The composite gain will be described later.
[0037]
The high-sensitivity image data High input to the digital processing unit 6 is input to the multiplication unit 66a through processing by the OB processing unit 61a, the LMTX 62a, the WB processing unit 63a, and the gamma correction unit 64a. Similarly, the low-sensitivity image data Low is input to the multiplication unit 66b after being processed by the OB processing unit 61b, the LMTX 62b, the WB processing unit 63b, and the gamma correction unit 64b. Then, the gain data from the combined gain LUT 65 is multiplied, and the multiplication result is added by the adder 67.
[0038]
The image data from the adder 67 is limited to be within a range that can be expressed by a limiter 68 in a predetermined number of bits (for example, 8 bits), and is input to the synchronization and Y / C processing unit 69. The synchronization and Y / C processing unit 69 interpolates point-sequential RGB signals, obtains RGB signals for each imaging position, and further converts the RGB signals into luminance signals Y and color difference signals Cr and Cb. Then, saturation correction is performed by the CMTX 70 and output as composite image data.
[0039]
The integration processing unit 71 integrates the low-sensitivity image data that has passed through the LMTX 62b. Specifically, the integration processing unit 71 divides the entire region of the image or the region excluding the peripheral portion of the image into 16 × 16, and the green in each divided region. Data G is integrated. The composition range calculation unit 72 obtains a composition range when performing image composition. The composite range is determined based on a ratio x between the maximum value of the subject luminance indicated by the low-sensitivity image data and the luminance value indicated by the maximum value that can be taken by the high-sensitivity image data. A method for obtaining the synthesis range will be described later.
[0040]
The output of the synthesis range calculation unit 72 is sent to the synthesis gain LUT 65 to output gain data corresponding to the synthesis range. In addition, the gamma correction characteristic of the low-sensitivity image data is sent to the gamma correction unit 64b according to the synthesis range. Further, the color difference data sent to the CMTX 70 and obtained by the synchronization and Y / C processing unit 69 is corrected with a gain corresponding to the synthesis range, and the saturation of the output synthesized image data is corrected.
[0041]
Note that the correction of the gamma characteristic and the saturation correction of the low-sensitivity image data based on the synthesis range from the synthesis range calculation unit 72 are not essential and may be omitted as appropriate.
[0042]
FIG. 2 shows a schematic sequence of digital signal processing up to the synthesis calculation processing at the time of photographing. After exposure is performed in the period t1, high-sensitivity image data is captured in the period t2. In the period t3, signal processing for high-sensitivity image data is performed simultaneously with the capture of low-sensitivity image data. Here, white balance gain determination processing (AWB), scratch correction processing (scratches), and preprocessing are performed. The preprocessing includes OB processing, LMTX calculation, WB processing, and gamma correction processing.
[0043]
When the capturing of the low-sensitivity image data is completed, the composite range calculation process is performed in the period t4. The calculation processing of the synthesis range here includes OB processing of low-sensitivity image data, LMTX calculation, and integration processing. When the composite range is determined, low-sensitivity image data preprocessing (including WB processing and gamma correction processing) is performed in a period t5. Then, in the period t6, a combination operation of the high sensitivity image data and the low sensitivity image data after the gamma correction is performed.
[0044]
Next, gain data stored in the combined gain LUT 65 will be described. The gain data is created based on the formula (1) for obtaining the composite data data.
[0045]
data = [high + MIN (high / th, 1) × low] × MAX [(− lg × high / th) +1, p] (1)
here,
high is high-sensitivity image data after gamma correction
low is low-sensitivity image data after gamma correction
th is a threshold value for changing the addition ratio of low-sensitivity image data
p is the total gain
lg is a value that depends on the ratio of the saturation output of the high-sensitivity image data and the low-sensitivity image data (for example, if the saturation ratio is 4: 1, p = 0.2).
[0046]
Expand equation (1),
h_gain = MAX [(− lg × high / th) +1, p] (2)
l_gain = MAX [(− lg × high / th) +1, p] × MIN (high / th, 1) = h_gain × wl (3)
Then,
data = h_gain × high + l_gain × low
It becomes.
[0047]
h_gain is a gain for high-sensitivity image data and has the characteristics shown in FIG. Further, l_gain is a gain for low-sensitivity image data and is h_gain × wl, and wl has the characteristics shown in FIG.
[0048]
In the composite gain LUT 65, the gain h_gain for high-sensitivity image data and the gain l_gain for low-sensitivity image data obtained by the above formulas (2) and (3) are tabular data corresponding to the high-sensitivity image data high. Is remembered as The stored gain data is stored in a number corresponding to the synthesis range output from the synthesis range calculation unit 72.
[0049]
Next, how to determine the synthesis range by the synthesis range calculation unit 72 will be described. First, a ratio x between the maximum value of the subject luminance indicated by the low-sensitivity image data and the luminance value indicated by the maximum value that can be taken by the high-sensitivity image data is obtained based on the integration data from the integration processing unit 71. The value obtained by dividing the integrated data by the number of integrated images is the average value of the G data for each divided region of the subject, and the subject brightness can be considered as 1.25 (80/64) times the G data. The maximum average value Gmax corresponds to the maximum value of the subject luminance indicated by the low-sensitivity image data.
[0050]
As shown in FIG. 5, the maximum value that the high-sensitivity image data can take is 4095, the maximum value that the low-sensitivity image data can take is 1023, and the luminance range of the low-sensitivity image data is relative to the luminance range of the high-sensitivity image data. If the luminance is 1.25 (80/64) times that of the green data G, the maximum value of the subject luminance indicated by the low-sensitivity image data and the luminance value indicated by the maximum value that can be taken by the high-sensitivity image data The ratio x is expressed by the following equation (4).
[0051]
x = Gmax × 80 × 400/1023 × 64 (%) (4)
[0052]
The synthesis range calculation unit 72 obtains the synthesis range based on the ratio x thus obtained. The synthesis range is determined by the ratio x, for example, as shown in FIG.
[0053]
Based on the synthesis range thus determined, gain data to be multiplied with the high-sensitivity image data and the low-sensitivity image data is selected. FIG. 7 shows an example of gradation characteristics corresponding to the composite range of the composite image data. As shown in FIG. 7, as the synthesis range increases (the luminance range of the subject increases), the reproduction range of the composite image data can be increased. The example in FIG. 7 shows an example in which the gamma characteristic of the low-sensitivity image data is not changed. However, the gamma characteristic of the low-sensitivity image data can be made a more preferable gradation characteristic according to the synthesis range.
[0054]
Next, an embodiment in which the synthesis range is changed according to the shooting scene will be described. The photographing scene is determined based on at least one of strobe on / off information, subject average luminance information, and backlight information. Then, the ratio x between the maximum value of the subject luminance indicated by the low-sensitivity image data and the luminance value indicated by the maximum value that can be taken by the high-sensitivity image data is corrected according to the shooting scene, and based on the corrected ratio x ′. Determine the synthesis range. The correspondence between the corrected ratio x ′ and the synthesis range may be the same as that for the ratio x shown in FIG. 6 or may be changed. The correction of the ratio x is performed by limiting the value of the ratio x to a certain value or less.
[0055]
(Correction example a) Low-brightness strobe on
A value obtained by limiting the ratio x by the saturation point sp (EV) is set as a corrected ratio x ′. FIG. 8 shows the relationship between the ratio x and the corrected ratio x ′. In this case, the saturation point sp (EV) is changed according to the average luminance value of the subject. When the average luminance value is large, the value is limited to a relatively low value, and when the average luminance value is low, the value is limited to a relatively high value. FIG. 9 shows the relationship between the average luminance value and the saturation point sp (EV).
[0056]
(Correction example b) For slow sync and daytime sync,
Regardless of the brightness of the subject, the ratio x is limited to a predetermined value. FIG. 10 shows the relationship between the ratio x and the corrected ratio x ′ in that case. In the example of FIG. 10, the ratio x is limited to 220%.
[0057]
(Correction example c) When the flash is off (correction during backlight scene)
As with the low-intensity strobe, the ratio x is limited by the saturation point sp (EV) as the corrected ratio x ′. As shown in FIG. 11, the saturation point sp (EV) is changed depending on the difference between the divided EV (division photometric value obtained by taking the weighted average of the center weight) and the peripheral EV (photometric value of the peripheral portion). Since it is determined that the scene is backlit as it goes to the negative side of the horizontal axis, the saturation point sp (EV) is reduced to reduce the synthesis range. Further, since the right side of the horizontal axis is a follow light scene, the saturation point sp (EV) is substantially maximized.
[0058]
Next, image composition processing during electronic zoom will be described. At the time of electronic zoom, the integration area for obtaining the ratio x between the maximum value of the subject brightness indicated by the low-sensitivity image data and the brightness value indicated by the maximum value that can be taken by the high-sensitivity image data is changed according to the zoom area. In this case, the integration processing unit 71 in FIG. 1 determines the size of the integration region based on the zoom step information changed by the operation unit 13, and further obtains image data of a region obtained by dividing the region into 16 × 16. Accumulate.
[0059]
When the ratio x is obtained in this way, the synthesis range can be determined more accurately. For example, even in a catchlight scene illuminated from the ceiling, a good composite image can be obtained according to the zoom magnification.
[0060]
【The invention's effect】
As is apparent from the above description, according to the present invention, an image composition method capable of obtaining more preferable composite image data according to the shooting scene, and a composition process using such an image composition method. A digital camera can be provided.
[Brief description of the drawings]
FIG. 1 is a schematic functional block diagram of a digital signal processing unit.
FIG. 2 is a diagram showing a schematic sequence of digital signal processing up to synthesis calculation processing at the time of shooting;
FIG. 3 is a diagram showing a gain characteristic for high-sensitivity image data.
FIG. 4 is a diagram showing a characteristic of a ratio wl between a gain for low-sensitivity image data and a gain for high-sensitivity image data.
FIG. 5 is a diagram illustrating an example of a relationship between high-sensitivity image data and low-sensitivity image data.
FIG. 6 is a diagram illustrating an example of a relationship between a ratio x between a maximum value of subject luminance indicated by low-sensitivity image data and a luminance value indicated by a maximum value that can be taken by high-sensitivity image data, and a synthesis range;
FIG. 7 is a diagram illustrating a change in tone characteristics of composite image data according to a composite range.
FIG. 8 is a diagram showing the relationship between the ratio x and the corrected ratio x ′ in the case of low-intensity strobe on.
FIG. 9 is a diagram showing a relationship between an average luminance value and a saturation point sp (EV) when a low-intensity strobe is on.
FIG. 10 is a diagram showing the relationship between the ratio x and the corrected ratio x ′ in the case of slow sync and daytime sync.
FIG. 11 is a diagram showing a method of determining a saturation point sp (EV) when the strobe is off.
FIG. 12 is a diagram showing a schematic configuration of a digital camera according to an embodiment of the present invention.
FIG. 13 is a diagram showing a schematic configuration of a solid-state imaging device capable of outputting a high-sensitivity imaging signal and a low-sensitivity imaging signal.
[Explanation of symbols]
1 ... Imaging unit
2 ... Analog signal processor
3 ... A / D converter
4 ... Drive unit
5 ... Strobe
6 ... Digital signal processor
7: Compression / decompression processor
8 ... Display section
9 ... System controller
10 ... Internal memory
11 ... Media interface
12 ... Recording media
13 ... Operation part
20 ... System bus
61a, 61b... OB (optical black) processing unit
62a, 62b ... LMTX (linear matrix)
63a, 63b ... WB (white balance) processing unit
64a, 64b ... gamma correction section
65 ... Composite gain LUT (look-up table)
66a, 66b ... multiplication unit
67... Addition unit
68 ... Limiter
69 ... Synchronization and Y / C processing section
70 ... CMTX (color difference matrix)
71 ... Integration processing unit
72 .. Composite range calculation unit

Claims (9)

An image synthesizing method for obtaining synthesized image data by synthesizing relatively first image data with relatively high sensitivity and second image data with relatively low sensitivity,
Obtaining a ratio x between the maximum value of the subject brightness indicated by the second image data and the brightness value indicated by the maximum value that can be taken by the first image data;
Correcting the ratio x based on at least one of strobe on / off information, subject average luminance information, and backlight information;
Determining the synthesis processing parameter based on the corrected ratio x ′;
An image composition method including a step of performing composition processing of the first image data and the second image data based on the composition processing parameter. The image composition method according to claim 1,
And determining a gamma correction parameter of the second image data based on the corrected ratio x ′;
Correcting the second image data based on the gamma correction parameter. The image composition method according to claim 1 or 2,
A step of determining a saturation correction parameter of the composite image data based on the corrected ratio x ′;
Correcting the saturation of the composite image data based on the saturation correction parameter. The image synthesizing method according to any one of claims 1 to 3,
The step of correcting the ratio x is an image composition method in which the value of the ratio x is limited based on at least one of strobe on / off information, subject average luminance information, and backlight information. An image synthesizing method for obtaining synthesized image data by synthesizing relatively first image data with relatively high sensitivity and second image data with relatively low sensitivity,
Obtaining a ratio x between the maximum value of the subject brightness indicated by the second image data and the brightness value indicated by the maximum value that can be taken by the first image data;
Determining a gamma correction parameter of the second image data and the synthesis processing parameter based on the ratio x;
Correcting the second image data based on the gamma correction parameter;
An image composition method including a step of performing composition processing of the first image data and the second image data based on the composition processing parameter. An image synthesizing method for obtaining synthesized image data by synthesizing relatively first image data with relatively high sensitivity and second image data with relatively low sensitivity,
Obtaining a ratio x between the maximum value of the subject brightness indicated by the second image data and the brightness value indicated by the maximum value that can be taken by the first image data;
Determining the composition processing parameter and the saturation correction parameter of the composite image data based on the ratio x;
Performing a combining process of the first image data and the second image data based on the combining processing parameter;
Correcting the saturation of the composite image data based on the saturation correction parameter. An image synthesizing method for obtaining synthesized image data by synthesizing relatively first image data with relatively high sensitivity and second image data with relatively low sensitivity,
Obtaining a ratio x between the maximum value of the subject brightness indicated by the second image data and the brightness value indicated by the maximum value that can be taken by the first image data;
Determining a gamma correction parameter of the second image data, the synthesis processing parameter, and a saturation correction parameter of the synthesized image data based on the ratio x;
Correcting the second image data based on the gamma correction parameter;
Performing a combining process of the first image data and the second image data based on the combining processing parameter;
Correcting the saturation of the composite image data based on the saturation correction parameter. The image composition method according to any one of claims 1 to 7,
The step of obtaining the ratio x at the time of electronic zoom photographing uses an object luminance maximum value indicated by the second image data in a different area according to a zoom magnification. A digital camera having a function of performing composite processing of relatively high sensitivity first image data and relatively low sensitivity second image data to obtain composite image data,
A digital camera that performs the composition processing using the image composition method according to any one of claims 1 to 8.

Images