JP5814780B2 - Imaging apparatus and control method thereof - Google Patents

Description

  The present invention relates to an imaging apparatus and a control method thereof.

  A simple addition method is known as a method of multiple exposure shooting with a digital camera. In the simple addition method, the image data obtained by adding a plurality of data obtained from the image pickup device by a plurality of shootings is developed and recorded on a recording medium in the same manner as the multiple exposure shooting by the conventional film camera. Record. An automatic exposure correction addition method is also known. In the automatic exposure compensation addition method, the data obtained from the image sensor by shooting is subjected to a division process that lowers the exposure by the number of shots to be added, then the addition process is performed, and the resulting image data is developed. Then, it is recorded on a recording medium.

  According to the automatic exposure correction addition method, no matter how many shot images are added, the exposure state is not saturated and the image does not fail. Because of these advantages, the automatic exposure compensation addition method is becoming popular as a method for multiple exposure shooting with a digital camera.

  Patent Document 1 is known as an invention related to an automatic exposure correction addition method. Hereinafter, a conventional automatic exposure correction addition method will be described with reference to FIG.

  Image data composed of RGB color components captured by the image sensor 300 of the digital camera and digitized by the A / D conversion unit 301 is subjected to white balance processing for each RGB signal by the white balance processing unit 311. A plurality of coefficient candidates used in the white balance processing unit 311 are prepared in advance as a white balance table 303. A user of the digital camera selects a coefficient to be used by the white balance table selection instruction input unit 302. Thereby, the white balance process corresponding to the color temperature of various light sources which irradiate a to-be-photographed object can be performed.

  The user can designate the number N of shooting cuts for multiple exposure shooting by using the multiple shot number instruction input unit 304. The image data subjected to the white balance processing is sequentially sent to the image memories 390-1 to 390-N (one memory for one screen) having a capacity of N sheets for each photographing. The N pieces of image data stored in the image memories 390-1 to 390 -N are read out for multiple addition after N times of photographing have been completed. Next, the gain correction processing unit 351 performs a process for reducing the exposure for each image data by the number of shooting cuts for multiple exposure shooting. For example, when N = 2, a process of multiplying the gain of 1/2 in the linear image data area subjected to the white balance process is performed so that the exposure of each image data is lowered by one stage. Similarly, when N = 3, a process of multiplying each image data by 1/3 is performed.

  The image data multiplied by the gain is added by the adder 361. Thereby, multiplexed RGB image data is generated. Thereafter, the multiplexed linear RGB image data is subjected to matrix conversion into a luminance component (Y component) and a color difference component (Cr, Cb component) by a matrix conversion processing unit 312.

  For the Y component, the luminance gamma processing unit 313 performs gamma processing, which is nonlinear image processing. The sharpness calculation processing unit 314 detects an edge component for performing sharpness processing on the multiplexed image from the Y component that has been subjected to gamma processing. The sharpness addition processing unit 315 performs sharpness processing for adding the detected edge component to the Y component that has been subjected to gamma processing. In the sharpness addition processing unit 315, for example, as indicated by reference numeral 601 in FIG. 6, a process of changing the weight of the sharpness to be applied according to the luminance of the image is also performed. In order to prevent sharpness from being applied to noise components in the low-brightness area, gain is applied so that the weight is lowered as the brightness is lower. Multiply.

  On the other hand, for the Cr and Cb components, the high luminance color suppression processing unit 316 performs color suppression processing that suppresses the color difference component according to the size of the Y component. Here, as indicated by reference numeral 501 in FIG. 5, a process is performed in which the color difference component of the high luminance part is multiplied by a negative gain so that the color is beautifully skipped in the high luminance part.

  Next, the color gamma processing unit 317 performs gamma processing, which is nonlinear image processing, on the Cr and Cb components. The color correction processing unit 318 performs color correction processing on the Cr and Cb components that have been subjected to gamma processing. Here, as indicated by reference numeral 801 in FIG. 8A, a gain is applied to the color difference signal, and a characteristic that weakens the gain in the high saturation region than in the low saturation region so as not to cause color saturation is given. The color of the output image is finished to a preferable state.

  Thereafter, the Y component, Cr, and Cb components are compressed and encoded by the file forming unit 362 and then filed, and are stored in the recording medium by the media recording unit 363. As described above, multiple-exposure shooting can be performed without saturation and failure of an image no matter how many times it is multiplexed.

JP 2006-128740 A

  In the conventional multiple exposure shooting method disclosed in Patent Document 1, it is possible to perform white balance processing corresponding to the color temperature of the light source that irradiates the subject by switching the white balance processing parameters for each cut. It is. However, for subsequent nonlinear image processing such as gamma processing and sharpness processing, the parameters cannot be changed for each cut. Therefore, it is difficult to perform optimal image processing for each cut.

  In addition, since the brightness of each cut is reduced by the gain correction processing in the gain correction processing unit 351, the subsequent image processing cannot be performed at the luminance level that should be originally processed. Specifically, for example, in the color suppression processing performed by the high luminance color suppression processing unit 316 in FIG. 3, as shown in FIG. 5, a gain for suppressing color saturation is given to a high luminance subject. think of. In this case, since the gain correction processing unit 351 applies a negative gain of 1 / N to the image data, the luminance is actually 1 / N even for a high-luminance subject. Therefore, only a negative gain that is weaker than the gain corresponding to the original luminance level before the 1 / N negative gain is applied to the high-luminance subject. As a result, sufficient color suppression is not performed on a high-luminance subject.

  Similarly, the sharpness processing performed by the sharpness calculation processing unit 314 and the sharpness addition processing unit 315 in FIG. 3 will be considered. As shown in FIG. 6, a case is considered in which a characteristic for weighting sharpness intensity according to luminance is given. In this case, even if the subject is actually a medium luminance, the luminance is reduced to 1 / N by the gain correction processing unit 351, so that a small weighting coefficient corresponding to the low luminance subject is selected. As a result, the image quality becomes blurred without applying sufficient sharpness processing corresponding to the original luminance. The same is true for high-luminance subjects. That is, the characteristics in FIG. 6 are intended to suppress the sharpness of a high-brightness object so that it looks natural, but the sharpness is originally intended because the brightness of the high-brightness object is reduced by the gain correction process. It cannot be suppressed as it did.

  The present invention has been made in view of such a situation, and an object thereof is to provide a technique for improving the image quality of image data obtained by multiple exposure shooting.

In order to solve the above-mentioned problems, a first aspect of the present invention is an imaging unit that shoots a subject to generate image data, and a first image processing unit that performs nonlinear image processing on the generated image data. Second image processing means for performing predetermined image processing on the image data subjected to the nonlinear image processing; and inverse characteristics to the nonlinear image processing for the image data subjected to the predetermined image processing. A third image processing means for performing the image processing, and N images generated by N times (N ≧ 2) photographing by the imaging means and processed by the first, second, and third image processing means Generating means for generating multiple-exposure image data by multiplying each of the data by multiplying each image data by a gain of 1 / N; and fourth for performing the non-linear image processing on the multiple-exposure image data. Image processing means , Wherein the first image processing means performs luminance gamma processing to the luminance component of the image data the generated, the second image processing means, with respect to the luminance component and the brightness gamma processing has been performed Sharpness processing is performed, and the third image processing means performs reverse luminance gamma processing having a reverse characteristic to the luminance gamma processing on the luminance component subjected to the sharpness processing, and the fourth image processing means Provided is an imaging apparatus characterized by performing the luminance gamma processing on luminance components of multiple exposure image data .

  Other features of the present invention will become more apparent from the accompanying drawings and the following description of the preferred embodiments.

  With the above configuration, according to the present invention, it is possible to improve the image quality of image data obtained by multiple exposure shooting.

1 is a block diagram showing a functional configuration of a digital camera according to a first embodiment. 5 is a flowchart showing a flow of multiple exposure shooting processing according to the first embodiment. The figure which shows the example of the system of the conventional multiple exposure imaging | photography. The conceptual diagram of a gamma process and a reverse gamma process. The conceptual diagram of a high-intensity color suppression process. The conceptual diagram of the weighting coefficient of sharpness intensity | strength. The conceptual diagram of a sharpness process. The conceptual diagram of a color correction process and a reverse color correction process.

  Embodiments of the present invention will be described below with reference to the accompanying drawings. The technical scope of the present invention is determined by the claims, and is not limited by the following individual embodiments. In addition, not all combinations of features described in the embodiments are essential to the present invention.

[First Embodiment]
An embodiment in which an imaging apparatus of the present invention is applied to a digital camera will be described. FIG. 1 is a block diagram illustrating a functional configuration of the digital camera according to the first embodiment. FIG. 2 is a flowchart showing the flow of the multiple exposure shooting process according to the first embodiment.

  In step S <b> 201, the user designates the number of cuts to be multiplexed using the multiple shot number instruction input unit 104. In S <b> 202, S <b> 203, and S <b> 204, the digital camera captures a subject and generates image data using the image sensor 100 and the A / D converter 101. In the present embodiment, the generated image data includes three components of R (red), G (green), and B (blue).

  In step S <b> 205, the user uses the image processing table selection instruction input unit 102 to select a white balance coefficient (or a photographing scene related to the white balance coefficient) from the white balance table 151. For example, the white balance table 151 stores in advance white balance coefficients suitable for outdoor sunlight, white balance coefficients suitable for indoor fluorescent lights, and the like, and the user selects a white balance coefficient by selecting a shooting scene. can do. Of course, the digital camera may include an auto white balance function that automatically determines the light source and automatically calculates the optimum white balance coefficient.

  In S206, the white balance processing unit 111 (an example of a fifth image processing unit) performs white balance processing on the image data generated in S202 to S204 according to the white balance coefficient selected in S205. The image data that has been subjected to the white balance processing is represented by RGB linear digital data at this point.

  In S207, the matrix conversion processing unit 112 converts the image data subjected to the white balance processing into a luminance component (luminance signal Y) and a color difference component (two color difference signals Cr and Cb) by a three-dimensional conversion process.

  In S208, the high luminance color suppression processing unit 119 (an example of the fifth image processing means) uses the high luminance color for suppressing color saturation (color distortion) generated by the high luminance signal with respect to the color difference signals Cr and Cb. Apply suppression processing. Here, as shown in FIG. 5, image processing is performed so that the color difference signals Cr and Cb are multiplied by an equal negative gain in the high-luminance portion so that the color of the high-luminance portion naturally goes out white. For example, a green leaf that is exposed to strong sunlight avoids being saturated as the intensity of sunlight increases, resulting in an unnatural blue-green color. Performs processing to lower the color gain so that it is clear and white.

  In the image processing after S208, gamma conversion processing for converting the original image data into non-linear image data is performed so that the brightness becomes optimum when output is displayed on a television monitor or printed on paper.

  Specifically, in step S209, the digital camera performs gamma characteristics from the luminance signal gamma / inverse gamma table 152 and the color signal gamma / inverse gamma table 153 in accordance with an instruction from the user through the image processing table selection instruction input unit 102. Select. The luminance signal gamma / inverse gamma table 152 and the color signal gamma / inverse gamma table 153 store gamma characteristics as indicated by reference numerals 403 to 405 in FIG. In the following, it is assumed that the gamma characteristic indicated by reference numeral 405 is selected for both the luminance signal gamma / inverse gamma table 152 and the color signal gamma / inverse gamma table 153.

  In S210, the luminance gamma processing unit 114 (an example of the first image processing unit) performs luminance gamma processing on the luminance signal Y according to the gamma characteristic selected in S209. For example, if the luminance signal Y is 8-bit linear characteristic data, the non-linear characteristic as indicated by reference numeral 401 in FIG. 4A (which corresponds to reference numeral 405 in FIG. 4B) has a luminance signal. Given to Y. As a result, a luminance signal having characteristics suitable for the output of a monitor or printer can be obtained. Similarly, the color gamma processing unit 120 (an example of the first image processing unit) performs color gamma processing on the color difference signals Cr and Cb according to the gamma characteristic selected in S209.

  In step S <b> 211, the digital camera selects a sharpness characteristic from the sharpness table 154 in accordance with an instruction from the user through the image processing table selection instruction input unit 102. The sharpness table 154 stores sharpness characteristics as indicated by reference numerals 601 to 603 in FIG. The sharpness characteristic in FIG. 6 is a characteristic for giving a negative gain to the sharpness signal so that the noise component is not strongly sharpened in the low luminance range. It is also a characteristic for giving a negative gain to the sharpness signal so that it does not become unsightly due to strong sharpness applied to places where high sharpness is applied to high luminance areas and expresses fine details in high luminance areas. .

  In S212, the sharpness calculation processing unit 115 detects an edge component of the luminance signal Y that has been subjected to the luminance gamma processing. The edge component is detected by, for example, calculating the amount of change between neighboring pixels of each pixel. For example, assuming that an edge is detected from the difference between reference numerals 701 and 702 in FIG. 7A, the sharpness calculation processing unit 115, as shown in FIG. 7B, sharpness intensity levels applied to the edge and its surrounding pixels. 703 and 704 are obtained. The sharpness calculation processing unit 115 multiplies the sharpness signal components corresponding to the sharpness intensity levels 703 and 704 by a gain corresponding to the magnitude of the luminance signal Y according to the sharpness characteristic selected in S211.

  The sharpness addition processing unit 116 (an example of the second image processing unit) adds the sharpness signal component calculated by the sharpness calculation processing unit 115 to the luminance signal Y that has been subjected to the luminance gamma processing (reference numeral in FIG. 7C). 705 to 707). Thereby, sharpness processing is performed on the luminance signal Y.

  In step S <b> 213, the digital camera selects a color correction characteristic from the color correction / reverse color correction table 155 in accordance with an instruction from the user through the image processing table selection instruction input unit 102. The color correction / reverse color correction table 155 stores color correction characteristics as indicated by reference numerals 803 to 805 in FIG. 8B. In the following, it is assumed that the color correction characteristic indicated by reference numeral 805 is selected.

  In step S214, the color correction processing unit 121 (an example of the first image processing unit) selects the color correction characteristics selected in step S213 for the color difference signals Cr and Cb that have been subjected to color gamma processing in order to suppress color saturation. The color correction process is performed according to For example, nonlinear characteristics as indicated by reference numeral 801 in FIG. 8A (corresponding to reference numeral 805 in FIG. 8B) are given to the color difference signals Cr and Cb. In other words, the gain is applied to the low saturation portion in a direction to make the saturation vivid, while the gain is gradually reduced to prevent color saturation in the high saturation portion.

  Each table of each image processing described above depends on the spectral characteristics of the RGB image data output by the image sensor 100. The characteristics (parameters) of the tables 152 to 155 are determined in advance after investigating the characteristics of the image sensor 100 using various standard charts such as color patches.

  With the above processing, image data obtained by one shooting can be developed to obtain image data having a non-linear characteristic and suitable for output on a monitor or printer. Next, processing for returning non-linear image data to linear image data in order to generate multiple exposure image data will be described.

  In step S215, the reverse color correction processing unit 122 (an example of the third image processing unit) performs image processing (reverse color correction) on the color difference signals Cr and Cb on which the color correction processing has been performed. Process). The color correction / reverse color correction table 155 also stores color correction characteristics of reverse color correction processing. When the color correction characteristic for the color correction process is selected in S213, the color correction characteristic opposite to the selected color correction characteristic is selected as the color correction characteristic for the reverse color correction process. As described above, since the color correction characteristic for the color correction process is indicated by reference numeral 801 in FIG. 8A, the color correction characteristic for the reverse color correction process is indicated by reference numeral in FIG. This is indicated by 802.

  In S216, the inverse luminance gamma processing unit 117 (an example of the third image processing unit) performs image processing (reverse luminance gamma processing) having characteristics opposite to those of the luminance gamma processing in S210 on the luminance signal Y subjected to the sharpness processing. Apply. The luminance signal gamma / inverse gamma table 152 also stores gamma characteristics of reverse luminance gamma processing. When a gamma characteristic for luminance gamma processing is selected in S210, a gamma characteristic opposite to the selected gamma characteristic is selected as a gamma characteristic for reverse luminance gamma processing. As described above, since the gamma characteristic for the luminance gamma processing is indicated by reference numeral 401 in FIG. 4A, the gamma characteristic for the reverse luminance gamma processing is indicated by reference numeral 402 in FIG. It is shown. Similarly, the reverse color gamma processing unit 123 (an example of the third image processing unit) performs image processing (reverse color) on the color difference signals Cr and Cb that have been subjected to the reverse color correction processing, in reverse characteristics to the color gamma processing in S210. Gamma treatment). The color signal gamma / inverse gamma table 153 also stores gamma characteristics of inverse color gamma processing. When a gamma characteristic for color gamma processing is selected in S210, a gamma characteristic opposite to the selected gamma characteristic is selected as a gamma characteristic for reverse color gamma processing. As described above, since the gamma characteristic for color gamma processing is indicated by reference numeral 401 in FIG. 4A, the gamma characteristic for reverse color gamma processing is indicated by reference numeral 402 in FIG. It is shown.

  In the luminance signal Y and the color difference signals Cr and Cb obtained by the image processing in S215 and S216, the image formation subjected to sharpness processing and the image formation subjected to high luminance color suppression processing are left. However, basic image data has been returned to almost linear image creation. The luminance signal Y and the color difference signals Cr and Cb are stored in the image memory 190-1 as data for one image in S217. In order to generate the multiple exposure image data, the digital camera performs control of repeating the processes of S202 to S217 N times (S218). In this way, N pieces of image data obtained by N times of photographing are sequentially stored in the image memories 190-1 to 190-N.

  Thereafter, in S219, the gain correction processing unit 130 reads N pieces of image data from the image memories 190-1 to 190-N. In step S220, the gain correction processing unit 130 multiplies each of the image data read in step S219 by a gain of 1 / N. In S221, the multiple addition processing unit 131 adds N pieces of image data multiplied by a gain of 1 / N. Thereby, one piece of multiple exposure image data is generated.

  In S222, the luminance gamma processing unit 142 (an example of the fourth image processing unit) performs luminance gamma processing having the same characteristics as the luminance gamma processing in S210 on the luminance component (luminance signal Y) of the multiple exposure image data. Similarly, the color gamma processing unit 144 (an example of the fourth image processing unit) performs color gamma processing having the same characteristics as the color gamma processing in S210 on the color difference components (color difference signals Cr and Cb) of the multiple exposure image data. Apply. Furthermore, in S223, the color correction processing unit 145 (an example of the fourth image processing unit) performs color correction with the same characteristics as the color correction processing in S214 on the color difference components (color difference signals Cr and Cb) of the multiple exposure image data. Apply processing. Thereby, final multiple exposure image data is generated.

  In S224, the filing unit 146 converts the multiple exposure image data into a file, and in S225, the media recording unit 147 records the filed multiple exposure image data on a recording medium.

  As described above, according to the present embodiment, the digital camera applies to each image data before multiplying the N pieces of image data for generating the multiple exposure image data by a gain of 1 / N. Various image processing including nonlinear image processing is performed. At that time, for example, sharpness processing is performed after luminance gamma processing which is nonlinear image processing. After that, the digital camera subjects each image data to image processing having a reverse characteristic to the applied non-linear image processing, multiplies each image data by multiplying each image data by multiplying each image data by multiple exposure. Generate image data. The digital camera performs the above-described nonlinear image processing again on the multiple exposure image data thus generated.

  Thus, according to the present embodiment, predetermined image processing such as sharpness processing is performed before a gain of 1 / N is applied to image data. Accordingly, predetermined image processing such as sharpness processing is performed according to the original luminance of the subject (luminance before the gain of 1 / N is applied). Thereby, the image quality of the image data obtained by the multiple exposure shooting is improved.

[Other Embodiments]
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

Claims (5)

An imaging means for capturing a subject and generating image data;
First image processing means for performing nonlinear image processing on the generated image data;
Second image processing means for performing predetermined image processing on the image data subjected to the nonlinear image processing;
Third image processing means for performing image processing with a reverse characteristic to the nonlinear image processing on the image data subjected to the predetermined image processing;
A gain of 1 / N is applied to each of the N pieces of image data generated by N times (N ≧ 2) photographing by the image pickup means and processed by the first, second, and third image processing means. Generating means for generating multiple exposure image data by multiplying and adding each image data;
Fourth image processing means for performing the non-linear image processing on the multiple exposure image data;
Equipped with a,
The first image processing means performs a luminance gamma process on the luminance component of the generated image data,
The second image processing means performs sharpness processing on the luminance component subjected to the luminance gamma processing,
The third image processing means performs reverse luminance gamma processing having a reverse characteristic to the luminance gamma processing on the luminance component subjected to the sharpness processing,
The fourth image processing means performs the luminance gamma processing on the luminance component of the multiple exposure image data.
An imaging apparatus characterized by that. The first image processing means performs color gamma processing and color correction processing on the color difference component of the generated image data,
The third image processing means performs reverse color gamma processing and reverse color correction processing having reverse characteristics to the color gamma processing and color correction processing on the color difference component subjected to the color gamma processing and color correction processing,
The imaging apparatus according to claim 1, wherein the fourth image processing unit performs the color gamma processing and the color correction processing on a color difference component of the multiple exposure image data. A white balance process is performed on the generated image data before the nonlinear image process is performed, and a high luminance color suppression process is performed on the color difference component of the image data subjected to the white balance process. the imaging apparatus according to claim 1 or 2, further comprising an image processing means. A method for controlling an imaging apparatus,
An imaging step in which the imaging means of the imaging device captures a subject and generates image data;
A first image processing step in which a first image processing means of the imaging device performs nonlinear image processing on the generated image data;
A second image processing step in which the second image processing means of the imaging device performs predetermined image processing on the image data subjected to the nonlinear image processing;
A third image processing step in which a third image processing unit of the imaging device performs image processing having a reverse characteristic to the nonlinear image processing on the image data on which the predetermined image processing has been performed;
The generation unit of the imaging device generates N image data generated by N times (N ≧ 2) of the imaging process and processed by the first, second, and third image processing processes. A generating step of generating multiple exposure image data by multiplying each image data by multiplying the gain by 1 / N;
A fourth image processing step in which a fourth image processing means of the imaging device performs the nonlinear image processing on the multiple exposure image data;
Equipped with a,
In the first image processing step, a luminance gamma process is performed on a luminance component of the generated image data,
In the second image processing step, sharpness processing is performed on the luminance component subjected to the luminance gamma processing,
In the third image processing step, the luminance component subjected to the sharpness processing is subjected to reverse luminance gamma processing having a reverse characteristic to the luminance gamma processing,
In the fourth image processing step, the luminance gamma processing is performed on the luminance component of the multiple exposure image data.
A control method characterized by that. The program for making a computer perform each process of the control method of Claim 4 .

Images