JP4066090B2 - Image processing apparatus, image processing program, and image processing method - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image processing apparatus and method for adjusting a gain of a saturation level according to a luminance level of an image signal.
The present invention relates to an image processing program for realizing the image processing apparatus on a computer.
[0002]
[Prior art]
2. Description of the Related Art Conventionally, an electronic camera that adjusts a saturation level gain according to the luminance level of an image signal is known.
In such an electronic camera, the color noise in the low luminance region can be reduced by suppressing the saturation level in the low luminance region of the image signal.
Further, in such an electronic camera, it is possible to suppress unnatural coloring seen in the high luminance region by suppressing the saturation level in the high luminance region of the image signal (for example, Japanese Patent Laid-Open No. 5-244623). See the publication).
[0003]
[Problems to be solved by the invention]
By the way, the color representation of the image signal changes sensitively depending on the color temperature of the light source, the color and brightness of the subject, and the imaging conditions of the electronic camera.
If the color noise in the low-brightness area is reduced at a certain rate without considering such changes in color expression, the color noise in the low-brightness area cannot be removed sufficiently. There are concerns about harmful effects such as losing more than necessary.
In view of such a problem, the present invention accurately determines a saturation modulation characteristic (a characteristic in which a saturation level gain is determined with respect to a luminance level) for various image signals. For the purpose.
[0004]
[Means for Solving the Problems]
The present invention will be described below.
<Claim 1>
An image processing apparatus according to a first aspect is an image processing apparatus that processes an image signal generated by an imaging unit, and includes a saturation modulation control unit and a saturation modulation unit.
The saturation modulation control unit, Ri each impact is Do different with affecting the color representation of the image signal, and a plurality of types of color-related information to the information acquisition including photographing conditions for generating an image signal by capturing Then, each color-related information value is converted in consideration of the difference in the degree of influence, and according to the combination of the saturation modulation characteristics based on the color-related information after the conversion , Controls the saturation modulation characteristic that determines the gain of the degree level.
Further, the saturation modulation unit performs gain adjustment of the saturation level according to the luminance level of the image signal in accordance with the saturation modulation characteristic controlled by the saturation modulation control unit.
By such an operation, it is possible to control the saturation modulation characteristic in response to a change in color expression estimated from the color related information.
<Claim 2>
The image processing apparatus according to claim 2 employs the imaging sensitivity of the imaging unit as at least one of the color related information. The saturation modulation control unit acquires information on the imaging sensitivity when the image signal is captured, and controls the saturation modulation characteristic according to the imaging sensitivity.
With such an operation, it is possible to control the saturation modulation characteristic in response to a change in color expression estimated from the imaging sensitivity.
<Claim 3>
The image processing apparatus according to claim 3 employs gradation conversion characteristics applied to the image signal as at least one of the color related information.
The saturation modulation control unit controls the saturation modulation characteristic according to the gradation conversion characteristic.
With such an operation, it is possible to control the saturation modulation characteristic in response to a change in color expression estimated from the gradation conversion characteristic.
<Claim 4>
An image processing program according to a fourth aspect causes a computer to function as the saturation modulation control unit and the saturation modulation unit according to any one of the first to third aspects.
<Claim 5>
The image processing method according to claim 5, the image the image signal generated by the imaging section an image processing method for image processing, Ri Do different each degree of influence with affects the color representation of the image signal, and an imaging Acquire information on a plurality of types of color related information including shooting conditions when generating a signal, convert each color related information value in consideration of the difference in the degree of influence, and each color related information after the conversion The step of controlling the “saturation modulation characteristic with a gain of the saturation level relative to the luminance level” according to the combination of the saturation modulation characteristics based on the luminance level of the image signal according to the controlled saturation modulation characteristic And adjusting the saturation level gain according to the step.
[0005]
DETAILED DESCRIPTION OF THE INVENTION
Embodiments according to the present invention will be described below with reference to the drawings.
[0006]
[General Description of Electronic Camera 11]
FIG. 1 is a diagram illustrating a configuration of an electronic camera 11 (including an image processing apparatus) according to the present embodiment. First, the configuration and schematic operation of the electronic camera 11 will be described with reference to FIG.
In FIG. 1, a photographing lens 12 is attached to the electronic camera 11. Inside the photographic lens 12, an in-lens MPU 12a is incorporated. On the image space side of the photographing lens 12, a light receiving surface of the image sensor 13 is disposed with a shutter mechanism (not shown) interposed therebetween.
[0007]
The image signal output from the image sensor 13 is output to the A / D converter 16 after the gain is adjusted by the amplifier 14 in accordance with the imaging sensitivity. The A / D conversion unit 16 digitizes the image signal for each pixel and outputs the digitized signal to the white balance processing unit 17, the color distribution evaluation unit 17 a, and the luminance distribution evaluation unit 25.
The color distribution evaluation unit 17a evaluates the color distribution of the image signal and estimates the color temperature of the light source. The white balance processing unit 17 performs white balance adjustment on the image signal according to the estimated value of the color temperature.
[0008]
The interpolation processing unit 18 performs a process of interpolating the missing signal component on the image signal after the white balance adjustment.
The noise suppression unit 19 performs noise suppression processing such as local product-sum operation on the image signal after the interpolation processing.
The gamma conversion unit 20 selects a gradation conversion characteristic according to the luminance distribution evaluation of the luminance distribution evaluation unit 25. The gamma conversion unit 20 performs gradation conversion on the image signal after noise suppression according to the selected gradation conversion characteristic.
[0009]
The matrix conversion unit 21 performs color coordinate conversion on the image signal after gradation conversion and converts it into a signal component such as YCbCr. At this time, the matrix conversion unit 21 uniformly controls the conversion gain of the signal component CbCr relating to the color with respect to the entire image, and adjusts the color reproducibility of the entire image. The conversion gain (that is, the color reproducibility adjustment value) of the color signal component is a value set based on a manual setting by the user, a photometric contrast of the subject, or the like.
[0010]
The matrix conversion unit 21 outputs the luminance component Y to the contour adjustment unit 22 and outputs the color difference component CbCr whose color reproducibility is adjusted to the color noise suppression unit 23.
The contour adjustment unit 22 performs contour enhancement processing such as an unsharp mask on the luminance component Y.
On the other hand, the color noise suppression unit 23 changes the signal gain of the color difference component CbCr according to the following equation in accordance with the signal level of the luminance component Y.
Cb ′ = Cb · K (Y) / 128
Cr ′ = Cr · K (Y) / 128
Here, K (Y) is a value determined with reference to a function table (for example, see FIG. 3) in which the gain of the saturation level is determined according to the luminance component Y. By the processing here, for example, color noise that tends to appear in a low luminance region or a high luminance region of an image can be selectively suppressed.
[0011]
After completing such a series of signal processing, the electronic camera 11 performs processing such as compression / recording storage on the image signal YCbCr.
In addition to the signal processing configuration described above, the electronic camera 11 is provided with a divided photometric sensor 29, a built-in or external flash device 30, and a shooting control unit 31 as a configuration for an imaging operation. It is done.
[0012]
The split photometry sensor 29 performs split photometry on the object scene or in a direct TTL (Through The Lens) method and outputs split photometry values. Based on this divided photometric value, the photographing control unit 31 determines an exposure condition at the time of imaging, the aperture value of the photographing lens 12, the shutter speed of the imaging element 13, the imaging sensitivity (gain) of the amplifier 14, and the flash device 30. Control the timing of light emission.
Further, the electronic camera 11 is provided with a saturation modulation control unit 24 as a characteristic component of the present invention.
[0013]
[Correspondence with Invention]
Next, the correspondence between the description items in the claims and the present embodiment will be described. Note that the correspondence relationship here illustrates one interpretation for reference, and does not limit the present invention.
The imaging unit described in the claims corresponds to the photographing lens 12, the imaging element 13, the amplifier 14, and the photographing control unit 31.
The saturation modulation control unit described in the claims corresponds to the saturation modulation control unit 24.
The saturation modulation unit described in the claims corresponds to the “function for varying the gain of the color difference component CbCr according to the luminance level” of the color noise suppression unit 23.
[0014]
[Description of Operation of Saturation Modulation Control Unit 24]
FIG. 2 is a flowchart for explaining the operation of the saturation modulation control unit 24. Hereinafter, the operation of the saturation modulation control unit 24 will be described along the step numbers in FIG.
[0015]
Step S1: The saturation modulation control unit 24 acquires color-related information that affects the color expression of the image signal for the image signal to be processed.
The “color expression” means how the color appears when an image signal is displayed or printed. For example, “color expression” includes color vividness, clarity, color dullness, hue bias, color tone (high key, low key, temperature control, cold control, etc.), how to see subtle color differences, How to see the color of the flat part, how the color of the edge part looks, how the color of the detail part looks, how the color of the low brightness area looks, how the color of the highlight part looks, color shift, unnatural color Or color noise.
For example, the saturation modulation control unit 24 collects information on the following color-related information.
(A) Color reproducibility adjustment value transmitted from the matrix conversion unit 21 (B) Imaging sensitivity information transmitted from the imaging control unit 31 (C) Tone conversion characteristic information transmitted from the gamma conversion unit 20 ( D) Photometric value (divided photometric value) of the divided photometric sensor 29
(E) Illumination information transmitted from the MPU 30a in the flash device (F) Exposure conditions transmitted from the imaging control unit 31 (G) Lens information transmitted from the MPU 12a in the lens (H) Transmitted from the color distribution evaluation unit 17a Color information (for example, light source color temperature, hue angle, saturation, etc.)
(I) Color concentration degree transmitted from the color distribution evaluation unit 17a (a value indicating the degree of screen occupation of color)
[0016]
Step S2: The saturation modulation control unit 24 normalizes these pieces of color related information. It should be noted that the appropriate increment of normalization varies greatly depending on the characteristics of the image sensor 13 to be used. Therefore, it is preferable to perform an imaging experiment of the electronic camera 11 to determine the influence of the color related information on the color expression, and to normalize the color related information having a larger influence in fine steps.
[0017]
Step S3: The saturation modulation control unit 24 generates a unique data reference address by combining the normalized values of the individual color related information. This data reference address corresponds to the address of the internal data area of the saturation modulation control unit 24. In this internal data area, the designation number of the saturation modulation characteristic is stored in correspondence with each combination of normalized values. These instruction numbers for the saturation modulation characteristics are determined in advance according to the evaluation of actual shooting and the rules described later.
[0018]
Step S4: The saturation modulation control unit 24 refers to the internal data area based on the data reference address, and obtains an instruction number for the saturation modulation characteristic. The saturation modulation control unit 24 transmits the instruction number thus obtained to the color noise suppression unit 23. The color noise suppression unit 23 selects the saturation modulation characteristic from the internal function table according to the instruction number, and executes the saturation modulation of the image signal.
[0019]
[Adjustment tendency of saturation modulation characteristics]
In each electronic camera, the optimum saturation modulation characteristic varies greatly depending on differences in the dynamic range, noise characteristics, and the like of the image sensor 13 to be used. Therefore, in the present specification, instead of increasing specific adjustment values for individual electronic cameras, more essential “saturation modulation characteristic adjustment tendency” will be described in detail.
A person skilled in the art can specifically implement this embodiment by changing a predetermined standard saturation modulation characteristic (hereinafter referred to as “standard characteristic”) according to the adjustment tendency described below.
[0020]
(1) Saturation Modulation Characteristic Based on Color Reproducibility Adjustment Value FIG. 3 is a diagram showing the adjustment tendency of the saturation modulation characteristic based on the color reproducibility adjustment value. FIG. 3 shows a plurality of curves of saturation modulation characteristics. By selecting and combining the low luminance region curve and the high luminance region curve from these curves, the saturation modulation characteristic can be selected with a high degree of freedom.
Normally, when the color reproducibility adjustment value increases, the signal level of the color signal generally increases. In this case, the color noise in the low luminance region and the unnatural coloring in the high luminance region are emphasized and become easily noticeable. Therefore, as shown in FIG. 3, the saturation modulation control unit 24 controls the saturation gains of the low luminance region and the high luminance region to be lower than the standard characteristics as the color reproducibility adjustment value increases. As a result, even if the color reproducibility is set higher than usual, the saturation levels of the low luminance region and the high luminance region are moderately suppressed, and color noise and unnatural coloring are not so much emphasized.
[0021]
(2) Saturation Modulation Characteristic Based on Adjustment Value of Imaging Sensitivity FIG. 4 is a diagram showing the adjustment tendency of the saturation modulation characteristic based on the setting value of imaging sensitivity.
Usually, when the set value of the imaging sensitivity increases, the S / N of the color signal generally deteriorates. In this case, the color noise in the low luminance region and the unnatural coloring in the high luminance region are deteriorated and become conspicuous. Therefore, as shown in FIG. 4, the saturation modulation control unit 24 controls the saturation gains of the low luminance region and the high luminance region to be lower as the imaging sensitivity setting value is higher. As a result, when the imaging sensitivity is set high, the saturation levels of the low luminance region and the high luminance region are suppressed, and the color noise and unnatural coloring are not so emphasized.
[0022]
(3) Saturation modulation characteristic based on adjustment value of gradation conversion characteristic FIG. 5 is a diagram illustrating an example of a soft gradation conversion characteristic. This soft gradation conversion characteristic is a gamma curve characteristic, and has a low luminance gain (gradient of gradation conversion in the low luminance area) and a high luminance gain (gradient of gradation conversion in the high luminance area). Set to medium.
On the other hand, FIG. 6 is a diagram illustrating an example of a hard gradation conversion characteristic. This high gradation conversion characteristic is an S-shaped characteristic that expands the contrast of the intermediate luminance region, and the low luminance gain is set small and the high luminance gain is also set small.
The saturation modulation control unit 24 controls the saturation modulation characteristic according to the low luminance gain and the high luminance gain of the gradation conversion characteristic.
FIG. 7 is a diagram illustrating the adjustment tendency of the saturation modulation characteristic based on the setting of the gradation conversion characteristic.
When the low luminance gain is increased by setting the gradation conversion characteristics, the amplitude level of the color signal in the low luminance region is increased, and the color noise is deteriorated. Therefore, as shown in FIG. 7, the saturation modulation control unit 24 controls the saturation gain of the low luminance region to be lower as the low luminance gain is increased. As a result, even if the low luminance gain is set large in the gradation conversion characteristics, the saturation level in the low luminance region is moderately suppressed, and the color noise is not so emphasized.
On the other hand, when the high luminance gain is increased by setting the gradation conversion characteristics, the amplitude level of the color signal in the high luminance region is increased, and unnatural coloring becomes conspicuous. Therefore, as shown in FIG. 7, the saturation modulation control unit 24 controls the saturation gain of the high luminance region to be lower as the high luminance gain is increased. As a result, even when a high luminance gain is set to be large in the gradation conversion characteristics, the saturation level of the high luminance region is moderately suppressed, and unnatural coloring is not so much emphasized.
[0023]
[Supplementary items of the embodiment]
In the embodiment described above, the saturation modulation characteristic is controlled by using a plurality of pieces of color related information such as color reproducibility, imaging sensitivity, and gradation conversion characteristics. Therefore, when it is determined that color noise or coloring is deteriorated in any one of the color related information, safe control such as suppression of saturation is realized. Conversely, even if color noise or coloring in some color-related information deteriorates, if it is judged that color noise or coloring is improved in other color-related information, the coordination of reducing the degree of saturation suppression is reduced. Can be realized.
[0024]
As described above, in the present embodiment, by using a plurality of pieces of color related information, it is possible to grasp the state of color expression of the image signal in detail and accurately, and to control the saturation modulation characteristic more appropriately .
[0025]
In such coordinated control, it is preferable that the standard characteristic is updated in order for each color-related information and the final update result is selected as the saturation modulation characteristic. Furthermore, regarding the order, it is safe, accurate, and preferable that color related information having a large influence on color noise and coloring is performed later (especially, information that works in a suppressive manner). When this order is fixedly implemented, it is preferable that the invention be implemented simply and quickly by referring to the table as in the above-described embodiment.
[0026]
In the above-described embodiment, saturation modulation is performed by manipulating the value of the signal component CbCr. However, the present invention is not limited to this. In general, in any color system (for example, a primary color system, a complementary color system, a luminance color difference color system, a Lab color system, or an HSB color system), a signal component related to saturation is manipulated. That's fine.
[0027]
In the above-described embodiment, the saturation modulation characteristic of the type that suppresses the saturation level (for example, see FIG. 3) has been described. However, the saturation modulation characteristic of the present invention is not limited to such a suppression type. A saturation modulation characteristic of a type that enhances the saturation level in some luminance regions may be an option. In this way, by enhancing the saturation in some luminance regions, the color dullness is corrected by dealing with the dullness of the color in some luminance regions (such as the skin color luminance region) due to the influence of the color-related information. It becomes possible.
[0028]
In the above-described embodiment, the electronic camera 11 has been described. However, the present invention is not limited to this. For example, the present invention may be realized as a single image processing apparatus separately from an imaging unit (such as an electronic camera or a video camera).
[0029]
Further, an image processing program (corresponding to claim 4 ) may be created by program-coding the processing operations related to the present invention described above. By executing this image processing program, the computer can function as the image processing apparatus of the present invention. Furthermore, it is also possible to provide the image processing method as described above via a communication line such as the Internet.
[0030]
In the above description, the example of the color related information is specifically described in detail. However, the present invention is not limited to these examples of color related information. The present invention can be implemented in various forms without departing from the spirit or main features shown in the claims.
[0031]
【The invention's effect】
In the present invention, Ri Do each impact is different with affects the color representation of the image signals, and a plurality of types of color-related information to the information acquisition including photographing conditions for generating an image signal in image capturing, the impact In consideration of the difference, the value of each color-related information is converted, and the final saturation modulation characteristic is controlled according to the combination of the saturation modulation characteristics based on each color-related information after the conversion .
Therefore, according to the present invention, it is possible to flexibly control the characteristics of the saturation modulation processing in accordance with the change state of the color expression grasped from the color related information.
By applying the present invention as described above, for example, it is possible to easily realize measures such as estimating the degree of color noise in the low luminance region from the color-related information and appropriately suppressing the saturation gain in the low luminance region. Further, for example, it is possible to easily realize a countermeasure such as estimating the degree of unnatural coloring in the high luminance region from the color related information and appropriately suppressing the saturation gain in the high luminance region. Furthermore, for example, it is possible to easily implement measures such as estimating a saturation change (excessive vividness, unnatural blur, etc.) in a specific luminance range from the color-related information and suppressing the saturation change.
[Brief description of the drawings]
FIG. 1 is a diagram illustrating a configuration of an electronic camera 11 (including an image processing apparatus).
FIG. 2 is a flowchart for explaining the operation of the saturation modulation control unit 24;
FIG. 3 is a diagram illustrating an adjustment tendency of saturation modulation characteristics based on an adjustment value of color reproducibility.
FIG. 4 is a diagram illustrating an adjustment tendency of saturation modulation characteristics based on a setting value of imaging sensitivity.
FIG. 5 is a diagram illustrating an example of soft gradation conversion characteristics;
FIG. 6 is a diagram illustrating an example of a gradation conversion characteristic of high contrast.
FIG. 7 is a diagram illustrating an adjustment tendency of saturation modulation characteristics based on setting of gradation conversion characteristics.
[Explanation of symbols]
11 Electronic camera 12 Shooting lens 12a MPU in lens
13 Image sensor 14 Amplifier 16 A / D conversion unit 17 White balance processing unit 17a Color distribution evaluation unit 17b Light source estimation unit 18 Interpolation processing unit 19 Noise suppression unit 20 Gamma conversion unit 21 Matrix conversion unit 22 Contour adjustment unit 23 Color noise suppression unit 24 Saturation modulation control unit 25 Luminance distribution evaluation unit 29 Split photometry sensor 30 Flash device 31 Shooting control unit

Claims (5)

An image processing apparatus that processes an image signal generated by an imaging unit,
The image signal Ri Do different each degree of influence with affects the color representation of, and a plurality of types of color-related information to the information acquisition including photographing conditions for generating the image signal in image capturing, the difference in the impact In consideration of the combination of the saturation modulation characteristics based on the color-related information after the conversion, the value of each color-related information is A saturation modulation control unit for controlling the saturation modulation characteristic that defines
A saturation modulation unit that adjusts the gain of the saturation level according to the luminance level of the image signal in accordance with the saturation modulation characteristic controlled by the saturation modulation control unit. Processing equipment. The image processing apparatus according to claim 1.
At least one of the color related information is imaging sensitivity of the imaging unit,
The image processing apparatus, wherein the saturation modulation control unit acquires information on the imaging sensitivity when the image signal is captured, and controls the saturation modulation characteristic according to the imaging sensitivity. The image processing apparatus according to claim 1.
At least one of the color related information is a gradation conversion characteristic applied to the image signal,
The image processing apparatus, wherein the saturation modulation control unit acquires information on the gradation conversion characteristic and controls the saturation modulation characteristic according to the gradation conversion characteristic.    An image processing program causing a computer to function as the saturation modulation control unit and the saturation modulation unit according to any one of claims 1 to 3. An image processing method for performing image processing on an image signal generated by an imaging unit,
The image signal Ri Do different each degree of influence with affects the color representation of, and a plurality of types of color-related information to the information acquisition including photographing conditions for generating the image signal in image capturing, the difference in the impact The color-related information values are converted in consideration of the above, and according to the combination of the saturation modulation characteristics based on the color-related information after the conversion , the gain of the saturation level is determined with respect to the luminance level. The step of controlling `` saturation modulation characteristics '',
Adjusting the gain of the saturation level according to the luminance level of the image signal according to the controlled saturation modulation characteristic.

Images