JP4868040B2 - Image processing apparatus, electronic camera, and image processing program - Google Patents

Description

  The present invention relates to an image processing device, an electronic camera, and an image processing program.

  2. Description of the Related Art Conventionally, a phenomenon is known in which a dark portion gradation of image data is darkened by photographing a subject having a large luminance difference. Therefore, in the invention of Patent Document 1, the tone is compressed by increasing the gain of the dark portion gradation, and the blackening of the dark portion gradation is improved.

Japanese Patent No. 2663189

  By the way, a general gradation conversion process is performed before the above-described compression of the dark area gradation. In this gradation conversion processing, a gradation curve having a so-called S-shaped input / output characteristic in addition to the input / output characteristic based on the output destination gamma is used. The S-shaped input / output characteristic has the effect of improving the appearance of the image by tightening the black expression in the dark portion and improving the contrast.

  However, due to such S-shaped input / output characteristics, the RGB ratio changes between the dark portion and the halftone. Therefore, the hue, saturation, and contrast in the dark part may be different from the hue, saturation, and contrast in the halftone. Normally, this problem is not so noticeable because the image is dark, but when the above-described compression of the dark area gradation is performed, the dark area becomes normal brightness, so the hue, saturation, and contrast change. The problem that becomes.

  The present invention has been made in view of the above problem, and an object thereof is to suppress occurrence of hue rotation, saturation change, and contrast change during gradation compression.

An electronic camera according to the present invention includes an imaging unit that captures an image of a subject to generate image data, an acquisition unit that acquires the image data from the imaging unit, and correction that improves the brightness of the dark portion gradation of the image data. And a gradation curve composed only of a power component represented by y = x ⁿ (where x is an input and y is an output) with respect to the image data before performing the correction. A first image processing unit that performs image processing including gradation conversion processing according to the input / output characteristics defined by the image data, and R image data, G image data, and B image data based on the image data acquired by the acquisition unit for each, based on the calculation result by the arithmetic unit, and a correcting unit performing the correction to improve the lightness of the dark area gradation of the image data subjected to image processing by the first image processing section, after the correction Wherein the image data correction is performed by the correction unit, the gradation conversion processing according to the input and output characteristics of the S-shaped as defined by a different tone curve and the gradation curve in the first image processing unit A second image processing unit.

An image processing apparatus according to the present invention includes an acquisition unit that acquires image data, a calculation unit that performs a calculation for improving the brightness of a dark portion gradation of the image data, and the image data before performing the correction. On the other hand, gradation conversion processing according to input / output characteristics defined by a gradation curve consisting of only a power component represented by y = x ⁿ (where x is an input and y is an output) at least halftones For each of R image data, G image data, and B image data based on the image data acquired by the acquisition unit and a first image processing unit that performs image processing including: after a correction unit that performs correction of improving lightness of the dark area gradation of the image data subjected to image processing, that the correction was carried out by the first image processing section, the image data by the correction unit corrects been performed Against it, and a second image processing unit for performing the gradation conversion processing according to the input and output characteristics of the S-shaped as defined by a different tone curve from the tone curve in the first image processing unit.

  In addition, a configuration obtained by converting the configuration related to the above invention into an image processing program for realizing image processing on image data to be processed by a computer is also effective as a specific aspect of the present invention.

  According to the present invention, it is possible to suppress occurrence of hue rotation, saturation change, and contrast change during gradation compression.

It is a figure which shows the structure of the electronic camera 1 of 1st Embodiment. It is a functional block diagram of electronic camera 1 of a 1st embodiment. It is a flowchart which shows the operation | movement at the time of imaging | photography of the electronic camera 1 of 1st Embodiment. FIG. 6 is a diagram for explaining a gradation curve in the first embodiment. It is a figure explaining a low pass filter. It is a figure explaining the parameter fg of gradation compression. FIG. 6 is another diagram for explaining a gradation curve in the first embodiment. It is a flowchart which shows the operation | movement at the time of imaging | photography of the electronic camera 1 of 2nd Embodiment. It is a figure explaining the gradation curve in 3rd Embodiment. It is another figure explaining the gradation curve in 3rd Embodiment.

<First Embodiment>
Hereinafter, a first embodiment of the present invention will be described with reference to the drawings. In the following first embodiment, a single lens reflex type electronic camera will be described as an example of the electronic camera of the present invention.

  FIG. 1 is a diagram illustrating a configuration of an electronic camera 1 according to the first embodiment. As shown in FIG. 1, an electronic camera 1 includes a photographing lens 2, an aperture 3, a quick return mirror 4, a sub mirror 5, a diffusion screen 6, a condenser lens 7, a pentaprism 8, a beam splitter 9, an eyepiece lens 10, and an imaging lens. 11, a photometric sensor 12, a shutter 13, an image sensor 14, and a focus detection unit 15.

  The imaging element 14 is a semiconductor device such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). For example, the focus detection unit 15 detects a focus state of the photographic lens 2 by performing phase difference type focus detection.

  The electronic camera 1 further includes a monitor 16 such as a liquid crystal monitor that displays an image generated by imaging, and a control unit 17 that controls each unit. The control unit 17 includes a memory (not shown) therein, and records a program for controlling each unit in advance.

  When not photographing, that is, when photographing is not performed, the quick return mirror 4 is disposed at an angle of 45 ° as shown in FIG. The light beam that has passed through the photographing lens 2 and the diaphragm 3 is reflected by the quick return mirror 4 and guided to the eyepiece lens 10 through the diffusion screen 6, the condenser lens 7, the pentaprism 8, and the beam splitter 9. The user confirms the composition by viewing the subject image through the eyepiece 10. On the other hand, the light beam split upward by the beam splitter 9 is re-imaged on the imaging surface of the photometric sensor 12 via the imaging lens 11. Further, the light beam transmitted through the quick return mirror 4 is guided to the focus detection unit 15 via the sub mirror 5.

  On the other hand, at the time of photographing, the quick return mirror 4 is retracted to the position indicated by the broken line, the shutter 13 is opened, and the light flux from the photographing lens 2 is guided to the image sensor 14.

  FIG. 2 is a functional block diagram of the electronic camera 1 according to the first embodiment. 2, the electronic camera 1 includes a timing generator 20, a signal processing unit 21, an A / D conversion unit 22, a buffer memory 23, a bus 24, a card interface 25, and a compression / decompression unit in addition to the configuration of FIG. 26, each part of the image display unit 27 is provided. The timing generator 20 supplies output pulses to the image sensor 14. The image data generated by the image sensor 14 is temporarily stored in the buffer memory 23 via the signal processing unit 21 (including a gain adjustment unit corresponding to the imaging sensitivity) and the A / D conversion unit 22. The buffer memory 23 is connected to the bus 24. The bus 24 is connected to the card interface 25, the control unit 17, the compression / decompression unit 26, and the image display unit 27 described in FIG. The card interface 25 is connected to a removable memory card 28 and records image data on the memory card 28. The control unit 17 is connected to a switch group 29 (including a release button (not shown)) of the electronic camera 1, a timing generator 20, and a photometric sensor 12. Further, the image display unit 27 displays an image or the like on the monitor 16 provided on the back surface of the electronic camera 1.

  An operation at the time of shooting in the electronic camera 1 having the above-described configuration will be described with reference to a flowchart shown in FIG.

  In step S <b> 1, the control unit 17 determines whether or not the user has instructed to start shooting via the switch group 29. If the control unit 17 determines that an instruction to start photographing is given, the process proceeds to step S2.

  In step S <b> 2, the control unit 17 controls each unit, captures a subject image with the image sensor 14, and generates image data. Image data generated by the imaging device 14 is temporarily stored in the buffer memory 23 via the signal processing unit 21 and the A / D conversion unit 22.

  In step S3, the control unit 17 reads the image data from the buffer memory 23 and performs normal image processing. Normal image processing includes white balance adjustment, interpolation processing, color tone correction processing, and the like. Since the specific method of each process is the same as that of a well-known technique, description is abbreviate | omitted.

  In step S4, the control unit 17 performs color space conversion processing on the image data subjected to the image processing in step S3. The color space conversion process is performed by the following equations 1 to 3.

Rm [x, y] = mx11 · Ri [x, y] + mx12 · Gi [x, y] + mx13 · Bi [x, y] (Formula 1)
Gm [x, y] = mx21 · Ri [x, y] + mx22 · Gi [x, y] + mx23 · Bi [x, y] (Formula 2)
Bm [x, y] = mx31 · Ri [x, y] + mx32 · Gi [x, y] + mx33 · Bi [x, y] (Formula 3)
Note that Ri [x, y], Gi [x, y], and Bi [x, y] in Equations 1 to 3 respectively represent RGB image data, and mx11 to mx33 are respectively predetermined coefficients. is there.

  In step S5, the control unit 17 performs a first gradation conversion process on the image data subjected to the color space conversion process in step S4. The first gradation conversion process is performed by the following equations 4 to 6.

Rg [x, y] = Gm1 [Rm [x, y]] (Formula 4)
Gg [x, y] = Gm1 [Gm [x, y]] (Formula 5)
Bg [x, y] = Gm1 [Bm [x, y]] (Formula 6)
Note that Gm1 in Expressions 4 to 6 corresponds to, for example, the gradation curve shown in FIG. As shown in FIG. 4, the gradation curve Gm1 is a gradation curve having a power of approximately 1 / 2.2. The gradation curve Gh in FIG. 4 is a gradation curve generally used in conventional gradation conversion processing (in addition to the input / output characteristics based on the output destination gamma, an S-shaped input / output characteristic). This is an example of a gradation curve having characteristics.

  In step S6, the control unit 17 performs a gradation compression process on the image data that has been subjected to the first gradation conversion process in step S5.

  First, the control unit 17 performs a low-pass operation on the image data that has been subjected to the first gradation conversion process in step S5. The low-pass calculation is performed by the following equations 7 to 8.

  Y [x, y] = kr · Rg [x, y] + kg · Gg [x, y] + kb · Bg [x, y] (Equation 7)

  In Equation 7, kr, kg, and kb are predetermined coefficients. From the sRGB image, the Y image among the YCbCr images is obtained by Expression 7. Further, Lpw in Expression 8 is a low-pass filter around the target pixel, and this low-pass filter has the characteristics shown in FIG. Then, according to Expression 8, image data of an LY image that is a low-pass image is generated from the Y image. The low-pass image is an example of a blurred image, and the low-pass image may be generated using another low-pass filter, or the blurred image may be generated using a method other than the low-pass processing.

  Next, the control unit 17 performs gradation compression processing. The gradation compression processing is performed by the following formulas 9 to 11.

Rc [x, y] = Rg [x, y] · fg (LY [x, y]) (Equation 9)
Gc [x, y] = Gg [x, y] · fg (LY [x, y]) (Equation 10)
Bc [x, y] = Bg [x, y] · fg (LY [x, y]) (Equation 11)
Note that fg in Equation 9 to Equation 11 is a gradation compression parameter. FIG. 6 is a diagram showing the gradation compression parameter fg. The parameter fg has a gain corresponding to the image data of the LY image as shown in FIG. The parameter fg increases as the image data of the LY image is smaller (the darker the neighborhood including the processing pixel is). On the contrary, the parameter fg approaches 1 as the image data of the LY image is larger (the neighborhood range including the processing pixel is brighter).

  In step S7, the control unit 17 performs a second gradation conversion process on the image data subjected to the gradation compression process in step S6. The second gradation conversion process is performed by the following equations 12 to 14.

Ro [x, y] = Gm2 [Rc [x, y]] (12)
Go [x, y] = Gm2 [Gc [x, y]] (Equation 13)
Bo [x, y] = Gm2 [Bc [x, y]] (Formula 14)
Note that Gm2 in Expressions 12 to 14 corresponds to, for example, the gradation curve shown in FIG. The gradation curve Gm2 is an S-shaped gradation curve as shown in FIG. By performing the gradation conversion process using such a gradation curve, the final appearance of the image can be improved. The gradation curve Gh in FIG. 7 is a gradation curve generally used in the conventional gradation conversion processing (in addition to the input / output characteristics based on the output destination gamma, an S-shaped input / output characteristic). This is an example of a gradation curve having characteristics.

  In step S8, the control unit 17 records the image data subjected to the second gradation conversion process in step S7 on the memory card 28 via the card interface 25, and ends the series of processes. Note that before the image data is recorded on the memory card 28, an image compression process (such as a JPEG compression process) may be performed as necessary via the compression / decompression unit 26.

As described above, according to the first embodiment, the image data is defined by a gradation curve composed of only a power component represented by y = x ⁿ (where x is an input and y is an output). The image processing including gradation conversion processing according to the input / output characteristics is performed, and then the correction is performed to improve the lightness of the dark portion gradation of the image data subjected to the first image processing, and finally the correction is performed. The second tone conversion process is performed on the broken image data according to the input / output characteristics defined by the tone curve different from the tone curve in the first tone conversion process. Conventionally, when correction for improving the brightness of the dark portion gradation is performed after the gradation conversion processing using a so-called S-shaped gradation curve or the like, locally due to the S-shaped gradation curve, Although there has been a problem that hue rotation, saturation change, and contrast change occur, according to the first embodiment, it is possible to suppress such hue rotation, saturation change, and contrast change during gradation compression. Can do.

<Second Embodiment>
The second embodiment of the present invention will be described below with reference to the drawings. The second embodiment is a modification of the first embodiment. Therefore, below, only a different part from 1st Embodiment is demonstrated. The same parts as those in the first embodiment will be described using the same reference numerals as those in the first embodiment.

  The second embodiment is an example in which an RGB image is converted into a YCbCr image before gradation compression processing, and gradation compression processing is performed on the YCbCr image.

  The operation at the time of photographing when the gradation compression process is performed on the YCbCr image will be described with reference to the flowchart shown in FIG.

  In step S11 to step S15, the control unit 17 performs the same processing as in step S1 to step S5 of FIG. 3 of the first embodiment.

  In step S <b> 16, the control unit 17 performs color space conversion processing of the image data that has been subjected to the first gradation conversion processing in step S <b> 15 to YCbCr. The color space conversion process to YCbCr is performed by the following equations 15 to 17.

Y [x, y] = my11 · Rg [x, y] + my12 · Gg [x, y] + my13 · Bg [x, y] (Formula 15)
Cb [x, y] = my21 · Rg [x, y] + my22 · Gg [x, y] + my23 · Bg [x, y] (Formula 16)
Cr [x, y] = my31 · Rg [x, y] + my32 · Gg [x, y] + my33 · Bg [x, y] (Expression 17)
Note that my11 to my33 in Equations 15 to 17 are predetermined coefficients, respectively. The sRGB image is converted into a YCbCr image by Expression 15 to Expression 17.

  In step S17, the control unit 17 performs a gradation compression process on the image data that has been subjected to the color space conversion process in step S16.

  First, the control unit 17 performs a low-pass operation on the image data that has undergone the color space conversion process in step S16. The low-pass calculation is performed by the following equation (18).

  Note that, according to Expression 18, image data of a LY image that is a low-pass image is generated from a Y image among YCbCr images. Lpw in Expression 18 is a low-pass filter similar to that described in step S6 of FIG. 3 of the first embodiment. The low-pass image is an example of a blurred image, and the low-pass image may be generated using another low-pass filter, or the blurred image may be generated using a method other than the low-pass processing.

  Next, the control unit 17 performs gradation compression processing. The gradation compression processing is performed by the following equations 19 to 21.

Yc [x, y] = Y [x, y] · fg (LY [x, y]) (Equation 19)
Cbo [x, y] = Cb [x, y] · fg (LY [x, y]) (Equation 20)
Cro [x, y] = Cr [x, y] · fg (LY [x, y]) (Formula 21)
Note that fg in Equations 19 to 21 is a tone compression parameter similar to that described in Step S6 of FIG. 3 of the first embodiment.

  In step S18, the control unit 17 performs the second gradation conversion process on the image data of the Y image indicating the luminance among the image data subjected to the gradation compression process in step S17. The second gradation conversion process is performed by the following Expression 22.

Yo [x, y] = Gm2 [Yc [x, y]] (Equation 22)
Note that Gm2 in Expression 22 corresponds to a gradation curve similar to that described in step S7 of FIG. 3 of the first embodiment.

  In step S19, the control unit 17 records the image data subjected to the second gradation conversion process in step S18 on the memory card 28 via the card interface 25, and ends the series of processes. Note that before the image data is recorded on the memory card 28, an image compression process (such as a JPEG compression process) may be performed as necessary via the compression / decompression unit 26.

  As described above, according to the second embodiment, the second gradation conversion process is performed only on the Y image data that is the luminance image data. Therefore, the same effect as that of the first embodiment can be obtained while reducing the calculation load as compared with the first embodiment.

In the first embodiment and the second embodiment, an example is shown in which the gradation conversion processing is performed using the gradation curve Gm1 shown in FIG. 4 in the gradation conversion processing (steps S5 and FIG. 7 in FIG. 3). S15). However, the present invention is not limited to this example. If the tone curve is composed only of a power component expressed by y = x ⁿ (where x is an input and y is an output), the tone curve other than the tone curve Gm1 shown in FIG. The same effect as the invention can be obtained. Although FIG. 4 illustrates the gradation curve Gm1 that is a gradation curve of approximately 1 / 2.2 based on the monitor gamma in the sRGB color space, for example, a gradation curve of 1/2 power may be used. .

<Third Embodiment>
The third embodiment of the present invention will be described below with reference to the drawings. The third embodiment is a modification of the first embodiment. Therefore, below, only a different part from 1st Embodiment is demonstrated. The same parts as those in the first embodiment will be described using the same reference numerals as those in the first embodiment.

  In the third embodiment, instead of the gradation curve Gm1 described in the first embodiment, the gradation curve Gm3 shown in FIG. 9 is used, and instead of the gradation curve Gm2 described in the first embodiment, FIG. The gradation curve Gm4 shown is used.

  That is, in the first gradation conversion process described in step S5 of FIG. 3, the control unit 17 performs the gradation conversion process using the gradation curve Gm3 shown in FIG. As shown in FIG. 9, the gradation curve Gm3 is only about 1 / 2.2 from the dark part to the halftone, and the bright part (highlight side) has the same level as the gradation curve Gm1. It is a gradation curve having a characteristic (knee characteristic) that realizes a low output level with respect to an input. This is because the brightness improvement of the dark part gradation resulting from the first gradation conversion process is performed only in the halftone from the dark part. As a result, in the bright portion, the change can be reduced by suppressing the occurrence of sudden whiteout due to the first gradation conversion processing. Note that the gradation curve Gm3 shown in FIG. 9 has a characteristic of approximately 1 / 2.2 to about 5% to 20% of the dark portion, for example.

In the second gradation conversion process described in step S7 of FIG. 3, the control unit 17 performs the gradation conversion process using the gradation curve Gm4 shown in FIG. As shown in FIG. 10, the gradation curve Gm4 has a characteristic that the knee characteristic in the bright portion (highlight side) is weaker than the gradation curve Gm2.
As described above, according to the third embodiment, for image data, at least a halftone or less is a floor composed of only a power component represented by y = x ⁿ (where x is an input and y is an output). Apply image processing including gradation conversion processing according to the input / output characteristics defined by the tone curve, then correct to improve the lightness of the dark portion gradation of the image data subjected to the first image processing, and finally In addition, a second gradation conversion process is performed on the corrected image data in accordance with input / output characteristics defined by a gradation curve different from the gradation curve in the first gradation conversion process. . Therefore, the same effect as that of the first embodiment can be obtained.

In the third embodiment, an example in which the gradation conversion process is performed using the gradation curve Gm3 illustrated in FIG. 9 in the gradation conversion process has been described. However, the present invention is not limited to this example. A gradation curve other than the gradation curve Gm3 shown in FIG. 9 is used as long as it is a gradation curve consisting only of a power component expressed by y = x ⁿ (where x is an input and y is an output) at least halftones. Even if is used, the same effect as the present invention can be obtained. For example, the gradation curve may be a half power of at least halftone.

  Further, in each of the above-described embodiments, the example in which the technique of the present invention is realized in the electronic camera 1 has been described. However, the present invention is not limited to this. For example, the present invention can be similarly applied to a compact type electronic camera, a movie camera for taking a moving image, and the like.

  Further, the image processing apparatus described in each of the above embodiments may be realized by software using a computer and an image processing program. In this case, what is necessary is just to make it the structure which implement | achieves part or all of the process after step S3 demonstrated with the flowchart of FIG. 3 with a computer. Or what is necessary is just to set it as the structure which implement | achieves a part or all of the process after step S13 demonstrated with the flowchart of FIG. 8 with a computer. In order to be realized by a computer, information such as whether or not the image is in the gradation compression mode may be supplied to the computer together with the image data. Such information can be supplied using the EXIF information of the image data. The same applies to the processing described in the third embodiment. By adopting such a configuration, it is possible to perform the same processing as in the present embodiment.

  The present invention can be implemented in various other forms without departing from the spirit or main features thereof. For this reason, the above-described embodiments are merely examples in all respects and should not be interpreted in a limited manner. The scope of the present invention is indicated by the scope of claims, and is not restricted by the text of the specification. Further, all modifications and changes belonging to the equivalent scope of the claims are within the scope of the present invention.

DESCRIPTION OF SYMBOLS 1 ... Electronic camera, 2 ... Shooting lens, 14 ... Image sensor, 17 ... Control part

Claims (3)

An imaging unit that captures a subject image and generates image data;
An acquisition unit for acquiring the image data from the imaging unit ;
A calculation unit that performs calculation for correction to improve the brightness of the dark portion gradation of the image data;
Before performing the correction, the image data is subjected to input / output characteristics defined by a gradation curve composed of only a power component represented by y = x ⁿ (where x is an input and y is an output). A first image processing unit that performs image processing including gradation conversion processing;
An image obtained by performing image processing on the R image data, G image data, and B image data based on the image data acquired by the acquisition unit based on a calculation result by the calculation unit by the first image processing unit. A correction unit for performing correction to improve the brightness of the dark portion gradation of the data;
An S-shaped input / output characteristic defined by a gradation curve different from the gradation curve in the first image processing unit for the image data corrected by the correction unit after performing the correction. An electronic camera comprising: a second image processing unit that performs gradation conversion processing according to the above. An acquisition unit for acquiring image data;
A calculation unit that performs calculation for correction to improve the brightness of the dark portion gradation of the image data;
Before performing the correction, the image data is defined by a gradation curve consisting of only a power component represented by y = x ⁿ (where x is an input and y is an output) at least halftones. A first image processing unit that performs image processing including gradation conversion processing according to input / output characteristics;
An image obtained by performing image processing on the R image data, G image data, and B image data based on the image data acquired by the acquisition unit based on a calculation result by the calculation unit by the first image processing unit. A correction unit for performing correction to improve the brightness of the dark portion gradation of the data;
An S-shaped input / output characteristic defined by a gradation curve different from the gradation curve in the first image processing unit for the image data corrected by the correction unit after performing the correction. An image processing apparatus comprising: a second image processing unit that performs gradation conversion processing according to the above. An image processing program for realizing image processing on image data to be processed by a computer,
An acquisition step of acquiring the image data;
A calculation step for performing calculation for correction to improve the brightness of the dark portion gradation of the image data;
Before performing the correction, the image data is subjected to input / output characteristics defined by a gradation curve composed of only a power component represented by y = x ⁿ (where x is an input and y is an output). A first image processing step for performing image processing including gradation conversion processing;
For each of R image data, G image data, and B image data based on the image data acquired in the acquisition step, an image that has been subjected to image processing in the first image processing step based on a calculation result in the calculation step A correction step for performing correction to improve the brightness of the dark portion gradation of the data;
An S-shaped input / output characteristic defined by a gradation curve different from the gradation curve in the first image processing step for the image data corrected in the correction step after the correction is performed. And a second image processing step for performing gradation conversion processing according to the above.

Images