JP2012105354A - Image processing apparatus, electronic camera, and image processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing apparatus, an electronic camera, and an image processing program that perform preferred image processing even in a case where, in outputting image data, preferable gray scale characteristics matching viewing conditions are realized.SOLUTION: The image processing apparatus comprises: an acquiring unit that acquires image data; an image processing unit that performs, on the image data, image processing including gray-scale conversion processing corresponding to input/output characteristics defined by a gray scale curve consisting of only a multiplication component represented by y=k×x(where x represents an input; y represents an output; and k and n represent coefficients); a correcting unit that performs correction for increasing a luminance of the gray scale of a dark part of the image data obtained by the image processing including the gray-scale conversion processing performed by the image processing unit; and a generating unit that, on the basis of the image data obtained by the image processing including the gray-scale conversion processing performed by the image processing unit, generates a gain map corresponding a luminance level.

Description

  The present invention relates to an image processing device, an electronic camera, and an image processing program.

  2. Description of the Related Art Conventionally, a phenomenon is known in which a dark portion gradation of image data is darkened by photographing a subject having a large luminance difference. Therefore, in the invention of Patent Document 1, the tone is compressed by increasing the gain of the dark portion gradation, and the blackening of the dark portion gradation is improved.

Japanese Patent No. 2663189

  By the way, there is a demand for realizing gradation characteristics corresponding to the viewing conditions (such as ambient brightness) of the output destination when outputting image data (monitor, printer, publication, etc.). In the invention of Patent Document 1 described above, since the gradation compression described above is performed after the gradation conversion process assuming a specific viewing condition, a preferable gradation characteristic is realized depending on the output destination environment. There are cases where it is not possible.

  The present invention has been made in view of the above problems, and performs suitable image processing so that it can cope with the case of realizing preferable gradation characteristics according to viewing conditions when outputting image data. For the purpose.

The image processing apparatus of the present invention is represented by an acquisition unit that acquires image data, and y = k · x ⁿ (where x is an input, y is an output, and k and n are coefficients) with respect to the image data. An image processing unit that performs image processing including gradation conversion processing according to input / output characteristics defined by a gradation curve that includes only a power component, and image processing including gradation conversion processing is performed by the image processing unit. And a gain map corresponding to the luminance level based on the image data that has been subjected to image processing including gradation conversion processing by the image processing unit. A generating unit for generating.

  Note that a recording unit that records the gain map generated by the generating unit and the image data corrected by the correcting unit in association with each other may be provided.

  The image processing unit further includes a low-pass processing unit that applies a low-pass filter to the image data that has undergone image processing including gradation conversion processing, and the correction unit uses the image data that has been subjected to the low-pass filter. Then, the image processing unit may perform correction for improving the brightness of the dark part gradation of the image data subjected to the image processing including the gradation conversion process.

  The image processing apparatus further includes a setting unit configured to generate a scene-referred image from the image data acquired by the acquiring unit, and the image processing is performed when the setting unit is configured to generate the scene-referred image. The unit may perform the image processing on the image data, the correction unit may perform the correction, and the generation unit may generate the gain map.

  In addition, the image capturing unit that captures a subject image and generates image data, and any one of the above-described image processing apparatuses, and the acquisition unit includes an electronic camera that acquires the image data from the image capturing unit. It is effective as a specific embodiment.

  In addition, a configuration obtained by converting the configuration related to the above invention into an image processing program for realizing image processing on image data to be processed by a computer is also effective as a specific aspect of the present invention.

  According to the present invention, it is possible to perform suitable image processing so as to be able to cope with the case where a preferable gradation characteristic corresponding to viewing conditions is realized when image data is output.

It is a figure which shows the structure of the electronic camera 1 of 1st Embodiment. It is a functional block diagram of electronic camera 1 of a 1st embodiment. It is a flowchart which shows the operation | movement at the time of imaging | photography of the electronic camera 1 of 1st Embodiment. It is a figure explaining a gradation curve. It is a figure explaining a low pass filter. It is a figure explaining the parameter fg of gradation compression. It is a flowchart which shows the operation | movement at the time of imaging | photography of the electronic camera 1 of 2nd Embodiment.

<First Embodiment>
Hereinafter, a first embodiment of the present invention will be described with reference to the drawings. In the following first embodiment, a single lens reflex type electronic camera will be described as an example of the electronic camera of the present invention.

  FIG. 1 is a diagram illustrating a configuration of an electronic camera 1 according to the first embodiment. As shown in FIG. 1, an electronic camera 1 includes a photographing lens 2, an aperture 3, a quick return mirror 4, a sub mirror 5, a diffusion screen 6, a condenser lens 7, a pentaprism 8, a beam splitter 9, an eyepiece lens 10, and an imaging lens. 11, a photometric sensor 12, a shutter 13, an image sensor 14, and a focus detection unit 15.

  The imaging element 14 is a semiconductor device such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). For example, the focus detection unit 15 detects a focus state of the photographic lens 2 by performing phase difference type focus detection.

  The electronic camera 1 further includes a monitor 16 such as a liquid crystal monitor that displays an image generated by imaging, and a control unit 17 that controls each unit. The control unit 17 includes a memory (not shown) therein, and records a program for controlling each unit in advance.

  When not photographing, that is, when photographing is not performed, the quick return mirror 4 is disposed at an angle of 45 ° as shown in FIG. The light beam that has passed through the photographing lens 2 and the diaphragm 3 is reflected by the quick return mirror 4 and guided to the eyepiece lens 10 through the diffusion screen 6, the condenser lens 7, the pentaprism 8, and the beam splitter 9. The user confirms the composition by viewing the subject image through the eyepiece 10. On the other hand, the light beam split upward by the beam splitter 9 is re-imaged on the imaging surface of the photometric sensor 12 via the imaging lens 11. Further, the light beam transmitted through the quick return mirror 4 is guided to the focus detection unit 15 via the sub mirror 5.

  On the other hand, at the time of photographing, the quick return mirror 4 is retracted to the position indicated by the broken line, the shutter 13 is opened, and the light flux from the photographing lens 2 is guided to the image sensor 14.

  FIG. 2 is a functional block diagram of the electronic camera 1 according to the first embodiment. 2, the electronic camera 1 includes a timing generator 20, a signal processing unit 21, an A / D conversion unit 22, a buffer memory 23, a bus 24, a card interface 25, and a compression / decompression unit in addition to the configuration of FIG. 26, each part of the image display unit 27 is provided. The timing generator 20 supplies output pulses to the image sensor 14. The image data generated by the image sensor 14 is temporarily stored in the buffer memory 23 via the signal processing unit 21 (including a gain adjustment unit corresponding to the imaging sensitivity) and the A / D conversion unit 22. The buffer memory 23 is connected to the bus 24. The bus 24 is connected to the card interface 25, the control unit 17, the compression / decompression unit 26, and the image display unit 27 described in FIG. The card interface 25 is connected to a removable memory card 28 and records image data on the memory card 28. The control unit 17 is connected to a switch group 29 (including a release button (not shown)) of the electronic camera 1, a timing generator 20, and a photometric sensor 12. Further, the image display unit 27 displays an image or the like on the monitor 16 provided on the back surface of the electronic camera 1.

  Further, the electronic camera 1 has a plurality of predetermined shooting modes. The plurality of shooting modes include, for example, a manual mode, a full auto mode, a portrait shooting mode (portrait mode, etc.), and other modes (landscape mode, close-up mode, night view mode, etc.). These shooting modes are selected in advance by the user via the switch group 29.

  The electronic camera 1 also includes a scene referred mode for generating a scene-referred image. Scene Referred images are defined by the International Color Consortium (abbreviation: ICC) etc. in the technology of Color Management System (abbreviation: CMS). It is a kind of image of “color space independent of”. The “device-independent color space” is a “scene-referred” color space that supports the input system and captures the scene as it is, and an output system that supports the output system and depends on the preferred reproduction in the media and output. And “Output-referred” color space. In the scene-referred mode described above, the electronic camera 1 generates such a scene-referred image.

  The scene referred mode may be set by a user operation via the switch group 29 or may be automatically set by the control unit 17. For example, the control unit 17 may set the scene referral mode in conjunction with the above-described shooting mode, or may set the scene referral mode according to the subject recognition result.

  In the electronic camera 1 having the above-described configuration, an operation at the time of shooting in a state where the scene referral mode is set will be described with reference to a flowchart shown in FIG.

  In step S <b> 1, the control unit 17 determines whether or not the user has instructed to start shooting via the switch group 29. If the control unit 17 determines that an instruction to start photographing is given, the process proceeds to step S2.

  In step S <b> 2, the control unit 17 controls each unit, captures a subject image with the image sensor 14, and generates image data. Image data generated by the imaging device 14 is temporarily stored in the buffer memory 23 via the signal processing unit 21 and the A / D conversion unit 22.

  In step S3, the control unit 17 reads the image data from the buffer memory 23 and performs normal image processing. Normal image processing includes white balance adjustment, interpolation processing, color tone correction processing, and the like. Since the specific method of each process is the same as that of a well-known technique, description is abbreviate | omitted.

  In step S4, the control unit 17 performs color space conversion processing on the image data subjected to the image processing in step S3. The color space conversion process is performed by the following equations 1 to 3.

Rm [x, y] = mx11 · Ri [x, y] + mx12 · Gi [x, y] + mx13 · Bi [x, y] (Formula 1)
Gm [x, y] = mx21 · Ri [x, y] + mx22 · Gi [x, y] + mx23 · Bi [x, y] (Formula 2)
Bm [x, y] = mx31 · Ri [x, y] + mx32 · Gi [x, y] + mx33 · Bi [x, y] (Formula 3)
Note that Ri [x, y], Gi [x, y], and Bi [x, y] in Equations 1 to 3 indicate image data of RGB images, respectively. Further, mx11 to mx33 are predetermined coefficients for performing conversion into the sRGB color space, for example.

  In step S5, the control unit 17 performs a gradation conversion process on the image data subjected to the color space conversion process in step S4. The gradation conversion processing is performed by the following equations 4 to 6.

Rg [x, y] = Gm1 [Rm [x, y]] (Formula 4)
Gg [x, y] = Gm1 [Gm [x, y]] (Formula 5)
Bg [x, y] = Gm1 [Bm [x, y]] (Formula 6)
Note that Gm1 in Expressions 4 to 6 corresponds to, for example, the gradation curve shown in FIG. As shown in FIG. 4, the gradation curve Gm1 is a gradation curve having a power of approximately 1 / 2.2.

  In step S6, the control unit 17 performs a gradation compression process on the image data that has been subjected to the gradation conversion process in step S5.

  First, the control unit 17 performs a low-pass operation on the image data that has been subjected to the gradation conversion process in step S5. The low-pass calculation is performed by the following equations 7 to 8.

  Y [x, y] = kr · Rg [x, y] + kg · Gg [x, y] + kb · Bg [x, y] (Equation 7)

  In Equation 7, kr, kg, and kb are predetermined coefficients. From the sRGB image, the Y image among the YCbCr images is obtained by Expression 7. Further, Lpw in Expression 8 is a low-pass filter around the target pixel, and this low-pass filter has the characteristics shown in FIG. Then, according to Expression 8, image data of an LY image that is a low-pass image is generated from the Y image. The low-pass image is an example of a blurred image, and the low-pass image may be generated using another low-pass filter, or the blurred image may be generated using a method other than the low-pass processing.

  Next, the control unit 17 performs gradation compression processing. The gradation compression process is performed by the following formulas 9 to 12.

gmp [x, y] = fg (LY [x, y]) (Equation 9)
Rc [x, y] = Rg [x, y] · gmp [x, y] (Equation 10)
Gc [x, y] = Gg [x, y] · gmp [x, y] (Equation 11)
Bc [x, y] = Bg [x, y] · gmp [x, y] (Equation 12)
Note that fg in Equation 9 is a gradation compression parameter. FIG. 6 is a diagram showing the gradation compression parameter fg. The parameter fg has a gain corresponding to the image data of the LY image as shown in FIG. The parameter fg increases as the image data of the LY image is smaller (the darker the neighborhood including the processing pixel is). On the contrary, the parameter fg approaches 1 as the image data of the LY image is larger (the neighborhood range including the processing pixel is brighter).

  In step S7, the control unit 17 records the image data subjected to the gradation compression process in step S6 as a scene-referred image on the memory card 28 via the card interface 25, and ends the series of processes. Note that before the image data is recorded on the memory card 28, an image compression process (such as a JPEG compression process) may be performed as necessary via the compression / decompression unit 26.

  Note that when recording the image data subjected to the gradation compression process, the control unit 17 may record gmp shown in Expression 9 together with the image data as a gain map. For example, if a gain map is recorded as incidental information of image data subjected to gradation compression processing, image data before gradation compression (shown in Expressions 4 to 6) is read based on the gain map when the image data is read. Rg, Gg, Bg) can be created.

  Processing for creating image data before gradation compression from image data that has been subjected to gradation compression processing is performed according to Equations 13 to 15 below.

Rg [x, y] = Rc [x, y] / gmp [x, y] (Equation 13)
Gg [x, y] = Gc [x, y] / gmp [x, y] (Expression 14)
Bg [x, y] = Bc [x, y] / gmp [x, y] (Equation 15)
By the operations of Expressions 13 to 15, it is possible to selectively output the image data after gradation compression and the image data before gradation compression as a scene-referred image when outputting the image data.

As described above, according to the first embodiment, for image data, only a power component represented by y = k · x ⁿ (where x is an input, y is an output, and k and n are coefficients). The image processing including gradation conversion processing according to the input / output characteristics defined by the gradation curve is performed, and correction is performed to improve the brightness of the dark portion gradation of the image data subjected to the image processing. Conventionally, when correction for improving the lightness of the dark portion gradation is performed after the gradation conversion process using a so-called S-shaped gradation curve, the hue locally depends on the S-shaped gradation curve. Although there has been a problem that rotation and saturation change occur, according to the first embodiment, occurrence of such hue rotation and saturation change can be suppressed. Accordingly, it is possible to perform suitable image processing so as to be able to cope with the case where a preferable gradation characteristic corresponding to viewing conditions is realized at the time of outputting image data.

  Further, according to the first embodiment, information indicating a calculation result for correction for improving the brightness of the dark portion gradation and the image data subjected to the above-described correction are recorded in association with each other. Therefore, by using the associated information, it is possible to easily obtain an image before correction for improving the brightness of the dark portion gradation as a scene-referred image.

Second Embodiment
The second embodiment of the present invention will be described below with reference to the drawings. The second embodiment is a modification of the first embodiment. Therefore, below, only a different part from 1st Embodiment is demonstrated. The same parts as those in the first embodiment will be described using the same reference numerals as those in the first embodiment.

  The second embodiment is an example in which a YCbCr image is recorded as a scene-referred image.

  An operation at the time of shooting when a YCbCr image is recorded as a scene-referred image will be described with reference to a flowchart shown in FIG.

  In step S11 to step S15, the control unit 17 performs the same processing as in step S1 to step S5 of FIG. 3 of the first embodiment.

  In step S16, the control unit 17 performs a gradation compression process on the image data that has been subjected to the gradation conversion process in step S15.

  First, the control unit 17 performs a low-pass operation on the image data that has been subjected to the gradation conversion process in step S14. The low-pass calculation is performed by the following equations 16 to 19.

Y [x, y] = my11 · Rg [x, y] + my12 · Gg [x, y] + my13 · Bg [x, y] (Formula 16)
Cb [x, y] = my21 · Rg [x, y] + my22 · Gg [x, y] + my23 · Bg [x, y] (Equation 17)
Cr [x, y] = my31 * Rg [x, y] + my32 * Gg [x, y] + my33 * Bg [x, y] (Equation 18)

  Note that my11 to my33 in Expressions 16 to 18 are predetermined coefficients, respectively. The sRGB image is converted into a YCbCr image by Expression 16 to Expression 18. Further, according to Expression 19, image data of an LY image that is a low-pass image is generated from a YCbCr image. Lpw in Equation 19 is a low-pass filter similar to that described in step S6 of FIG. 3 of the first embodiment. The low-pass image is an example of a blurred image, and the low-pass image may be generated using another low-pass filter, or the blurred image may be generated using a method other than the low-pass processing.

  Next, the control unit 17 performs gradation compression processing. The gradation compression process is performed by the following Expression 20 to Expression 23.

gmp [x, y] = fg (LY [x, y]) (Equation 20)
Yc [x, y] = Y [x, y] · gmp [x, y] (Expression 21)
Cbc [x, y] = Cb [x, y] · gmp [x, y] (Equation 22)
Crc [x, y] = Cr [x, y] · gmp [x, y] (Equation 23)
Note that fg in Expression 20 is a tone compression parameter similar to that described in step S6 of FIG. 3 of the first embodiment.

  In step S18, the control unit 17 records the image data subjected to the gradation compression process in step S17 as a scene-referred image on the memory card 28 via the card interface 25, and ends the series of processes. Note that before the image data is recorded on the memory card 28, an image compression process (such as a JPEG compression process) may be performed as necessary via the compression / decompression unit 26.

  When recording the image data subjected to the gradation compression process, the control unit 17 may record gmp shown in Expression 20 together with the image data as a gain map. For example, if a gain map is recorded as incidental information of image data subjected to gradation compression processing, image data before gradation compression (shown in Expressions 16 to 18) is read based on the gain map when the image data is read. Y, Cb, Cr) can be created.

  Processing for creating image data before gradation compression from image data that has been subjected to gradation compression processing is performed according to the following Expression 24 to Expression 26.

Y [x, y] = Yc [x, y] / gmp [x, y] (Equation 24)
Cb [x, y] = Cbc [x, y] / gmp [x, y] (Equation 25)
Cr [x, y] = Crc [x, y] / gmp [x, y] (Equation 26)
According to the calculations of Expression 24 to Expression 26, image data after gradation compression and image data before gradation compression can be selectively output as a scene-referred image when outputting image data.

  As described above, according to the second embodiment, the luminance image data and the color difference image data are corrected to improve the brightness of the dark portion gradation. Therefore, the same effect as that of the first embodiment can be obtained.

In each of the above-described embodiments, an example in which the gradation conversion process is performed using the gradation curve Gm1 illustrated in FIG. 4 in the gradation conversion process (step S5 in FIG. 3 and step S15 in FIG. 7). However, the present invention is not limited to this example. If the tone curve is composed only of a power component expressed by y = k · x ⁿ (where x is an input, y is an output, k and n are coefficients), the tone curve other than the tone curve Gm1 shown in FIG. Even if the gradation curve is used, the same effect as the present invention can be obtained. Although FIG. 4 illustrates the gradation curve Gm1 that is a gradation curve of approximately 1 / 2.2 based on the monitor gamma in the sRGB color space, for example, a gradation curve of 1/2 power may be used. .

  In each of the embodiments described above, an example in which the gain map is recorded together with the image data when the image data subjected to the gradation compression process is recorded has been described. May be recorded, and gradation compression processing may be performed when the image data is read out.

  In each of the above embodiments, the sRGB color space has been described as an example of the color space of the image data to be recorded, but the present invention is not limited to this example. For example, a color space such as AdobeRGB may be used.

  Further, in each of the above-described embodiments, the example in which the technique of the present invention is realized in the electronic camera 1 has been described. However, the present invention is not limited to this. For example, the present invention can be similarly applied to a compact type electronic camera, a movie camera for taking a moving image, and the like.

  Further, the image processing apparatus described in each of the above embodiments may be realized by software using a computer and an image processing program. In this case, what is necessary is just to make it the structure which implement | achieves part or all of the process after step S3 demonstrated with the flowchart of FIG. 3 with a computer. Or what is necessary is just to set it as the structure which implement | achieves a part or all of the process after step S13 demonstrated with the flowchart of FIG. 7 with a computer. In order to be realized by a computer, information such as whether or not the image is in the gradation compression mode may be supplied to the computer together with the image data. Such information can be supplied using the EXIF information of the image data. By adopting such a configuration, it is possible to perform the same processing as in the present embodiment.

  The present invention can be implemented in various other forms without departing from the spirit or main features thereof. For this reason, the above-described embodiments are merely examples in all respects and should not be interpreted in a limited manner. The scope of the present invention is indicated by the scope of claims, and is not restricted by the text of the specification. Further, all modifications and changes belonging to the equivalent scope of the claims are within the scope of the present invention.

DESCRIPTION OF SYMBOLS 1 ... Electronic camera, 2 ... Shooting lens, 14 ... Image sensor, 17 ... Control part

Claims (9)

An acquisition unit for acquiring image data;
With respect to the image data, input / output characteristics defined by a gradation curve consisting only of a power component represented by y = k · x ⁿ (where x is an input, y is an output, and k and n are coefficients). Accordingly, an image processing unit that performs image processing including gradation conversion processing,
A correction unit for performing correction to improve the brightness of the dark portion gradation of the image data subjected to image processing including gradation conversion processing by the image processing unit;
An image processing apparatus comprising: a generation unit configured to generate a gain map corresponding to a luminance level based on image data on which image processing including gradation conversion processing has been performed by the image processing unit. The image processing apparatus according to claim 1.
An image processing apparatus comprising: a recording unit that records the gain map generated by the generating unit and the image data corrected by the correcting unit in association with each other. The image processing apparatus according to claim 1 or 2,
A low-pass processing unit that applies a low-pass filter to the image data subjected to image processing including gradation conversion processing by the image processing unit;
The correction unit uses the image data subjected to the low-pass filter to perform correction for improving the brightness of the dark portion gradation of the image data subjected to image processing including gradation conversion processing by the image processing unit. An imaging apparatus characterized by that. The image processing apparatus according to any one of claims 1 to 3,
A setting unit configured to generate a scene-referred image from the image data acquired by the acquiring unit;
When set to generate the scene-referred image by the setting unit,
The image processing unit performs the image processing on the image data,
The correction unit performs the correction,
The image processing apparatus, wherein the generation unit generates the gain map. An imaging unit that captures a subject image and generates image data;
An image processing apparatus according to any one of claims 1 to 4,
The electronic camera, wherein the acquisition unit acquires the image data from the imaging unit. An image processing program for realizing image processing on image data to be processed by a computer,
An acquisition step of acquiring the image data;
With respect to the image data, input / output characteristics defined by a gradation curve consisting only of a power component represented by y = k · x ⁿ (where x is an input, y is an output, and k and n are coefficients). Therefore, an image processing step for performing image processing including gradation conversion processing,
A correction step for performing correction to improve the brightness of the dark gradation of the image data subjected to image processing including gradation conversion processing in the image processing step;
An image processing program that realizes, on a computer, a generation step that generates a gain map according to a luminance level based on image data that has undergone image processing including gradation conversion processing in the image processing step. The image processing program according to claim 8,
An image processing program comprising: a recording step of recording the gain map generated in the generation step in association with the image data corrected in the correction step. The image processing program according to claim 6 or 7,
The image processing step further includes a low-pass processing step of applying a low-pass filter to the image data subjected to image processing including gradation conversion processing;
In the correction step, using the image data that has been subjected to the low-pass filter, correction is performed to improve the brightness of the dark portion gradation of the image data that has been subjected to image processing including gradation conversion processing in the image processing step. An image processing program characterized by that. In the image processing program according to any one of claims 6 to 8,
A setting step for setting to generate a scene-referred image from the image data acquired in the acquiring step;
When set to generate the scene-referred image in the setting step,
In the image processing step, the image processing is performed on the image data,
In the correction step, the correction is performed,
In the generation step, the gain map is generated.

Images