JP2015100091A - Image processing device, imaging apparatus, image processing method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technology for preventing a gradation in a specific region from being lost when converting a gradation property of image data.SOLUTION: An image processing device includes: detection means for detecting a specific region from image data; acquisition means for acquiring a representative luminance value in the specific region; generation means that generates a gamma curve for converting the representative luminance value into a predetermined target luminance value, converting a minimum luminance value in the specific region into a value based on a low luminance side luminance difference that is a difference between the representative luminance value and the minimum luminance value, and converting a maximum luminance value in the specific region into a high luminance side luminance difference that is a difference between the representative luminance value and the maximum luminance value; and correction means for correcting the image data in accordance with the gamma curve. As the low luminance side luminance difference is larger, the value based on the low luminance side luminance difference is smaller and as the high luminance side luminance difference is larger, the value based on the high luminance side luminance difference is larger.

Description

  The present invention relates to an image processing device, an imaging device, an image processing method, and a program.

  In recent years, imaging devices such as digital still cameras and digital camcorders equipped with a face detection function for correcting the brightness and color of a photographed person's face to be more appropriate have become widespread. By installing this face detection function, it is possible to properly capture a face even in a scene where originally it is difficult to properly capture a face such as a backlight scene. In many cases, a captured image captured by the imaging device is displayed on a display device such as a liquid crystal display, a projector, or a large liquid crystal television mounted on the imaging device. In order to display a captured image on a display device, it is necessary to convert a captured image signal into an output range that can be displayed on the display device by applying a gamma curve for converting input / output characteristics. This is because the characteristics of the outputable range of the imaging device and the displayable range of the display device are different. When the characteristics of the captured image signal do not match the characteristics of the display device, it is not possible to display with accurate gradation.

  In many imaging devices, the range that can be output to the display device with respect to the captured image signal output from the imaging device is narrow, and thus the captured image signal needs to be greatly compressed. For example, the output of the imaging device is 10 to 14 bits, whereas the output of the display device is often 8 bits, so compression of 1/4 to 1/64 or more is necessary. When performing compression, in many imaging apparatuses, it is common to increase the slope of the gamma curve in the low-brightness part and reduce the slope of the gamma curve in the high-brightness part. That is, since the captured image signal is greatly compressed as the luminance becomes high, non-linear conversion is performed.

  For this reason, the range of output values after conversion varies depending on the level of the luminance value of the subject even in a subject with a uniform gradation of luminance values. For example, when the luminance value of the subject exists on the low luminance side, such as in a backlight scene, the contrast increases. Conversely, when the luminance value of the subject exists on the high luminance side, the contrast decreases. Thereby, the gradation of the display image displayed on the display device is biased. In particular, when a gradation deviation occurs in a person's face, it causes the user to feel a greater sense of discomfort than a natural object or artificial object in the background. Therefore, when converting a captured image signal including a face into output characteristics of a display device, it is important to correct the gamma curve so that the conversion is performed while maintaining the gradation of the face.

  According to the technique disclosed in Patent Literature 1, a gamma curve is generated so that an average value of luminance values from a specific area such as a person's face becomes a median value of output. Thereafter, an average value of luminance values on the higher luminance side than the average value and an average value of luminance values on the lower luminance side than the average value are calculated. Then, the gamma curve is converted so that the average value on the high luminance side becomes the median value on the high luminance side and the average value on the low luminance side becomes the median value on the low luminance side. According to this technique, the brightness value of the face can be kept substantially constant by setting a specific area such as a person's face to the median value of the output. In addition, since the conversion is performed with the output median as a target, the compression associated with the conversion is not so large, and the gradation of the face can be maintained.

  According to the technique disclosed in Patent Document 2, the gamma curve is corrected so that the slope of the range indicating the face area on the histogram becomes about 1 from the detected face area histogram. At this time, correction is performed so that the continuity of the gamma curve is maintained. According to this technique, it is possible to maintain the gradation of the face in order to correct the gradient of the gamma curve to be constant regardless of the range of the luminance value of the face area in the captured image signal.

JP 2009-200743 A JP 2010-74222 A

  However, in Patent Document 1, since the gamma curve is generated so that the average value of the luminance values of the specific region is always the center position of the output, the face luminance value is always the same no matter what face is photographed. . In addition, since the overall shape of the gamma curve is finally determined by determining the median value of the gamma curve in the high brightness area and the low brightness area according to the brightness value other than the specific area, the face depends on the brightness value of the subject other than the face There is a problem that the gradation of the image is affected.

  In Patent Document 2, since the inclination of the face area of the gamma curve is corrected to about 1, it is possible to maintain the face gradation. However, when the face area exists in the low-luminance area or the high-luminance area, the gamma curves before and after the correction are greatly different, and there is a problem that the gradation of the background area other than the face is sacrificed.

  The present invention has been made in view of such a situation, and an object of the present invention is to provide a technique for suppressing the loss of gradation in a specific area when converting gradation characteristics of image data.

  In order to solve the above-described problems, the present invention provides a detection unit that detects a specific area from image data, an acquisition unit that acquires a representative luminance value of the specific area, and converts the representative luminance value into a predetermined target luminance value. And converting the minimum luminance value of the specific area to a value based on a low luminance side luminance difference that is a difference between the representative luminance value and the minimum luminance value, and converting the maximum luminance value of the specific area to the representative luminance value. Generating means for generating a gamma curve for converting to a value based on a high-luminance side luminance difference that is a difference between the maximum luminance value and a correcting means for correcting the image data by the gamma curve, A value based on the low-luminance side luminance difference is small as the side luminance difference is large, and an image processing device is provided wherein the value based on the high-luminance side luminance difference is large as the high-luminance side luminance difference is large.

  Other features of the present invention will become more apparent from the accompanying drawings and the following description of the preferred embodiments.

  According to the present invention, it is possible to suppress the loss of the gradation of the specific area when converting the gradation characteristics of the image data.

1 is a block diagram showing a schematic configuration of an imaging apparatus 100 according to a first embodiment. The conceptual diagram of the selection process of the gamma curve (reference | standard gamma curve) by the gamma curve production | generation part 7. FIG. The conceptual diagram of the correction process of the gamma curve by the gamma curve production | generation part 7. FIG. The conceptual diagram of the correction process of a gamma curve when the correction amount on the high luminance side is too large. The conceptual diagram of the correction process of a gamma curve when the amount of corrections on the low luminance side is too large. 3 is a flowchart showing a flow of image processing by the imaging apparatus 100.

  Embodiments of the present invention will be described below with reference to the accompanying drawings. The technical scope of the present invention is determined by the claims, and is not limited by the following individual embodiments. In addition, not all combinations of features described in the embodiments are essential to the present invention.

[First Embodiment]
FIG. 1 is a block diagram illustrating a schematic configuration of an imaging apparatus 100 according to the first embodiment. As shown in FIG. 1, the imaging device 100 is configured as an imaging device such as a so-called digital still camera. The imaging apparatus 100 is configured to use the imaging data captured by the imaging element of the imaging processing unit 1, generate a captured image by the image processing unit 2, and record the captured image by the recording unit 8. However, the imaging apparatus 100 may not include the imaging processing unit 1. In this case, the imaging apparatus 100 operates not as an imaging apparatus but as an image processing apparatus that processes image data acquired from the outside.

  The imaging apparatus 100 includes an operation unit 9 for performing a shooting operation, menu setting, and the like, and a display unit 10 for confirming a captured image, shooting information, and the like. In addition, the imaging apparatus 100 includes a face detection unit 4 for detecting a human face region from a captured image, and a luminance acquisition unit 5 for acquiring a luminance value of the face region detected by the face detection unit 4. In addition, the imaging apparatus 100 includes a histogram generation unit 6 that generates a histogram from the luminance value of the face area acquired by the luminance acquisition unit 5, and a gamma that converts the captured image into an output image for display on the display unit 10. And a gamma curve generation unit 7 for generating a curve. Furthermore, the imaging apparatus 100 includes a control processing unit 3 that controls each unit of the imaging apparatus 100.

  The imaging processing unit 1 includes an optical unit including a plurality of lenses, a diaphragm, an imaging device, a driver for driving the imaging device, a timing generation circuit, a CDS / AGC circuit, an A / D converter, and the like. The The optical unit includes a diaphragm for adjusting the amount of incident light incident from the outside, an ND (Neutral Density) filter, and the like. In addition, the imaging processing unit 1 drives the lens group with respect to the optical axis, thereby reducing the in-focus of the photographic subject and the captured image blur such as camera shake. The imaging device images a subject by photoelectric conversion, and the CDS / AGC circuit samples and amplifies image information based on charges (image signals) stored in each pixel of the imaging device. In sampling, correlated double sampling (CDS) (Correlated Double Sampling) is performed, and in amplification, automatic gain control (AGC) (Auto Gain Control) is performed. The A / D converter converts the image information (analog signal) output from the CDS / AGC circuit into a digital signal. A driver and timing generation circuit for driving the image sensor supplies a drive pulse for driving the image sensor to the image sensor, and reads an image captured by the image sensor and adjusts an exposure time.

  The image processing unit 2 performs various signal processing such as AWB (Auto White Balance) and gamma control on the image information (digital signal) output from the A / D converter, and generates a final image. The control processing unit 3 is a circuit called a microcomputer or a microcontroller, and controls the operation of the imaging apparatus 100 in an integrated manner.

  The face detection unit 4 detects a human face from the captured image captured by the imaging processing unit 1 and notifies the control processing unit 3 of the detection position and range. The luminance acquisition unit 5 acquires a luminance value from the captured image captured by the imaging processing unit 1. Here, the luminance acquisition unit 5 can also acquire the luminance value of only the face area detected by the face detection unit 4 instead of the luminance value of the entire captured image.

  The histogram generation unit 6 generates a histogram from the luminance values acquired by the luminance acquisition unit 5. Here, the histogram generation unit 6 can also generate a histogram of only the face area detected by the face detection unit 4 instead of the histogram of the entire captured image.

  The gamma curve generation unit 7 generates a gamma curve to be applied to the captured image by the image processing unit 2. The gamma curve generation unit 7 can select one of a plurality of preset gamma curves and correct the selected gamma curve based on information such as a luminance value and a histogram.

  The display unit 10 displays the final image (display image) generated by the image processing unit 2. The display unit 10 can superimpose an image indicating the face position and range detected by the face detection unit 4 on the display image.

  The recording unit 8 stores the final image generated by the image processing unit 2 in an internal memory or an external memory such as a memory card. A user (photographer) uses the operation unit 9 to set key operations, setting menus, and the like when photographing with the imaging apparatus 100.

  1 does not have to be independent as shown in FIG. For example, all or some of the functions of each unit may be included in the control processing unit 3.

  Next, the basic principle of image processing (particularly, gamma curve application processing) by the imaging apparatus 100 will be described. In the present embodiment, the face detection unit 4 acquires a human face area (face information indicating the position and range of the face) from the captured image captured by the imaging processing unit 1. Thereafter, the luminance acquisition unit 5 acquires the luminance information of the face region based on the captured image captured by the imaging processing unit 1 and the face information acquired by the face detection unit 4. In addition, the histogram generation unit 6 generates a luminance histogram of the face area.

  Note that in this embodiment, the imaging apparatus 100 acquires luminance information and a histogram of the face area in order to focus on the face area, but the area of interest is not limited to the face area. When attention is paid to other types of regions, the configuration of the imaging apparatus 100 is appropriately changed according to the nature of the region of interest. For example, when paying attention to a plant, the face detection unit 4 detects a plant region instead of the face region, and the luminance acquisition unit 5 and the histogram generation unit 6 acquire luminance information and a histogram of the detected plant region. Such an area of interest is also referred to as a “specific area”.

  The image processing unit 2 generates an image for recording and display by applying a gamma curve to the captured image captured by the imaging processing unit 1. The gamma curve generation unit 7 generates a gamma curve applied here. Specifically, the gamma curve generation unit 7 performs input such that the representative luminance value of the face area (for example, the average luminance of the face area) obtained by the luminance acquisition unit 5 is converted into the target luminance value of the face. A gamma curve having output characteristics is selected from a plurality of candidates. The target brightness value of the face may be a predetermined value set in advance, or may be a value determined by the control processing unit 3 based on some condition (for example, the brightness of the captured image). The plurality of candidates are stored in a memory (not shown) such as a ROM, for example. In addition, when there is no gamma curve that can convert the representative luminance value of the face region into the target luminance value among the candidates, the gamma curve generation unit 7 selects the gamma curve that can be converted closest to the target luminance value. To do. In this case, the gamma curve generation unit 7 calculates a difference between the gamma curve for converting the representative luminance value of the face area so as to match the target luminance value and the selected gamma curve, and later corrects the gamma curve. Consider the difference.

  FIG. 2 is a conceptual diagram of a gamma curve (reference gamma curve) selection process by the gamma curve generation unit 7. In the example of FIG. 2, it is assumed that three gamma curves 201, 202, and 203 are stored as candidates in a memory (not shown). In this case, the gamma curve generation unit 7 generates a gamma curve 203 for converting the representative luminance value of the face area into the target luminance value.

  Here, the gamma curve generation unit 7 uses the average value of the luminance of the face area as the representative luminance value, but is not limited to the average value, and represents, for example, the median value or the mode value of the luminance of the face area. It may be used as a luminance value.

  Thereafter, the gamma curve generation unit 7 calculates the range of luminance values of the face area from the luminance value acquired by the luminance acquisition unit 5 and the histogram generated by the histogram generation unit 6. Here, the range of the luminance value of the face area includes a range from the average luminance value of the face to the minimum luminance value on the low luminance side and a range from the average luminance value to the maximum luminance value on the high luminance side. Then, the gamma curve generation unit 7 calculates these two ranges, calculates a correction amount for each of the low luminance side and the high luminance side based on the size of the calculated range, and converts the calculated correction amount into a gamma curve. Apply (see Figure 3). The correction amount is calculated according to a predetermined calculation formula. Alternatively, the gamma curve generation unit 7 may acquire the correction amount from a table that associates the size of the range with the correction amount.

  As a result, a gamma curve is generated that passes through the target luminance value of the face region, the corrected minimum point on the low luminance side, and the corrected maximum point on the high luminance side. At this time, when an inflection point occurs, the gamma curve generation unit 7 generates a smooth gamma curve by performing an approximation process.

  Note that, regardless of the method for calculating the correction amount, the correction amount on the low luminance side is larger as the range on the lower luminance side is larger. That is, the larger the range on the low luminance side, the smaller the output value corresponding to the minimum luminance value of the face area. The correction amount on the high luminance side is larger as the range on the high luminance side is larger. That is, the larger the range on the high luminance side, the larger the output value corresponding to the maximum luminance value of the face area. As a result, the slope of the gamma curve corresponding to the luminance of the face area is maintained at an appropriate level, and the gradation of the face area is appropriately maintained.

  By the way, since the correction amount on the high luminance side is too large and hits the maximum output value (high luminance side threshold value), all the calculated correction amounts may not be applied. In this case, the gamma curve generation unit 7 performs correction in the direction of increasing the gradation of the face by increasing the slope of the gamma curve at the portion connecting the target luminance value and the maximum output value on the high luminance side (see FIG. 4). As shown in FIG. 4, strictly speaking, the gradient of the gamma curve is large at the maximum output value on the high luminance side, but the gradient of the gamma curve is small at the target luminance value. . What is important here is to suppress the loss of gradation on the high luminance side because all the correction amounts on the high luminance side cannot be applied. As shown in FIG. 4, the maximum value of the output on the high luminance side is suppressed. This objective can be achieved by increasing the slope of the gamma curve.

  In addition, when the correction amount on the low luminance side is large, there arises a problem that a low luminance subject is output darker in the background other than the face. Therefore, the gamma curve generation unit 7 corrects the gamma curve of the low-luminance side portion with the target luminance value (low-luminance side threshold) of the entire screen as a limit in order to prevent black crushing of the entire screen (FIG. 5 reference). By performing such control, it is possible to appropriately correct the gradation of the face.

  Next, image processing by the imaging apparatus 100 will be described with reference to FIG. The processing of each step in FIG. 6 is performed by the control processing unit 3 controlling each unit of the imaging apparatus 100 unless otherwise specified.

  In S101, the imaging processing unit 1 performs imaging. In S102, the control processing unit 3 determines whether or not the operation mode of the imaging apparatus 100 is a gamma curve correction mode for performing gamma curve correction. If it is not in the gamma curve correction mode, the process of this flowchart ends. If it is the gamma curve correction mode, the process proceeds to S103.

  In S103, the face detection unit 4 detects the face of a person existing in the captured image, and calculates the position and range (face area) of the face when a face is detected. In S104, the luminance acquisition unit 5 acquires a representative luminance value of the face area (here, the average value of the luminance of the face area) from the pixel information existing in the face area detected in S103. In S105, the histogram generation unit 6 generates a histogram of luminance values of the face area from the pixel information existing in the face area detected in S103.

  In S106, the gamma curve generation unit 7 determines whether or not there is a gamma curve for converting the average luminance value of the face area into the target luminance value among the gamma curve candidates stored in the ROM (not shown). judge. If present, in S107, the gamma curve generation unit 7 selects a gamma curve for converting the average luminance value of the face area into the target luminance value. On the other hand, when such a gamma curve does not exist, in S108, the gamma curve generation unit 7 selects a gamma curve for converting the average luminance value of the face area closest to the target luminance value. The gamma curve selected in S107 or S108 becomes a gamma curve (reference gamma curve) before correction.

  In S109, the gamma curve generation unit 7 calculates the luminance range of the face area from the average value of the luminance of the face area acquired in S104 and the histogram acquired in S105. Specifically, the gamma curve generation unit 7 determines the luminance difference between the average value and the minimum luminance value on the low luminance side (low luminance side luminance difference), and the maximum luminance value on the high luminance side with respect to the average value. And the luminance difference (high luminance side luminance difference) is calculated.

  In S110, the gamma curve generation unit 7 corrects two points of the minimum luminance point on the low luminance side and the maximum luminance point on the high luminance side of the reference gamma curve by using the luminance difference between the low luminance side and the high luminance side. A correction amount for performing is calculated. As described with reference to FIG. 3, the correction amount can be calculated using, for example, a predetermined calculation formula or a table. Here, a table is used. That is, the correction amount is calculated using a table that associates the magnitude of the luminance difference (luminance range) from the average luminance value with the difference (correction amount) from the target luminance value.

  In S111, the gamma curve generation unit 7 corrects the reference gamma curve so that it passes through three points of the target luminance value of the face region, the minimum luminance value on the low luminance side, and the maximum luminance value on the high luminance side. In S108, the gamma curve for converting the average value closest to the target luminance value may be selected as the reference gamma curve. In this case, the gamma curve generation unit 7 calculates the difference between the gamma curve that can convert the average value into the target luminance value and the reference gamma curve, and corrects the reference gamma curve to match the gamma curve that can be converted into the target luminance value Determine the amount.

  In S112, the gamma curve generation unit 7 determines whether the correction result of the maximum luminance point on the high luminance side exceeds the maximum value (upper limit value) of the output. When exceeding the upper limit, in S113, the gamma curve generation unit 7 corrects the output value of the maximum luminance point on the high luminance side of the gamma curve to the maximum value. In addition, the gamma curve generation unit 7 increases the slope of the gamma curve portion that connects the target luminance value and the maximum luminance point on the high luminance side. As a result, it is possible to suppress a decrease in the gradation of the face area while suppressing the output value within the upper limit value.

  In S114, the gamma curve generation unit 7 determines whether the correction result of the minimum luminance point on the low luminance side is less than the target luminance value of the entire screen (less than the low luminance side threshold value). If it is less than the target luminance value of the entire screen, in S115, the gamma curve generation unit 7 sets a limit so that the correction value of the minimum luminance point on the low luminance side becomes the target luminance value of the entire screen. Thereby, generation | occurrence | production of black crushing can be suppressed.

  In S116, the image processing unit 2 applies the gamma curve finally generated to the gamma curve generated by the gamma curve generation unit 7 to the captured image acquired in S101, and generates an output image. As a result, the gradation of the face of the person in the captured image can be made appropriate.

  As described above, according to the present embodiment, the imaging apparatus 100 can be divided into the representative brightness value of the face area, the brightness range of the face area on the high brightness side, and the brightness range of the face area on the low brightness side. Based on this, a gamma curve to be applied to the image data is generated.

  As a result, it is possible to suppress the loss of the gradation of the specific area when converting the gradation characteristics of the image data.

[Other Embodiments]
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, etc.) of the system or apparatus reads the program. It is a process to be executed.

Claims (10)

Detecting means for detecting a specific area from the image data;
Obtaining means for obtaining a representative luminance value of the specific area;
The representative luminance value is converted into a predetermined target luminance value, and the minimum luminance value of the specific area is converted into a value based on a low luminance side luminance difference that is a difference between the representative luminance value and the minimum luminance value, Generating means for generating a gamma curve for converting the maximum luminance value of the specific region into a value based on a high luminance side luminance difference which is a difference between the representative luminance value and the maximum luminance value;
Correction means for correcting the image data by the gamma curve;
With
The value based on the low luminance side luminance difference is small as the low luminance side luminance difference is large, and the value based on the high luminance side luminance difference is large as the high luminance side luminance difference is large. When the value based on the high luminance side luminance difference exceeds the high luminance side threshold, the generation unit converts the maximum luminance value of the specific region into the high luminance side threshold by the gamma curve, 2. The image according to claim 1, wherein the gamma curve is generated such that a slope of the gamma curve is larger than a value based on the high-luminance-side luminance difference does not exceed a high-luminance-side threshold value. Processing equipment. When the value based on the low-luminance side luminance difference is less than the low-luminance side threshold value, the generation unit converts the gamma curve so that the gamma curve converts the minimum luminance value of the specific region into the low-luminance side threshold value. The image processing apparatus according to claim 1, wherein the image processing apparatus generates an image. The generating means includes
A gamma curve for converting the representative luminance value to the target luminance value is selected from a plurality of candidates for the gamma curve,
The selected gamma curve converts the minimum luminance value of the specific region into a value based on the low luminance side luminance difference, and converts the maximum luminance value of the specific region into a value based on the high luminance side luminance difference. The image processing apparatus according to any one of claims 1 to 3, wherein the gamma curve is generated by correcting the selected gamma curve. When the gamma curve for converting the representative luminance value to the target luminance value does not exist in the plurality of candidates, the generation unit includes:
From the plurality of candidates, select a gamma curve for converting the representative luminance value closest to the target luminance value,
The selected gamma curve converts the representative luminance value to the target luminance value, converts the minimum luminance value of the specific area to a value based on the low luminance side luminance difference, and sets the maximum luminance value of the specific area to the The image processing apparatus according to claim 4, wherein the gamma curve is generated by correcting the selected gamma curve so as to be converted into a value based on a luminance difference on the high luminance side. The image processing apparatus according to claim 1, wherein the specific area is a human face area. The representative luminance value is an average value of luminance of the specific region, a median value of luminance of the specific region, or a mode value of luminance of the specific region. The image processing apparatus according to item. An image processing apparatus according to any one of claims 1 to 7,
Imaging means for imaging the image data;
An imaging apparatus comprising: An image processing method by an image processing apparatus,
A detection step in which the detection means of the image processing apparatus detects a specific area from the image data;
An acquisition step in which an acquisition unit of the image processing apparatus acquires a representative luminance value of the specific area;
The generation unit of the image processing device converts the representative luminance value into a predetermined target luminance value, and sets the minimum luminance value of the specific area as a low luminance side luminance that is a difference between the representative luminance value and the minimum luminance value. Generating a gamma curve for converting to a value based on the difference and converting the maximum luminance value of the specific region into a value based on a high luminance side luminance difference that is a difference between the representative luminance value and the maximum luminance value; ,
A correction step in which the correction means of the image processing apparatus corrects the image data with the gamma curve;
With
The value based on the low-luminance side luminance difference is small as the low-luminance side luminance difference is large, and the value based on the high-luminance side luminance difference is large as the high-luminance side luminance difference is large.   The program for functioning a computer as each means of the image processing apparatus of any one of Claims 1 thru | or 7.

Images