JP2011003048A - Image processing apparatus and image processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain a satisfactory gradation correction result, even for a section where haze occurs, in an image during gradation correction in the image, such as a picked-up image.

SOLUTION: An ε-filter 107 extracts a broad luminance component (Y_low) from a luminance component (Y) of an image to be processed. A contrast component generating section 108 subtracts the broad-luminance component (Y_low) from the luminance component (Y) and generates a contrast component (Y_high) for haze elimination, corresponding to a high-frequency component of the image to be processed. A contrast-component adjusting section 109 obtains a haze degree (degree of haze) in the image to be processed, on the basis of distribution area information (RGBmax and RGBmin) of the luminance of the image to be processed, and adjusts the contrast component (Y_high) for haze elimination, according to the obtained haze degree. By having A luminance correcting section 110 corrects the gradation of the image to be processed, and adding the adjusted contrast component (Y'_high) to the luminance component (Y) of the image to be processed. Gradation correction which is optimal for haze elimination in the image to be processing can be performed.

COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image processing technique, and more particularly to an image gradation correction process.

  Conventionally, for example, Patent Document 1 describes an image processing technique for performing the following first to third processes in image gradation correction processing. In the first processing, a texture component (details in an image is formed by subtracting, from the input image data, a structure component (a component corresponding to the contour of an object in the image) extracted from the input image data by a nonlinear filter. Component). In the second process, a luminance histogram is acquired from the input image data, and the contrast of the structure component is corrected based on a correction curve generated based on the acquired histogram. In the third process, new image data is generated by adding the corrected structure component and the previously amplified texture component.

  In the above technique, it can be expected that the contrast of the entire image is improved and details in the image are enhanced. In addition, since the contrast of the structure component is corrected using a correction curve corresponding to a luminance histogram in the input image data, the corrected image data can be more effectively subjected to contrast correction.

JP 2005-309945 A

  However, in the technique of Patent Document 1, even if the contrast of the entire image can be effectively corrected and the details in the image can be enhanced, there is a part in which wrinkles occur in the image to be corrected. In such a case, a satisfactory gradation correction result is not always obtained for a portion where wrinkles occur, and there is a problem that the gradation correction effect on the image is not yet sufficient.

  Note that the portion where the image is wrinkled is a portion where the image is wrinkled because the difference in brightness level between the pixels in the local region is small. Examples of wrinkled parts include parts that appear whitish due to diffuse reflection of light, such as distant parts of images taken of landscapes, and images taken through the water surface or glass surface. Thus, it is a part that looks whitish due to reflection of light.

  The present invention has been made in view of such a conventional problem, and an object of the present invention is to obtain a favorable gradation correction result even in a portion where wrinkles occur in an image when performing gradation correction in an image such as a captured image. And

  In the image processing apparatus according to the first aspect of the present invention, an extraction unit that extracts a high-frequency component from the processing target image, and acquisition of the degree information indicating the degree of wrinkles of the stagnated portion in the processing target image Means for adjusting the high frequency component extracted by the extracting means according to the degree of wrinkle indicated by the degree information acquired by the acquiring means, and the high frequency component adjusted by the adjusting means to be processed And adding means for adding to the image.

  In the image processing program according to the first aspect of the present invention, the computer obtains a procedure for extracting a high-frequency component from the processing target image and the degree information indicating the level of wrinkles in the processing target image. A procedure for adjusting the high-frequency component extracted by the extracting means according to the degree of wrinkle indicated by the degree-of-degree information acquired by the acquiring means, and the high-frequency component adjusted by the adjusting means to be processed And a procedure of adding to the image.

  According to the present invention, it is possible to obtain an excellent gradation correction result even in a portion where wrinkles are generated in an image when performing gradation correction in an image such as a captured image.

It is a block diagram which shows an example of the hardware constitutions of the digital still camera to which this invention is applied. It is a functional block diagram of a digital still camera to which the present invention is applied. It is a functional block diagram which shows the principal part of an image processing part. It is a figure which shows the input / output characteristic of the RGB data at the time of the gamma conversion by a gamma conversion part. It is a figure which shows the 1st correction characteristic line and 2nd correction characteristic line which a correction characteristic production | generation part acquires. FIG. 10 is a characteristic diagram showing a change in contrast enhancement level with respect to the degree of brightness. It is a flowchart which shows the processing content in an image process part.

  Embodiments to which the present invention is applied will be described below with reference to the drawings. FIG. 1 is a block diagram showing an outline of a hardware configuration of a digital still camera (hereinafter simply referred to as a digital camera) 1 exemplified as an embodiment of the present invention.

  As shown in FIG. 1, the digital camera 1 has a CCD (Charge Coupled Device) 2 for imaging a subject. The CCD 2 has a structure in which a color filter having a specific color arrangement such as a Bayer arrangement is provided on a photosensitive surface on which an optical image of a subject is formed by an optical system (not shown). The CCD 2 is driven by horizontal and vertical transfer drive signals supplied from the horizontal / vertical driver 3 to convert an optical image of a subject into an electric signal, and the converted electric signal is used as an imaging signal for CDS ( This is supplied to a CDS / AD circuit 4 composed of a Correlated Double Sampling) circuit and an A / D converter (Analog-to-Digital converter).

  The horizontal / vertical driver 3 operates based on a timing signal generated by a TG (Timing Generator) 5 to generate the above-described horizontal and vertical transfer drive signals and drive the CCD 2. The timing signal generated by the TG 5 is also supplied to the CDS / AD circuit 4. The CDS / AD circuit 4 operates based on the timing signal supplied from the TG 5 to remove noise included in the imaging signal output from the CCD 2 and convert the imaging signal after noise removal into a digital signal. The later digital signal is supplied to a DSP (Digital Signal Processor) 6.

  The DSP 6 has a configuration including a buffer memory 6a for processing a digital signal supplied from the CDS / AD circuit 4, that is, image data including data of each pixel having only single color information. The series of signal processing described below is performed. That is, the DSP 6 interpolates the color information that is insufficient for each pixel from the peripheral pixels of each pixel with respect to the image data supplied from the CDS / AD circuit 4, so that R (Red), G (Green) is obtained for each pixel. De-mosaic processing for generating image data having B (Blue) color component information, that is, RGB data, is performed.

  Further, the DSP 6 performs YUV conversion processing for converting RGB data generated by demosaic processing into image data expressed by a luminance component (Y) and two color difference components (Cb, Cr) for each pixel, that is, YUV data. The YUV data after conversion is subjected to white balance adjustment processing and various filter processing. Further, the DSP 6 performs gradation correction processing, which will be described later, in the series of signal processing described above.

  Then, the DSP 6 sequentially supplies YUV data after a series of signal processing to an SDRAM (Synchronous dynamic random-access memory) 7. The SDRAM 7 temporarily stores the YUV data supplied from the DSP 6.

  Further, the DSP 6 stores the image data stored in the SDRAM 7 every time one frame of YUV data (image data) is stored in the SDRAM 7 in a state where the recording mode is set as the operation mode in the digital camera 1. And the read image data is supplied to an LCD (Liquid Crystal Display) 8.

  The LCD 8 includes a liquid crystal display (not shown), a drive circuit that drives the liquid crystal display, and the like, and displays an image based on the image data supplied from the DSP 6 on the screen of the liquid crystal display as a live view image.

  On the other hand, the above-described TG 5 and DSP 6 are connected to a CPU (Central Processing Unit) 9 via a bus 13, and operations of the TG 5 and DSP 6 are controlled by the CPU 9.

  The CPU 9 controls the entire operation of the digital camera 1 by operating according to a program stored in a flash memory 10 which is an EEPROM (Electric Erasable Programmable Read Only Memory) whose rewritable contents are stored.

  Further, the CPU 9 compresses the image data temporarily stored in the SDRAM 7 by a predetermined compression method such as a JPEG (Joint Photographic Expert Group) method at the time of shooting in a state where the recording mode is set as the operation mode in the digital camera 1. Then, the compressed image data is recorded in the external memory 11 as an image file. The external memory 11 is a memory card that is detachable from the camera body connected to the bus 13 via a card interface (not shown).

  In addition, when the playback mode is set as the operation mode for the digital camera 1, the CPU 9 reads a predetermined image file (compressed image data) from the external memory 11 as necessary, and decompresses the read data. To SDRAM7. Further, the CPU 9 supplies the data (YUV data) developed on the SDRAM 7 to the LCD 8 via the DSP 6 so that the recorded image is displayed on the LCD 8.

  A key input unit 12 is connected to the bus 13. The key input unit 12 includes various operation keys necessary for the user to operate the digital camera 1, such as a power key and a shutter key, a mode setting key for setting a recording mode and a reproduction mode, and the like. Then, the CPU 9 sequentially detects the operation state of each operation key in the key input unit 12, and executes various processes according to the program according to the user's request determined based on the detected operation state.

  FIG. 2 is a functional block diagram showing the configuration of the digital camera 1 by function. As shown in FIG. 2, the digital camera 1 mainly includes an imaging unit 51, an image processing unit 52, a control unit 53, a work memory 54, a program memory 55, an image recording unit 56, a display unit 57, and an operation unit 58. Has been.

  Each functional block is realized by one or a plurality of hardware shown in FIG. 1 as described later. In other words, the imaging unit 51 is realized by the CCD 2, the horizontal / vertical driver 3, the CDS / AD circuit 4, and the TG 5, and is a functional part that captures a subject and acquires a captured image. The image processing unit 52 is realized by the DSP 6 and is a functional part that performs the above-described various image processing on the captured image. The control unit 53 is realized by the CPU 9. The work memory 54 is realized by the SDRAM 7, and the program memory 55 is realized by the flash memory 10. The image recording unit 56 is realized by the external memory 11, the display unit 57 is realized by the LCD 8, and the operation unit 58 is realized by the key input unit 12 described above.

  FIG. 3 is a main part of the image processing unit 52, and the image processing unit 52 corrects the gradation of a captured image (hereinafter referred to as a processing target image) during the demosaic process and the YUV conversion process described above. It is the functional block diagram which showed the detail of the functional part. As shown in FIG. 3, the image processing unit 52 includes an image buffer 101, a gamma conversion unit 102, a brightness distribution area acquisition unit 103, a correction characteristic generation unit 104, a gradation correction unit 105, a YUV conversion unit 106, and an ε filter. 107, a contrast component generation unit 108, a contrast component adjustment unit 109, and a luminance correction unit 110.

  Hereinafter, each unit of the image processing unit 52 illustrated in FIG. 3 will be described. The image buffer 101 is a functional part realized by the above-described memory 6a (see FIG. 1), and stores image data of an image to be processed and RGB data (RGB) generated by demosaic processing. The pixel values of the R, G, and B color components in the RGB data are in the range of “0 to 255”.

  The gamma conversion unit 102 performs basic gradation correction processing common to all processing target images on the processing target image. That is, the gamma conversion unit 102 reads out the RGB data stored in the image buffer 101 and performs gamma conversion according to predetermined gamma characteristics corresponding to various characteristics of the digital camera 1. More specifically, the gamma conversion unit 102 is determined in advance for the purpose of securing a natural gradation in the processing target image when displaying the processing target image, for example, according to the input / output characteristics shown in FIG. The pixel value of RGB data is adjusted. Then, the gamma conversion unit 102 supplies the RGB data after the gamma conversion to the gradation correction unit 105.

  On the other hand, the brightness distribution area acquisition unit 103 reads out the RGB data stored in the image buffer 101 and, for every pixel, the minimum value (RGBmin) of the R pixel value, the G pixel value, and the B pixel value. ) And the maximum value (RGBmax) of the R pixel value, the G pixel value, and the B pixel value. Next, the brightness distribution area acquisition unit 103 acquires the minimum color component value among the minimum values (RGBmin) in all pixels as the minimum brightness level (black_lev) in the processing target image. In addition, the brightness distribution area acquisition unit 103 acquires the maximum component value among the maximum values (RGBmax) in all pixels as the maximum brightness level (white_lev) in the processing target image. Then, the brightness distribution area acquisition unit 103 corrects the minimum brightness level (black_lev) and the maximum brightness level (white_lev) as distribution area information indicating the brightness distribution area in the processing target image. This is supplied to the generation unit 104, the contrast component adjustment unit 109, and the luminance correction unit 110.

  The correction characteristic generation unit 104 is a functional part that implements generation means. Based on the minimum brightness level (black_lev) and the maximum brightness level (white_lev) supplied from the brightness distribution area acquisition unit 103, the correction characteristic generation unit 104 performs the basic gradation correction described above on the processing target image. A correction characteristic line used to perform a tone correction process specific to a processing target image different from the process is acquired. Then, the correction characteristic generation unit 104 supplies the acquired correction characteristic line to the gradation correction unit 105.

  A procedure for generating a correction characteristic line in the correction characteristic generation unit 104 will be described below. That is, the correction characteristic generation unit 104 acquires the first correction characteristic line illustrated in FIG. 5A and then adjusts the acquired first correction characteristic line to thereby obtain the first correction characteristic line illustrated in FIG. 2 correction characteristic lines are acquired as correction characteristic lines unique to the processing target image. In FIG. 5, the horizontal axis represents the brightness level (input level) of the pixel before gradation correction processing, and the vertical axis represents the brightness level (output level) of the pixel after gradation correction processing.

  The first correction characteristic line shown in FIG. 5A is a straight line that rises to the right, and increases the brightness of each pixel of the processing target image in proportion to the brightness level of each pixel. At the same time, the first correction characteristic line converts the minimum brightness level (black_lev) at the input level to the minimum level “0” at the output level, and the maximum brightness level (white_lev) at the input level at the output level. The maximum level is converted to “255”. That is, it is a correction characteristic line that expands the brightness distribution width in the processing target image to the maximum width.

  Further, the second correction characteristic line shown in FIG. 5B is a right-upward S-shaped curve, and the correction characteristic generation unit 104 acquires the second correction characteristic line illustrated in the following procedure. First, the correction characteristic generation unit 104 calculates an average brightness level (ave_lev) between the minimum brightness level (black_lev) and the maximum brightness level (white_lev) in the processing target image. Further, the correction characteristic generation unit 104 calculates a brightness level (white side_lev) on the bright side located between the intermediate brightness level (ave_lev) and the maximum brightness level (white_lev).

  Next, the correction characteristic generation unit 104 generates a point A corresponding to the minimum brightness level (black_lev) on the first correction characteristic line in FIG. 5A and the above-mentioned brightness level (white side_lev). 3 points are identified: a point B corresponding to 1 and a point C corresponding to the maximum brightness level (white_lev).

  Thereafter, the correction characteristic generation unit 104 calculates an S-shaped curve that passes through the three points A, B, and C and has the point B as an inflection point, and uses the calculated S-shaped curve as a second correction characteristic line. Get as. The correction characteristic generation unit 104 calculates an S-shaped curve by connecting the above points A, B, and C by a known spline curve interpolation method.

  In other words, the correction characteristic generation unit 104 has a slope of the specific section X on the bright side centered on the above-described brightness level (white side_lev) as a correction characteristic line specific to the processing target image, rather than the specific section X. A second correction characteristic line larger than the gradient of the dark section and the section brighter than the specific section X is acquired.

  The gradation correction unit 105 is a functional part that implements correction means. The tone correction unit 105 uses the second correction characteristic line generated by the correction characteristic generation unit 104 to calculate the pixel values for all color components of the RGB data after gamma conversion supplied from the gamma conversion unit 102 based on the second correction characteristic line. to correct. That is, the tone correction unit 105 further performs tone correction processing specific to the processing target image in which the content of the processing target image is reflected on the image after the basic tone correction processing is performed by gamma conversion. . Then, the gradation correction unit 105 supplies the RGB data after the gradation correction to the YUV conversion unit 106.

  The YUV conversion unit 106 performs YUV conversion processing for converting the RGB data after the above-described two-stage gradation correction processing into YUV data, and supplies the converted YUV data to the luminance correction unit 110. Further, the YUV conversion unit 106 supplies the luminance component (Y (x, y)) of each pixel generated by the YUV conversion process to the ε filter 107 and the contrast component generation unit 108.

  The ε filter 107 is an image filter mainly used for the purpose of removing small amplitude noise (high frequency component) superimposed on image data with a steep change, that is, for smoothing the image. This is a smoothing filter having edge retention performance. The ε filter 107 removes small amplitude noise (high frequency component) from the luminance component (Y (x, y)) supplied from the YUV conversion unit 106 by a filter process described later to thereby obtain a general luminance component ( Low-frequency component), and the generated overall luminance component (y_low (x, y)) is supplied to the contrast component generation unit 108.

  Here, filter processing in the ε filter 107 will be described. The ε filter 107 pays attention to a pixel area composed of three vertical and horizontal pixels with each pixel as a target pixel (a total of nine pixel areas centered on each target pixel). That is, attention is paid to the target pixel and eight peripheral pixels located in the vicinity thereof. Then, the pixel value of each peripheral pixel is adjusted such that each difference value between the pixel value of the target pixel and the pixel value of each peripheral pixel is equal to or less than a predetermined threshold T (T = 20). Further, the sum of each pixel value obtained by multiplying the original pixel value of the target pixel and the adjusted pixel value of each peripheral pixel by 1/9 as a predetermined coefficient is calculated. A new pixel value of the pixel is set. In the above filter processing, the range of the pixel region of interest, the value of the threshold T, and the coefficient by which the pixel value of each pixel is multiplied can be changed as appropriate.

  The contrast component generation unit 108 is a functional part that realizes an extraction unit in cooperation with the YUV conversion unit 106 and the ε filter 107. The contrast component generation unit 108 is based on the luminance component (Y (x, y)) supplied from the YUV conversion unit 106 and the general luminance component (Y_low (x, y)) supplied from the ε filter 107 as follows. The wrinkle removal contrast component (Y_high (x, y)), which is the high-frequency component of the processing target image, is acquired according to the equation (1).

Y_high (x, y) = Y (x, y) −Y_low (x, y) (1)
However,
Y (x, y): input Y component (0 to 255)
Y_low (x, y): Overall luminance component (0 to 255)
Y_high (x, y): Contrast adjustment component for removing wrinkles (−255 to 255)

  That is, the contrast component generation unit 108 subtracts the high-frequency component obtained by subtracting the general luminance component (Y_low (x, y)) from the original luminance component (Y (x, y)), and the contrast component (Y_high) for removing wrinkles. (X, y)). Then, the contrast component generation unit 108 supplies the acquired contrast component (Y_high (x, y)) to the contrast component adjustment unit 109.

  The contrast component adjustment unit 109 is a functional part that realizes the acquisition unit in cooperation with the brightness distribution area acquisition unit 103, and is a functional part that simultaneously realizes the adjustment unit. The contrast component adjustment unit 109 performs adjustment described later on the wrinkle removal contrast component (Y_high (x, y)) supplied from the contrast component generation unit 108, and the adjusted contrast component (Y_high ′ (x, y, y)) is supplied to the luminance correction unit 110.

  Specific adjustment contents of the contrast component (Y_high (x, y)) in the contrast component adjustment unit 109 will be described below. When adjusting the contrast component (Y_high (x, y)), the contrast component adjustment unit 109 first assumes the degree of darkness (darkness) in the blurred portion assuming that the blurred portion exists in the processing target image. Acquire the index (K) as an index.

  That is, the contrast component adjustment unit 109 calculates the degree of degree (K) based on the minimum brightness level (black_lev) and the maximum brightness level (white_lev) supplied from the brightness distribution area acquisition unit 103 by the following formula ( Calculate according to 2).

K = (black_lev + (255−white_lev)) (2)
However,
K: Degree degree (0-255)
black_lev: Minimum brightness level (0 to 255) of the processing target image
white_lev: Maximum brightness level of the processing target image (0 to 255)

  That is, the contrast component adjustment unit 109 assumes that there is a stagnation part in the processing target image, and uses the degree (K) as an index of the degree of stagnation (darkness) in the stagnation part. A value indicating the difference between the brightness distribution width and the maximum expressible brightness width (0 to 255) is acquired.

  Next, the contrast component adjustment unit 109 acquires the contrast enhancement level (adj_lev) using a predetermined arithmetic function that uses the degree of degree (K) calculated by Expression (2) as a parameter. The arithmetic function is a function having a characteristic that the contrast enhancement level (adj_lev) increases in proportion to the degree of power (K). That is, the arithmetic function is a function in which the contrast enhancement level (adj_lev) changes in a range of “384 to 255” as shown in FIG. 6 in proportion to the degree of brightness (K). The arithmetic function only needs to have a characteristic (a characteristic that rises to the right) in which the contrast enhancement level (adj_lev) increases in proportion to the degree of brightness (K).

  Next, the contrast component adjusting unit 109 converts the wrinkle removal contrast component (Y_high (x, y)), the contrast enhancement level (adj_lev), and the luminance component (Y ( x, y)). At the time of such adjustment, the contrast component adjustment unit 109 checks whether or not the wrinkle removal contrast component (Y_high (x, y)) is equal to or greater than “0”.

  Then, the contrast component adjustment unit 109 determines that the contrast component (Y_high (x, y)) of a pixel having a value of “0” or more is a highlight emphasis contrast component, and the value is less than “0”. The contrast component (Y_high (x, y)) of the pixel is determined as a contrast component for shadow enhancement.

  Thereafter, the contrast component adjusting unit 109 separately adjusts the contrast component for removing wrinkles (Y_high (x, y)) into a contrast component for highlight emphasis and a contrast component for shadow emphasis. The bandwidth of the component (Y_high (x, y)) is adjusted.

  More specifically, the contrast component adjustment unit 109 adjusts the contrast component for highlight enhancement according to the following equation (3-1), and the contrast component for shadow enhancement includes the following equation (3-2). Adjust according to

但し、
Ｙ（ｘ，ｙ） ：入力Ｙ成分（０〜２５５）
ａｄｊ＿ｌｅｖ ：コントラスト強調レベル（３８４〜５１２）
Ｙ＿ｈｉｇｈ（ｘ，ｙ） ：霞除去用のコントラスト成分（−２５５〜２５５）
Ｙ＿ｈｉｇｈ'（ｘ，ｙ）：調整後のコントラスト成分

However,
Y (x, y): input Y component (0 to 255)
adj_lev: Contrast enhancement level (384 to 512)
Y_high (x, y): Contrast component for removing wrinkles (−255 to 255)
Y_high ′ (x, y): Contrast component after adjustment

  That is, the contrast component adjustment unit 109 adds the first adjustment coefficient composed of “(adj_lev−256) / 256” to the contrast component (Y_high (x, y)) for removing wrinkles, and “(256−Y (x, y)) / 256 ”highlight adjustment second adjustment coefficient or“ Y (x, y) / 256 ”shadow enhancement second adjustment coefficient multiplied by the contrast after adjustment Acquired as a component (Y_high ′ (x, y)).

  Here, the first adjustment coefficient is proportional to the contrast enhancement level (adj_lev), that is, the degree (K) of the processing target image, and varies in the range of “0.5” to “1”. Therefore, the adjusted contrast component (Y′_high (x, y)) is smaller as the degree of brightness (K) is smaller, and conversely, it is larger as the degree of hue (K) is larger.

  On the other hand, the second adjustment coefficient when the contrast component (Y_high (x, y)) for wrinkle removal is for highlight enhancement (value is “0” or more) (hereinafter, second adjustment for highlight enhancement). The coefficient is reduced as the original luminance component (Y (x, y)) increases, and changes in the range of “1” to “1/256”. Therefore, when the contrast component for removing wrinkles (Y_high (x, y)) is for highlight enhancement (value is “0” or more), the adjusted contrast component (Y′_high (x, y)) is, for example, If the first adjustment coefficient is the maximum (“1”), the value is any value from “0” to “256”, that is, “0” or a positive value. Also, the adjusted contrast component (Y′_high (x, y)) becomes smaller as the original luminance component (Y (x, y)) becomes larger.

  Also, a second adjustment coefficient when the contrast component for removing wrinkles (Y_high (x, y)) is for shadow enhancement (value is less than “0”) (hereinafter referred to as a second adjustment coefficient for shadow enhancement). .) Becomes smaller as the original luminance component (Y (x, y)) is smaller, and changes in the range of “255/256” to “0”. Therefore, when the contrast component (Y_high (x, y)) for removing wrinkles is for shadow enhancement (value is less than “0”), the adjusted contrast component (Y′_high (x, y)) is, for example, If the adjustment coefficient of 1 is the maximum (“1”), the value is any value from “0” to “−255”, that is, a value equal to or less than “0”. In addition, the adjusted contrast component (Y′_high (x, y)) becomes smaller as the original luminance component (Y (x, y)) becomes smaller.

  The luminance correction unit 110 is a functional part that realizes the adding means, and at the same time, a functional part that realizes a part of the adjusting means. The luminance correction unit 110 adds the adjusted contrast component (Y_high ′ (x, y) supplied from the contrast component adjustment unit 109 to the luminance component (Y (x, y)) of the YUV data supplied from the YUV conversion unit 106. )) Is added according to the following formula (4).

但し、
Ｙ（ｘ，ｙ） ：入力Ｙ成分（０〜２５５）
Ｙ＿ｈｉｇｈ'（ｘ，ｙ）：調整後のコントラスト成分
ｂｌａｃｋ＿ｌｅｖ ：処理対象画像の最小の明るさレベル（０〜２５５）
ｗｈｉｔｅ＿ｌｅｖ ：処理対象画像の最大の明るさレベル（０〜２５５）
Ｙ'（ｘ，ｙ） ：輝度補正処理後のＹ成分（０〜２５５）

However,
Y (x, y): input Y component (0 to 255)
Y_high ′ (x, y): Adjusted contrast component black_lev: Minimum brightness level (0 to 255) of the processing target image
white_lev: Maximum brightness level of the processing target image (0 to 255)
Y ′ (x, y): Y component (0 to 255) after luminance correction processing

  That is, the luminance correction unit 110 adds an addition amount when adding the adjusted contrast component (Y_high ′ (x, y)) to the original luminance component (Y (x, y)) in the processing target image. Adjust according to the brightness distribution range. That is, the luminance correction unit 110 decreases the amount of addition of the adjusted contrast component (Y_high ′ (x, y)) to the original luminance component (Y (x, y)) as the brightness distribution range is narrower. . In addition, the luminance correction unit 110 stores the YUV data after the luminance correction processing in the image buffer 101. Thereby, the gradation correction processing for the processing target image in the image processing unit 52 is completed.

  Then, as described above, the image processing unit 52 corrects the gradation of the processing target image by a series of correction processing by the operation of each unit in FIG. 3, and then temporarily stores the corrected YUV data in the image buffer 101. . Thereafter, the image processing unit 52 reads the corrected YUV data stored in the image buffer 101, and performs other digital signal processing such as white balance adjustment processing and various filter processing.

  FIG. 7 is a flowchart showing the contents of the series of correction processes described above by the respective units in FIG. 3 of the image processing unit 52. Note that the details of the processing of each step are as described above, and thus description thereof is omitted.

  As shown in FIG. 7, in the correction processing in the image processing unit 52, the gamma conversion unit 102 performs gamma conversion processing on the RGB data of the processing target image stored in the image buffer 101 (step S1). Next, the brightness distribution acquisition unit 103 acquires the maximum and minimum brightness levels of the processing target image (step S2). Further, the correction characteristic unit 104 generates a correction characteristic line (second correction characteristic line) unique to the processing target image based on the maximum and minimum brightness levels (step S3). Thereafter, the gradation correction unit 105 performs gradation correction processing on the RGB data after the gamma conversion in accordance with the correction characteristic line specific to the processing target image (step S4).

  Subsequently, the YUV conversion unit 106 converts the RGB data after the gradation correction processing into YUV data (step S5). Next, the ε filter 107 extracts a general luminance component from the YUV data (step S6). Further, the contrast component generation unit 108 acquires a wrinkle removal contrast component based on the luminance component and the low frequency component of the YUV data (step S7). Subsequently, the contrast component adjustment unit 109 acquires the degree of the processing target image, and acquires a contrast enhancement level proportional to the acquired degree (step S8). Further, the contrast component adjustment unit 109 adjusts the contrast component for removing wrinkles using the contrast enhancement level and the luminance component of the YUV data (step S9).

  Then, the luminance correction unit 110 performs luminance correction processing for adding the adjusted contrast component to the luminance component of the YUV data (step S10), and stores the YUV data after the luminance correction processing in the image buffer 101 (step S11). .

  In the gradation correction processing in the image processing unit 52 described above, the gradation correction unit 105 performs an image (RGB data) on which the basic gradation correction processing has been performed by gamma conversion with respect to FIG. ), The tone correction process specific to the image to be processed is performed using the second correction characteristic line, so that a good tone can be secured in the processed image.

  In addition, the second correction characteristic line does not simply widen the brightness distribution width in the processing target image, and as described above, the bright specific section X centered on the bright side brightness level (white side_lev). Is greater than the slope of a section darker than the specific section X and a section brighter than the specific section X. The specific section X is generally wrinkled in the photographed image, that is, since the difference in brightness level between pixels in the local region is small, a state in which the image is wrinkled is likely to occur (easily noticeable). It is an area.

  For this reason, in the tone correction processing for RGB data by the tone correction unit 105, wrinkles are generated in the processing target image by making the contrast of the brightness region where wrinkles easily occur higher than other brightness regions. When there is a portion, it is possible to effectively reduce the wrinkled state of the portion where wrinkles are generated or to remove the wrinkles.

  That is, for example, when gradation correction processing for RGB data is performed using a correction curve generated based on a luminance histogram of a processing target image, wrinkles are easily generated (conspicuous) in an image of a specific brightness region. If the occupied area is different, the gradation correction result for a specific brightness region is different accordingly. On the other hand, since the second correction characteristic line generated by the correction characteristic generation unit 104 is used in the gradation correction unit 105, gradation correction is always suitable for removing wrinkles for a specific brightness region. Processing can be performed.

  Therefore, in the above-described tone correction processing by the image processing unit 52, a good tone correction result can be obtained even for a portion where wrinkles occur in the processing target image.

  Further, in the above-described tone correction processing by the image processing unit 52, the brightness correction unit 110 adjusts the high frequency component of the processing target image to the YUV data after YUV conversion (Y′_high (x, y )) Is added, it is possible to secure a better gradation in the image after the gradation correction process for the following reason.

  That is, the pre-adjustment contrast component (Y_high (x, y)) that is the basis of the contrast component (Y′_high (x, y)) to be added to the YUV data is the luminance component (Y (x, y)) of the processing target image. ) Is a high-frequency component of the processing target image obtained by subtracting the global luminance component (Y_low (x, y)) from.

  Here, since the high-frequency component (Y (x, y) −Y_low (x, y)) is a positive value on the bright region side near the local light / dark boundary in the processing target image, the contrast before adjustment. The component (Y_high (x, y)) is a contrast component for highlight enhancement, and the adjusted contrast component (Y′_high (x, y)) is also a positive value (including “0”). Therefore, along with the luminance correction processing by the luminance correction unit 110, on the bright region side near the local light / dark boundary in the processing target image, the luminance of the pixels is higher than the original luminance by the luminance correction processing by the luminance correction unit 110. growing. That is, the pixels on the bright area side become brighter than before the luminance correction process.

  On the other hand, since the high frequency component (Y (x, y) −Y_low (x, y)) is a negative value on the dark region side near the local light / dark boundary in the processing target image, the contrast component before adjustment (Y_high (x, y)) is a contrast component for shadow enhancement, and the adjusted contrast component (Y′_high (x, y)) is a negative value. Therefore, along with the luminance correction processing by the luminance correction unit 110, on the dark region side near the local light / dark boundary in the processing target image, the luminance of the pixel is smaller than the original luminance along with the luminance correction processing by the luminance correction unit 110. Become. That is, the pixels on the dark area side are darker than before the luminance correction process.

  Therefore, in the image after the luminance correction processing by the luminance correction unit 110, the luminance difference (contrast) in the vicinity of the local light / dark boundary in the image is larger than the luminance difference (contrast) between light and dark before the luminance correction processing. (Contrast is emphasized). As a result, the gradation of details of the processing target image can be improved to a good state by the luminance correction processing in the luminance correction unit 110. At the same time, in the luminance correction process by the luminance correction unit 110, the gradation correction in the gradation correction unit 105 is performed by increasing the luminance difference (contrast) in the vicinity of the local light / dark boundary in the image as compared with that before the luminance correction process. It is possible to improve a state in which a wrinkle is generated in a processing target image that cannot be improved in the processing stage.

  In addition, as already described, the contrast component (Y′_high (x, y)) after the adjustment in the contrast component adjustment unit 109 is smaller as the degree (K) is smaller, and conversely, the degree (K) is larger. It gets bigger. In other words, in the luminance correction processing in the luminance correction unit 110, the local brightness / darkness in the image is adjusted by adjusting the bandwidth of the contrast component (Y_high (x, y)) before adjustment in accordance with the degree (K). The degree of contrast enhancement near the boundary is controlled in accordance with the degree (darkness) of wrinkles in the processing target image.

  Therefore, in the luminance correction processing by the luminance correction unit 110, the contrast near the local light / dark boundary in the image can be emphasized without excess or deficiency, and therefore remains after the gradation correction processing by the gradation correction unit 105. It is possible to reliably improve the stagnation state of the part where the crease is occurring.

  Further, in the luminance correction processing by the luminance correction unit 110, for example, the following effects are obtained as compared with the case where the degree of contrast enhancement in the vicinity of the local light / dark boundary in the image is not controlled according to the degree of haze (darkness). Is obtained. That is, when the degree of contrast enhancement is not controlled according to the degree (darkness) of the wrinkle, a state in which a portion where wrinkles remain after the gradation correction processing on the RGB data in the gradation correction unit 105 has occurred In order to improve the image quality without fail, it is necessary to set a greater degree of contrast enhancement, that is, the amount of addition of the contrast component to the luminance (Y) component. Therefore, when the degree of contrast enhancement is not controlled, a good tone correction result cannot be obtained by performing excessive contrast enhancement unnecessarily despite the low degree of wrinkles (thin wrinkles are thin) There is. However, in the luminance correction processing by the luminance correction unit 110, the degree of contrast enhancement is controlled according to the degree (darkness) of the wrinkles determined from the processing target image. The gradation of the details of the image can always be improved to a good state.

  Furthermore, as described above, the contrast component (Y′_high (x, y)) after adjustment in the contrast component adjustment unit 109 increases or decreases according to the size of the original luminance component (Y (x, y)). . That is, the adjusted contrast component (Y′_high (x, y)) is the original luminance component (Y (Y ()) when the wrinkle removal contrast component (Y_high (x, y)) is for highlight enhancement. x, y)) is larger and smaller when the wrinkle removal contrast component (Y_high (x, y)) is for shadow enhancement, the smaller the original luminance component (Y (x, y)) is. Get smaller.

  Therefore, in the luminance correction processing in the luminance correction unit 110, the degree of contrast enhancement near the local brightness boundary in the processing target image is controlled in accordance with the size of the original luminance component (Y (x, y)). As a result of the brightness correction process, “brightness” occurs on the bright region side near the local light / dark boundary in the processing target image, or near the local light / dark boundary in the processing target image accompanying the brightness correction process. It is possible to prevent the occurrence of “blackout” on the dark area side.

  Further, in the brightness correction processing in the brightness correction unit 110, the narrower the brightness distribution area in the processing target image is, the smaller the contrast component (Y_high ′ (x, y, x) after adjustment with respect to the original brightness component (Y (x, y))). Reduce the amount of addition in y)). Therefore, in the gradation correction processing by the image processing unit 52, the following effects can be obtained in addition to the effects described above.

  That is, in the tone correction processing for RGB data in the tone correction unit 105, when the brightness distribution area in the processing target image is narrow, the effect of removing wrinkles on the portion where wrinkles have occurred in the processing target image is large. Therefore, when the brightness distribution area in the processing target image is narrow, excessively adding the adjusted contrast component (Y_high ′ (x, y)) to the original luminance component (Y (x, y)), A good gradation correction result cannot be obtained for the image after the luminance correction processing. Therefore, as the brightness distribution area in the processing target image is narrower, the amount of addition of the adjusted contrast component (Y_high ′ (x, y)) to the original luminance component (Y (x, y)) is decreased. Thus, it is possible to always ensure a good gradation correction result in the image after the luminance correction processing.

  Here, in the present embodiment described above, the image processing unit 52 includes the above-described tone correction unit 105, and the tone correction unit 105 applies the RGB data to the RGB data prior to the luminance correction processing for the YUV data. The gradation correction processing is performed. However, the image processing unit 52 may be configured to perform only the luminance correction processing on the YUV data in the luminance correction unit 110 without the gradation correction unit 105. Even if the image processing unit 52 is configured to perform only the luminance correction processing on the YUV data in the luminance correction unit 110, the image after the gradation correction processing also has a good level even in a portion where wrinkles occur in the processing target image. The key correction result can be secured.

  Further, the configuration of the image processing unit 52 can be appropriately changed within the scope of the present invention. Therefore, for example, the image processing unit 52 may be configured to perform the same process as the brightness correction process for the YUV data described above on the RGB data instead of the YUV data. Further, the luminance correction processing for the RGB data described above may be performed on YUV data instead of RGB data.

  The ε filter 107 may be any filter that can extract the general luminance component (y_low (x, y)) from the luminance component (Y (x, y)), and can be changed to another smoothing filter. However, in order to obtain a high-frequency component in a good state from the processing target image, the smoothing filter in place of the ε filter 107 is a general luminance component (y_low (x, y)) that is likely to occur in a portion where the luminance difference is large in the image. It is preferable to employ an edge holding performance with little overshoot and undershoot. As a smoothing filter having edge retention performance, for example, another weight average filter such as a bilateral filter can be used.

  Further, when the ε filter 107 is used as in the present embodiment, the image processing unit 52 is an image having a luminance component (Y (x, y)) smaller than the image size of the processing target image, for example, before the ε filter 107. A reduction processing unit for converting the luminance component (Y (x, y)) into the size, and the overall luminance component (y_low (x, y)) after the ε filter 107 as the image size of the processing target image It is good also as a structure which provided the expansion part process part converted into (y_low (x, y)). If such a configuration is adopted in the image processing unit 52, the time required for the filter processing of the ε filter 103 can be shortened.

  In addition, the image processing unit 52 is provided with a low-pass filter for removing a noise component from the contrast component (Y_high (x, y)) for removing wrinkles, for example, subsequent to the contrast component generation unit 108 to remove the noise component. The contrast component for removing wrinkles later may be supplied to the contrast component adjustment unit 109.

  In the implementation of the present invention, for example, the contrast component adjustment unit 109 uses only the first adjustment coefficient composed of “(adj_lev−256) / 256” as the contrast component (Y_high (x, y)) for removing wrinkles. It is good also as a structure which acquires the value which multiplied by as a contrast component (Y_high '(x, y)) after adjustment.

  Further, the image processing unit 52 adds the contrast component (Y_high ′ (x, y)) after adjustment to the original luminance component (Y (x, y)) in the luminance correction unit 110. It is also possible to adopt a configuration in which the contrast component adjustment unit 109 realizes a function of adjusting the image according to the brightness distribution range of the processing target image.

  In other words, the contrast component adjustment unit 109 adds the above-described first adjustment coefficient and second adjustment coefficient or only the first adjustment coefficient to the wrinkle removal contrast component (Y_high (x, y)), and “( The contrast component (Y_high ′ (x, y)) after the adjustment can be obtained by the multiplication by the third adjustment coefficient consisting of “white_lev−black_lev) / 256”. When the configuration related to the contrast component adjustment unit 109 is adopted, the luminance correction unit 110 simply adds the adjusted contrast component (Y_high ′ (x, y)) to the original luminance component (Y (x, y)). It becomes only composition.

  Further, here, the digital camera 1 including the image processing apparatus of the present invention has been described as an embodiment of the present invention. However, the present invention is not limited to the digital camera 1 and has a configuration capable of recording moving images, for example. The present invention can also be applied to this imaging apparatus. In addition to the above-described CCD, the imaging apparatus to which the present invention can be applied is capable of capturing a moving image in addition to a digital camera equipped with, for example, a MOS (Complementary Meta10xide Semiconductor) type solid-state imaging device or a still image. Various imaging devices such as a digital camera capable of capturing a moving image and a digital video camera having a main function of capturing a moving image are included.

  In addition, the present invention is not limited to an imaging apparatus, and can be applied to any image processing apparatus as long as it has a configuration for performing image processing on an image stored as image data in an arbitrary storage medium. it can. The arbitrary image processing apparatus includes a printer that prints an image based on image data, for example.

  When the present invention is applied to an arbitrary image processing apparatus, the image processing unit 52 shown in FIG. 2 includes an ASIC (Application Specified Integrated Circuit), an arbitrary computer CPU, a memory, a program loaded in the memory, and the like. Can also be realized.

1 Digital camera 2 CCD
6 DSP
9 CPU
DESCRIPTION OF SYMBOLS 10 Flash memory 11 External memory 51 Image pick-up part 52 Image processing part 53 Control part 101 Image buffer 102 Gamma conversion part 103 Brightness distribution area acquisition part 104 Correction characteristic generation part 105 Gradation correction part 106 YUV conversion part 107 ε filter 108 Contrast component Generation unit 109 Contrast component adjustment unit 110 Brightness correction unit

Claims (9)

Extraction means for extracting high-frequency components from the processing target image;
Acquisition means for acquiring the degree information indicating the degree of wrinkles of the stagnation part in the processing target image;
Adjusting means for adjusting the high frequency component extracted by the extracting means according to the degree of wrinkle indicated by the degree information acquired by the acquiring means;
An image processing apparatus comprising: addition means for adding a high-frequency component adjusted by the adjustment means to a processing target image. The image according to claim 1, wherein the acquisition unit acquires the degree information based on brightness information indicating minimum and maximum brightness among the brightness of each pixel in the processing target image. Processing equipment. The minimum and maximum brightness respectively indicated by the brightness information are maximum and minimum pixel values among pixel values for each of a plurality of color components in each of all pixels of the processing target image. The image processing apparatus according to claim 2. The said acquisition means acquires the difference of the distribution width of the brightness in the process target image shown with the said brightness information, and the maximum width of the expressible brightness as said brightness degree information. Or the image processing apparatus of 3.   The image processing apparatus according to claim 1, wherein the adjustment of the high frequency component by the adjusting unit is adjustment of a bandwidth of the high frequency component. The adjustment unit adds the high-frequency component extracted by the extraction unit to each pixel according to the brightness of the corresponding pixel of the processing target image in addition to the degree of wrinkle indicated by the degree information acquired by the acquisition unit. The image processing apparatus according to claim 1, wherein the image processing apparatus is adjusted to the following. Generating means for generating a correction curve corresponding to the processing target image based on brightness information indicating the minimum and maximum brightness among the brightness of each pixel in the processing target image;
Correction means for correcting the gradation of the processing target image according to the correction curve generated by the generation means,
The extraction unit extracts a high frequency component from the processing target image after the gradation correction by the correction unit,
The acquisition means acquires the degree information indicating the degree of wrinkles of the stagnation part in the processing target image before gradation correction by the correction means,
The image processing apparatus according to claim 1, wherein the adding unit adds the high-frequency component adjusted by the adjusting unit to a processing target image after gradation correction by the correcting unit. The adjusting means adds the high-frequency component extracted by the extracting means to each pixel in the processing target image before gradation correction by the correcting means, in addition to the degree of wrinkle indicated by the degree information acquired by the acquiring means. The image processing apparatus according to claim 7, wherein the image processing apparatus is adjusted in accordance with a distribution width of brightness. On the computer,
A procedure for extracting high-frequency components from the processing target image;
A procedure for obtaining the degree information indicating the degree of wrinkles of the stagnation part in the processing target image;
A procedure for adjusting the high-frequency component extracted by the extracting means according to the degree of wrinkle indicated by the degree information acquired by the acquiring means;
And a step of adding a high-frequency component adjusted by the adjusting means to a processing target image.

Images