JP4111980B2 - Image processing apparatus and image processing method - Google Patents

Description

  The present invention relates to an image processing apparatus and an image processing method for appropriately improving the dynamic range of an input image.

  Conventionally, as a method for improving the gradation characteristics of an image, gradation conversion is performed so that the cumulative frequency (histogram) obtained by accumulating the number of pixels of the same luminance from an input image on one screen is evenly distributed. Histogram equalization is proposed (see, for example, Patent Document 1).

  Also, by calculating a weighted average of spatial luminance changes from the input image and calculating an improved luminance signal from the logarithmic conversion value of the calculated weighted average and the logarithmic conversion value of the input image, the dynamic image of the image is obtained. A method called RETINEX for improving the range has been proposed (see, for example, Non-Patent Document 1 and Patent Document 2).

JP 2002-27285 A (paragraphs 0029-0041, FIG. 1) Japanese Patent Laying-Open No. 2005-38119 (paragraphs 0028-0031, FIG. 1) Z. Rahman et al. "A Multiscale Retinex For Color Rendition and Dynamic Range Compression", XIX Proc. SPIE Vol. 2847, pp. 183-191, Nov. 1996

  However, in the above-described histogram equalization, histogram data of an image for at least one screen is recorded, histogram bias is analyzed from the recorded histogram data, and gradation characteristics are determined based on the analysis result. For this reason, a timing shift of one screen or more occurs between the image used for the analysis and the image reflecting the analysis result, so that the dynamic range of the input image may not be improved appropriately. For example, when there is a timing shift of one or more screens as described above in a moving image, the optimal level for the image reflecting the analysis result is different depending on the difference between the image used for the analysis and the image reflecting the analysis result. There was a problem that the tonal characteristics could not be determined.

  Further, in the method using RETINEX, calculation processing such as convolution integration for using a weighted average and logarithm calculation of the weighted average and the input signal is complicated, so that hardware (for example, an ASIC (Application Specific Integrated Circuit) or FPGA) is used. (Field Programmable Gate Array)) or RETINEX is executed by an embedded microcomputer, there is a problem that processing time becomes long and mounting capacity (gate scale, memory capacity) becomes large.

  Therefore, the present invention has been made to solve the above-described problems of the prior art, and an object of the present invention is to provide an image processing apparatus and image processing capable of appropriately improving the dynamic range of an input image with a simple configuration. It is to provide a method.

An image processing apparatus according to the present invention is an image processing apparatus that corrects an input image signal for each pixel and generates a corrected image signal, and the luminance of the pixels to be corrected and the peripheral pixels of the pixel to be corrected Filter means for obtaining and outputting a distribution; correction gain detecting means for obtaining a correction gain of a pixel to be corrected from the output of the filter means; and using the correction gain obtained by the correction gain detecting means, Arithmetic means for performing an operation for each pixel of the input image signal, and the output of the filter means is an average luminance of a pixel to be corrected and peripheral pixels of the pixel to be corrected, and the calculation by the calculating means a multiplier, the correction gain detecting means, so that a value obtained by multiplying the correction gain on the average luminance is monotonically increasing function, and wherein when the average luminance is large, the correction gain is small That a characteristic, wherein when the average brightness is small, so as to have a characteristic that the correction gain is increased, is characterized by calculating the correction gain.

The image processing method of the present invention is an image processing method for correcting an input image signal for each pixel to generate a corrected image signal, and includes a pixel to be corrected and a peripheral pixel of the pixel to be corrected. Obtaining a luminance distribution; obtaining a correction gain of a pixel to be corrected from the luminance distribution; and performing an operation for each pixel of the input image signal using the obtained correction gain. The luminance distribution in the step of obtaining the luminance distribution is an average luminance of pixels to be corrected and peripheral pixels of the pixel to be corrected, and the calculation in the step of performing the calculation is a multiplication There, the correction gain in the step of obtaining the correction gain, the so values above were multiplied by the correction gain average brightness is monotonically increasing function, and when the average luminance is greater Is a characteristic of the correction gain becomes smaller, wherein when the average brightness is small, so as to have a characteristic that the correction gain is increased, is characterized by calculating the correction gain.

  According to the present invention, the correction gain is obtained for each pixel based on the luminance distribution of the surrounding pixels of the pixel to be corrected, and the correction pixel is corrected using this correction gain. Can be improved.

  Further, according to the present invention, since it is not necessary to perform a complicated calculation and calculation and processing can be simplified, the configuration can be simplified, and as a result, the cost can be reduced.

Embodiment 1 FIG.
FIG. 1 is a block diagram schematically showing a configuration of an image processing apparatus 5 (an apparatus that performs an image processing method according to Embodiment 1) 5 according to Embodiment 1 of the present invention. As shown in FIG. 1, the image processing apparatus 5 according to the first embodiment includes a luminance detection unit 1, a filter unit 2, a correction gain detection unit 3, and a calculation unit 4. In the present application, the configuration called “... Means” may be configured by hardware such as an electric circuit, software operated by a program, or a combination of hardware and software. For example, the luminance detection means 1, the filter means 2, the correction gain detection means 3, and the calculation means 4 can be realized by software using a microcomputer (not shown).

The image processing apparatus 5 according to the first embodiment, calculates a correction gain G _k for each pixel based on an input image signal Xin, calculated correction gain G _k correction processing for each pixel in the input image signal Xin with To generate a corrected image signal Xout. In the correction processing of the image processing device 5, for example, the dynamic range of a captured image captured using a solid-state image sensor is improved. By this correction processing, it is possible to improve the contrast and the imaging performance by improving the contrast of the low-brightness area in the imaging screen, which has been easily crushed in the past.

  The input image signal Xin is, for example, an image signal of three colors of red (R), green (G), and blue (B) that is two-dimensionally arranged in horizontal 640 × vertical 480 pixels with 8-bit gradation (hereinafter, “RGB Signal "). The R signal level of the input image signal Xin is expressed as R (M, N), the G signal level of the input image signal Xin is expressed as G (M, N), and the B signal level of the input image signal Xin is expressed as B (M, N). . Here, M represents a horizontal pixel position, and N represents a vertical pixel position.

  The input image signal Xin is not limited to an RGB signal, and may be a YCbCr signal, an L * a * b * signal, or an HSV (Hue, Saturation, Value) signal. When the YCbCr signal, the L * a * b * signal, or the HSV signal is used as the input image signal Xin, the image processing apparatus 5 performs color conversion means (not shown) that performs color conversion processing on each color space signal to an RGB signal. ). The number of gradations of the input image signal Xin is not limited to the above 8 bits, and may be other gradations such as 10 bits or 12 bits used in the still image file. Further, the number of pixels of the input image signal Xin is not limited to the above value, and may be another number of pixels such as horizontal 1024 × vertical 960 pixels.

The luminance detecting means 1 obtains a luminance signal component from the input image signal Xin and outputs it. ITU-R BT. In the case of 709, the luminance signal Y can be obtained from the RGB signal by the following equation (1).
Y = 0.299 × R (M, N) + 0.587 × G (M, N) + 0.114 × B (M, N)
... (1)
The conversion formula for obtaining the luminance signal Y from the RGB signal is not limited to the above formula (1), but is defined by the color space standard adopted by the system that performs image processing. When the input image signal Xin includes the luminance signal Y, the luminance detection unit 1 outputs the luminance signal Y of the input image signal Xin to the filter unit 2 without performing calculation for obtaining the luminance signal Y. .

The filter unit 2 is a one-dimensional n-tap acyclic digital filter, and includes a delay unit 6, a coefficient unit 7, and an addition unit 8. The delay means 6 includes a delay element DL (−1) that delays the luminance signal of the input image signal Xin, a delay element DL (0) that delays an output of the delay element DL (−1), and a delay element DL (0). A delay element DL (1) for delaying the output is included. The coefficient means 7 is a multiplier that multiplies the output of the delay element DL (−1) by a coefficient a _k (−1), a multiplier that multiplies the output of the delay element DL (0) by a coefficient a _k (0), and A multiplier for multiplying the output of the delay element DL (1) by a coefficient a _k (1); The tap number n satisfies n = 2 × k + 1 (k is a positive integer).

The filter unit 2 performs a filtering process on the luminance signal output from the luminance detecting unit 1 and outputs a filtered signal after the filtering process. FIG. 1 shows a case where the number of taps n is 3 taps. The filter signal output from the filter unit 2 is, for example, the average luminance Yavg, and can be obtained by the following equation (2).

In equation (2), Y (−1), Y (0), and Y (1) are the luminance signal of the pixel one pixel after the pixel to be corrected, the luminance signal of the pixel to be corrected, and the correction, respectively. The luminance signal of the pixel immediately before the pixel to be performed is shown. Here, if the coefficient a _k (−1) = a _k (0) = a _k (1) = 1, the denominator of the equation (2) is as follows.

  Therefore, the average luminance of the pixel to be corrected and the peripheral pixels of the pixel to be corrected can be obtained by Expression (2). Note that the “peripheral pixels of the pixel to be corrected” refers to the pixel from the pixel before i pixel of the pixel to be corrected to the pixel one pixel before the pixel to be corrected when i is a predetermined integer. From the pixel one pixel after the pixel performing the pixel to the pixel after the i pixel of the pixel performing the correction. When the predetermined integer i is 1, the “peripheral pixels of the pixel to be corrected” are a pixel one pixel before the pixel to be corrected and a pixel one pixel after the pixel to be corrected.

  Thus, by using the configuration of the one-dimensional acyclic digital filter, the filter output in the one-dimensional direction of the luminance signal Y of the luminance detecting means 1 can be obtained. The obtained filter output is configured to obtain the luminance signal Y and the average value of the peripheral pixels of the pixel to be corrected, whereby the change in the brightness distribution in the one-dimensional direction can be obtained. Therefore, a correction gain corresponding to a change in brightness distribution can be detected, and a signal contrast can be corrected in consideration of a change in brightness distribution. In addition, the digital signal processing circuit has a general configuration, which can simplify the circuit scale and reduce the gate scale and reduce the cost.

  The number of taps n is not limited to 3 taps, and can be any number of taps. By increasing the number of taps, the characteristics of the cutoff frequency can be set finely, and a gradual change in luminance over a wide range can be detected. Thus, by switching the number of taps n, the optimum filter means 2 can be configured according to the change in the luminance distribution due to different illumination conditions in the input image.

  Although the case where the filter unit 2 is a one-dimensional acyclic digital filter has been described above, the filter unit 2 may be a two-dimensional acyclic digital filter. By using a two-dimensional acyclic digital filter, it is possible to detect a regional change in luminance of the input image.

  The filter unit 2 is not limited to a configuration that performs the process of calculating the average luminance Yavg based on the above formula (2), and may be any configuration that can determine a change in brightness distribution, and outputs a weighted average. Other configurations, such as a configuration using a low-pass filter, a configuration using a band-pass filter, or the like.

Correction gain detecting means 3, calculates and outputs a correction gain G _k based on the average luminance Yavg a filtered signal output from filter means 2. Details of the correction gain _Gk calculation process will be described later.

Calculating means 4, the correction gain G _k output from the correction gain detecting means 3 is input, the input image signal Xin to be multiplied by the correction gain G _k outputs.

FIG. 2 is a flowchart schematically showing the operation of the image processing apparatus 5 according to the first embodiment (image processing method according to the first embodiment). As shown in FIG. 2, the image processing apparatus 5 according to the first embodiment includes a correction process (step S2), which is a filter process (step S1), and a correction gain calculation process (pixel gain calculation process) for each pixel. ) And a calculation process (step S3) which is a process of correcting the input image signal using the correction gain. In the filter process (step S1), the filter processing unit 2 performs a filter process on the average luminance Yavg of the input image signal Xin, and outputs a luminance signal after the filter process. In the next correction gain detection process (step S2), and obtains the correction gain G _k for each pixel in accordance with the output from the filter processing unit 2. Next correction gain calculation processing in (step S3), and computing means 4 performs for each pixel the operation of the input image signal Xin using the correction gain G _k obtained by correcting the gain detection process (step S2), and the corrected An image signal Xout is output.

式（３）において、Ｙａｖｇは、フィルタ手段２から出力され補正利得検出手段３に入力される平均輝度を示し、Ｇｍａｘは、補正利得の最大値である最大利得を示し、Ｙｍａｘは、フィルタ手段２の最大出力である最大輝度を示す。 Hereinafter, the operations of the correction gain detection means 3 and the calculation means 4 will be described in detail. Correction gain detecting means 3, for example, obtains the correction gain G _k by the following equation (3).

In equation (3), Yavg represents the average luminance output from the filter unit 2 and input to the correction gain detection unit 3, Gmax represents the maximum gain that is the maximum value of the correction gain, and Ymax represents the filter unit 2 The maximum luminance that is the maximum output of.

  The maximum luminance Ymax is uniquely determined by the digital resolution (the number of gradations) of the image signal. For example, in the case of 8-bit gradation, the maximum luminance Ymax is 255, and in the case of 10-bit gradation, the maximum luminance Ymax is 1023.

  The maximum gain Gmax is a correction gain obtained statistically and experimentally in advance so that the dynamic range of the input image can be improved and the contrast can be improved. By performing the γ correction, the image processing has no sharpness and low contrast, but by using the correction gain, an image with a high contrast feeling is obtained, and the display quality is improved. The correction gain detection means 3 can be configured so that the maximum gain Gmax is a constant value regardless of the level of the input image, or the input image frequency, the frequency of the black level and the white level, the average video signal level, etc. The maximum gain Gmax may be switched according to information such as the brightness of the entire screen and the subject, contrast, luminance distribution, luminance histogram, and color distribution obtained from the image. When the maximum gain Gmax can be adjusted in this way, when used for a moving image, a correction gain optimum for the brightness distribution in the moving image can be obtained, and the image quality can be optimized.

Figure 3 is a graph showing a correction gain G _k output from the correction gain detecting means 3, FIG. 4 is a table showing a correction gain G _k output from the correction gain detecting means 3. 3, the horizontal axis represents the average brightness YAVG / Ymax standardized by the maximum brightness, the vertical axis represents the correction gain _{G k.}

As shown in FIGS. 3 and 4, when the maximum gain Gmax is greater than 1, as the average luminance Yavg / Ymax normalized by the maximum luminance increases (that is, as the average luminance Yavg increases), correction gain _{G k} decreases from a maximum gain Gmax, the average luminance YAVG is equal to the maximum luminance Ymax (i.e., at the YAVG / Ymax = 1), the correction gain _{G k} becomes 1 times. When the maximum gain Gmax is 1, the correction gain _Gk is 1 time.

Incidentally, as a correction gain detecting means 3 has described an arrangement for obtaining a correction gain G _k by performing the operation according to equation (3) holds the correction gain G _k corresponding to advance the average luminance Yavg as a look-up table It can also be left. When such a lookup table is used, it is not necessary to perform a division process, so that the calculation process in the correction gain detecting means 3 can be simplified.

FIG. 5 is a graph showing a value obtained by multiplying the correction gain used in the first embodiment by the average luminance normalized by the maximum luminance. In FIG. 5, the horizontal axis indicates the average luminance Yavg / Ymax normalized by the maximum luminance, and the vertical axis indicates the value G _{k obtained} by multiplying the average luminance Yavg / Ymax normalized by the maximum luminance by the correction gain G _k. X Indicates Yavg / Ymax.

Here, the correction gain G _k is a value obtained so that the correction luminance G _k × Yavg / Ymax is a monotonically increasing function. FIG. 5 shows the corrected luminance G _k × Yavg / Ymax when the maximum gain Gmax is 1, 3 or 5 times. As the correction luminance, a correction luminance based on a calculation other than G _k × Yavg / Ymax may be used. As can be seen from FIG. 5, when the maximum gain Gmax is 1, the input image signal Xin is output as it is. As can be seen from FIG. 5, as the maximum gain Gmax increases, the slope on the low brightness side increases and the slope on the high brightness side decreases. By increasing the inclination on the low luminance side, it is possible to amplify and output a low-frequency signal component that tends to be blacked out, and to improve the contrast of the low luminance portion. Further, the luminance signal and contrast signal on the high luminance side are maintained by reducing the inclination on the high luminance side to about 1.0 times that on the low luminance side. Accordingly, it is possible to prevent the problem that the high luminance side is crushed white, and it is possible to extract a high contrast signal even in a high luminance or low luminance signal, thereby improving visibility.

  FIG. 6 is a diagram illustrating an example of an image before correction processing by the image processing device 5 according to the first embodiment, and FIG. 7 is an example of an image after correction processing by the image processing device 5 according to the first embodiment. FIG.

  In FIG. 6, the bright area, that is, the scenery outside the window (the part indicated by the symbol DS1) is clearly reproduced, but the dark area, that is, the indoor person HD1, etc., is almost blacked out. It shows that.

  FIGS. 8A and 8B relate to processing of a bright region of a captured image (that is, the region DS1 where the outside of the room can be seen through the window in FIG. 6) in the image processing apparatus according to the first embodiment. FIG. 8A is a diagram illustrating the normalized luminance signal Xin / Ymax and the normalized average luminance Yavg / Ymax from the pixel position p0 to the pixel position p6, and FIG. 8B is a diagram illustrating FIG. FIG. 6 is a diagram illustrating a normalized luminance signal Xin / Ymax and a normalized output signal Xout / Ymax at the same pixel position p0 to pixel position p6.

  As indicated by a broken line in FIG. 8A, the standardized luminance signal Xin / Ymax of the input image signal is 0.6 at the pixel position p1, 0.7 at the pixel position p2, and 0. 0 at the pixel position p3. 8. 0.7 at pixel position p4, 0.8 at pixel position p5, and 0.6 at pixel position p6.

  Therefore, as indicated by a solid line in FIG. 8A, when the number of taps n is 3, the standardized average luminance Yavg / Ymax is 0.66 at the pixel position p1 and 0.70 at the pixel position p2. The pixel position p3 is 0.73, the pixel position p4 is 0.76, the pixel position p5 is 0.70, and the pixel position p6 is 0.70.

When the maximum gain Gmax is three times, than the average luminance Yavg and expressions determined (3), the correction gain _{G k} is 1.29 times the pixel position p1, 1.25 times the pixel position p2, the pixel position p3 1.22 times, 1.19 times at pixel position p4, 1.25 times at pixel position p5, and 1.25 times at pixel position p6. Thus, the correction gain Gk of each pixel can be _obtained by obtaining the average luminance Yavg of each pixel.

  FIG. 8B is a diagram illustrating the normalized luminance signal Xin / Ymax and the normalized output signal Xout / Ymax at the same pixel position p0 to pixel position P6 as in FIG. 8A. As shown by a broken line in FIG. 8B, the standardized luminance signal Xin / Ymax of the input image signal is 0.6 at the pixel position p1, 0.7 at the pixel position p2, and 0. 0 at the pixel position p3. 8. 0.7 at pixel position p4, 0.8 at pixel position p5, and 0.6 at pixel position p6.

Coordinates (M, N) of the pixels of the gain corrected output image signal Xout (M, N), using the coordinates (M, N) the input image signal Xin (M, N) of pixels with a gain _{G k,} the following It can be obtained by equation (4).
_{Xout (M, N) = G} k × Xin (M, N) ... (4)

  As shown by a solid line in FIG. 8B, the normalized output image signal Xout / Ymax after gain correction is 0.77 at the pixel position p1, 0.88 at the pixel position p2, and 0 at the pixel position p3. .98, 0.83 at the pixel position p4, 1.00 at the pixel position p5, and 0.75 at the pixel position p6.

In general, when the input image is an RGB signal, the following expressions (5a), (5b), and (5c) hold.
Rout (M, N) = G _k × Rin (M, N) (5a)
Gout (M, N) = G _k × Gin (M, N) (5b)
Bout (M, N) = G _k × Bin (M, N) (5c)
Here, Rout (M, N) is an output R signal obtained by correcting the gain of the pixel at coordinates (M, N), and Rin (M, N) is an input R signal of the pixel at coordinates (M, N). Yes, Gout (M, N) is an output G signal obtained by correcting the gain of a pixel at coordinates (M, N), and Gin (M, N) is an input G signal of a pixel at coordinates (M, N). , Bout (M, N) is an output B signal obtained by correcting the gain of the pixel at coordinates (M, N), and Bin (M, N) is an input B signal of the pixel at coordinates (M, N).

In general, when the input image is a YCbCr signal, the following equations (6a), (6b), and (6c) hold.
Yout (M, N) = G _k × Yin (M, N) (6a)
_{Cbout (M, N) = G} k × (Cbin (M, N) -Cbof) + Cbof
... (6b)
_{Crout (M, N) = G} k × (Crin (M, N) -Crof) + Crof
... (6c)
Here, Yout (M, N) is a luminance signal obtained by correcting the gain of a pixel at coordinates (M, N), and Yin (M, N) is an input luminance signal of a pixel at coordinates (M, N). , Cbout (M, N) and Crout (M, N) are color difference signals obtained by correcting the gain of the pixel at coordinates (M, N), and Cbin (M, N) and Crin (M, N) are represented by coordinates ( M, N) are input color difference signals of pixels, and Cbof and Crof are offset amounts when the color difference signals are processed.

Further, equation (5a), (5b), as shown in (5c), by multiplying the same correction gain _{G k} into RGB signals uniformly, white balance in the local area, without displacement, improve the dynamic range can do.

  FIGS. 9A and 9B relate to processing of a dark region (low luminance region) HD1 of a captured image in the image processing apparatus according to Embodiment 1, and FIG. FIG. 9B is a diagram illustrating a normalized luminance signal Xin / Ymax and a normalized average luminance Yavg / Ymax up to a position q6, and FIG. 9B is a diagram illustrating a pixel position q6 from the same pixel position q0 as FIG. 9A. It is a figure which shows the standardized brightness signal Xin / Ymax and the standardized output signal Xout / Ymax.

As shown by a broken line in FIG. 9A, the standardized luminance signal Xin / Ymax of the input image is 0.1 at the pixel position q1 in the low-luminance region HD1, 0.2 at the pixel position q2, The pixel position q3 is 0.3, the pixel position q4 is 0.2, the pixel position q5 is 0.3, and the pixel position q6 is 0.1. Further, as indicated by a solid line in FIG. 9A, the normalized average luminance Yavg / Ymax is 0.16 at the pixel position q1, 0.20 at the pixel position q2, 0.23 at the pixel position q3, The pixel position q4 is 0.26, the pixel position q5 is 0.20, and the pixel position q6 is 0.20. Further, the correction gain G _k is 2.25 times at the pixel position q1, 2.14 times at the pixel position q2, 2.05 times at the pixel position q3, 1.97 times at the pixel position q4, and 2.7 at the pixel position q5. 14 times and 2.14 times at pixel position q6.

  As shown by a solid line in FIG. 9B, assuming that the signal of the input image matches the luminance signal level, the gain-corrected output image signal Xout of each pixel is 0.23 at the pixel position q1. The pixel position q2 is 0.43, the pixel position q3 is 0.62, the pixel position q4 is 0.39, the pixel position q5 is 0.64, and the pixel position q6 is 0.21.

  As can be seen from FIG. 8B, in the bright region DS1, the correction gain is about 1.2 times, and an output image having the same signal level as the input image is output. Thereby, the contrast of the pixel unit of a bright area is maintained. On the other hand, as can be seen from FIG. 9B, the correction gain is approximately doubled in the dark region HD1. This indicates that the signal level compressed at the black level becomes brighter and the contrast of the pixel unit in the dark region is also amplified.

As described above, the average luminance calculated correction gain G _k for each pixel by YAVG, by a process of multiplying each pixel a correction gain G _k in the image signal subjected, dark regions HD1 is clearly (in FIG. 7 HD2), the bright area DS1 can improve the dynamic range as in the case of maintaining the same contrast (DS2 in FIG. 7).

  FIG. 10 is a diagram showing the occurrence frequency (frequency) at each luminance level of the input image as a histogram, and FIG. 11 is a diagram showing the occurrence frequency (frequency) at each luminance level of the improved image as a histogram. FIGS. 10 and 11 show the effects of dynamic range improvement of FIGS. 6, 7, 8A, 9B, 9A, and 9B using histograms.

  As shown in FIGS. 10 and 11, since the bright region (DS1 in FIG. 10) of the input image is a high luminance region, the captured image is distributed at a high luminance level. Also, since the dark area HD1 of the input image is a low luminance area, the captured image is distributed at a low luminance level.

  As shown in FIGS. 10 and 11, when the input image is corrected, the correction gain is small in the bright area DS1, so the signal indicated by the two-dot chain line distributed in the high luminance level of the area DS2 is Although the signal is indicated by a solid line in the region DS2, the change is small. On the other hand, the signal indicated by the two-dot chain line distributed in the low luminance level of the region HD1 becomes a signal indicated by the solid line of the region HD2, and changes greatly.

  This is because the correction using the expression (3) improves the contrast at low luminance and high luminance, eliminates the blackout on the low luminance side, and the low luminance signal. Indicates that an image having a sharper dynamic range than visibility can be obtained by moving to a higher luminance side. Further, by performing the correction, the average luminance is distributed near the center, and the display quality can be improved even in a display device having a narrow dynamic range (for example, a liquid crystal display).

  Further, according to the image processing apparatus 5 according to the first embodiment, the dynamic range of the pixel to be corrected is corrected based on the luminance distribution of the peripheral pixels of the pixel to be corrected, so that the reflection timing of the analysis result is made as short as possible. The dynamic range of the input image can be improved appropriately.

  Furthermore, according to the image processing apparatus 5 according to the first embodiment, it is possible to realize dynamic range expansion in units of pixels without using an external frame memory, and it is not necessary to perform complicated calculations, and calculation and processing are simplified. Therefore, it is possible to simplify the configuration and, as a result, reduce the cost.

Embodiment 2. FIG.
FIG. 12 is a block diagram schematically showing a configuration of an image processing apparatus 12 (an apparatus that performs the image processing method according to the second embodiment) 12 according to the second embodiment of the present invention. The image processing apparatus 12 according to the second embodiment includes a luminance detection unit 1, a filter unit 2, a correction gain detection unit 3, a multiplication unit 4, an offset detection unit 9, an offset subtraction unit 10, and an offset addition unit. 11. In FIG. 12, the same reference numerals are given to the same or corresponding components as those in FIG.

  The image processing apparatus 12 according to the second embodiment has means for adjusting the offset of the signal level of the input image. The offset is such that the imaging device (not shown) that captured the input image is backlit, the lens flare of the imaging device affects the image, or the signal level is black depending on the imaging environment and conditions of the subject. It is in the state.

The detailed correction operation when the offset adjustment is performed will be described below. The offset detection means 9 detects the minimum signal of the input image Xin, thereby obtaining an offset amount indicating the degree of black float of the input image. The offset amount Offset can be obtained by the following equation (7).
Offset = P × MIN (R, G, B) (7)
Here, MIN (R, G, B) indicates the minimum value of the input image RGB signal, and P is a positive real number that satisfies 0 ≦ P ≦ 1. The offset amount Offset is automatically detected by storing and storing the minimum value MIN (R, G, B) of the R, G, B signal of the image for one screen effective for correcting one frame or more before. Can be detected.

  Although FIG. 12 shows a configuration in the case where the offset amount Offset is automatically detected from the input image Xin by the offset detection means 9, a configuration in which the offset amount Offset is input from an external device may be employed. In the external device, the performance of the external device can be improved by improving the correction gain for the signal to be corrected. Specifically, the external device performs advanced image processing such as biometric authentication of fingerprints, veins, faces, etc., shape authentication, character recognition, etc., and detects feature points of the subject (face in the case of face authentication) And having a function of performing authentication based on the detection result of the feature points. In the external device, the signal of the feature point is emphasized by obtaining and setting the offset amount Offset from the result of detecting the region having the feature point and the signal level in the region from the image of the image processing device 5. Is possible. In addition, by increasing the signal level of the feature point, it is possible to improve performance such as detection accuracy and authentication rate of the external device.

The offset subtracting means 10 calculates the offset amount Offset obtained by the offset detecting means 9 from the input R signal Rin (M, N) at coordinates (M, N) and the input G signal Gin (M, N) at coordinates (M, N). ), Subtract from each of the input B signals Bin (M, N) of coordinates (M, N),
Rin (M, N) -Offset,
Gin (M, N) -Offset,
Bin (M, N) -Offset is output.

Calculating means 4, the signal obtained by subtracting the offset amount Offset from the offset subtraction means 10, by multiplying the correction gain G _k obtained by correcting the gain detecting means 3,
G _k × (Rin (M, N) −Offset),
G _k × (Gin (M, N) −Offset),
G _k × (Bin (M, N) −Offset) is output.

The offset addition means 11 receives the multiplication signal from the multiplication means 8, adds the same offset amount Offset as the offset subtraction means 10,
G _k × (Rin (M, N) −Offset) + Offset,
G _k × (Gin (M, N) −Offset) + Offset,
G _k × (Bin (M, N) −Offset) + Offset is output.

When the processing of the offset subtracting means 10, the calculating means 4, and the offset adding means 11 is summarized, the following expressions (8a), (8b), and (8c) are obtained.
_{Rout (M, N) = G} k × (Rin (M, N) -Offset) + Offset
... (8a)
_{Gout (M, N) = G} k × (Gin (M, N) -Offset) + Offset
... (8b)
_{Bout (M, N) = G} k × (Bin (M, N) -Offset) + Offset
... (8c)

If you do not offset correction, the correction gain G _k ends up amplifying offset Offset, it decreases the correction gain to the signal desired to improve the contrast, resulting in conversion to totally free signal contrast. By performing offset correction, it is possible to increase the correction gain for a signal whose contrast is to be improved, and to realize a process with high contrast.

Note that the offset amounts of the offset subtracting means 10 and the offset adding means 11 may be different values within a range satisfying 0 ≦ P ≦ 1. Particularly, by making the offset correction amount Offset1 of the offset adding means 11 smaller than the offset amount Offset of the offset subtracting means 10, there is an effect of reducing the black float. In other words, the signal after offset correction is prevented by the offset amount Offset1, and the black floating before the offset correction is corrected, and the image quality without sharpness and without black signals can be improved. That is, an image with a black level can be obtained. Processing for reducing the offset correction amount Offset1 of the offset adding means 11 as compared with the offset amount Offset of the offset subtracting means 10 is expressed by the following equations (9a), (9b), (9c), and (9d).
_{Rout (M, N) = G} k × (Rin (M, N) -Offset) + Offset1
... (9a)
_{Gout (M, N) = G} k × (Gin (M, N) -Offset) + Offset1
... (9b)
_{Bout (M, N) = G} k × (Bin (M, N) -Offset) + Offset1
... (9c)
Offset> Offset1 (9d)

  In the second embodiment, points other than those described above are the same as those in the first embodiment.

  According to the image processing device 12 according to the second embodiment, it is possible to detect the offset amount of the image, and by performing the offset correction based on the detected offset amount, the image quality by improving the contrast of the signal distributed in the low luminance. Can be improved.

  In the image processing apparatus according to the second embodiment, it is possible to switch between the subtraction amount of the offset amount before the correction gain processing and the addition amount of the offset amount after the correction gain processing, so that the black signal after the offset correction can be switched. The margin can be improved and the image quality can be improved.

ここで、Ｇｍｉｎは、入力画像の高輝度信号に乗算する最小利得を示している。 Embodiment 3 FIG.
In the image processing apparatus according to the third embodiment, the calculation formula for calculating the correction gain by the correction gain detecting means 3 is different from the formula (3) shown in the first embodiment. Specifically, the correction gain _{G k,} using the maximum gain Gmax, the minimum gain Gmin, the average luminance YAVG, the maximum luminance Ymax, obtained by the following equation (10).

Here, Gmin represents a minimum gain that is multiplied by the high luminance signal of the input image.

Figure 13 is a graph showing a correction gain G _k output from the correction gain detecting means 3 for calculation of the equation (10), FIG. 14, a correction gain used in the third embodiment, normalized by the maximum intensity It is a graph which shows the value which multiplied the average brightness which carried out. In FIG. 14, the horizontal axis indicates the average luminance Yavg / Ymax normalized by the maximum luminance, and the vertical axis indicates a value G _{k obtained} by multiplying the average luminance Yavg / Ymax normalized by the maximum luminance by the correction gain G _k. X Indicates Yavg / Ymax.

The correction gain in the third embodiment is that Gmin is the minimum value of the correction gain obtained by experimental and statistical processing, and even if Yavg / Ymax is larger than that in equation (1), the correction gain G _k has a value less than one. In the region where the average luminance is high, the signal level varies from pixel to pixel around the average luminance. The correction gain can be made smaller than the equation (3) with respect to the variation in signal for each pixel, and whiteout in the high-luminance portion can be prevented. It is possible to prevent the contrast signal in the high luminance portion from being lost due to white-out, and to compress and store the contrast signal with the correction gain Gmin.

  The correction gain characteristic is not limited to the characteristic shown in FIG. 14, and for example, a correction gain having the characteristic shown in FIG. 15 or FIG. 16 can be realized. By using such a correction gain, it is possible to improve the dynamic range optimum for the display device and the imaging device in consideration of the gradation characteristics of the display device (not shown) and the imaging device.

  Further, in the third embodiment, the points other than the above are the same as those in the first or second embodiment.

  According to the image processing device 5 or the image processing device 12 of the third embodiment, it is possible to maintain the contrast information of the high-intensity signal having a bright brightness distribution, and prevent whitening.

Embodiment 4 FIG.
In the image processing apparatus according to the fourth embodiment of the present invention, the contents of the filter processing by the filter means 2 are different from those in the first to third embodiments. In the fourth embodiment, the filter means includes, for example, an epsilon filter (ε filter) that is a nonlinear filter.

  FIG. 17 is a diagram showing an interval linear function when the nonlinear filter of the filter means according to the fourth embodiment is configured by an ε filter. FIG. 18 is a diagram showing the luminance signal level after correction in the comparative example using a linear filter as the filter means. FIG. 19 is a diagram showing the luminance signal level after correction when the nonlinear filter of the fourth embodiment is used.

  As shown in FIG. 18, in the case of the comparative example using a linear filter as the filter means, since the input image is smoothed, an area in which the area luminance changes rapidly (for example, luminance before correction in FIG. 18). If the level changes abruptly between pixel positions 5 and 6) in the input image, the average luminance Yavg is affected by the abrupt luminance change. Note that the average luminance (filter output) shown in FIG. 18 is a value obtained with the number of taps n = 5.

  At the pixel position 5, the average luminance is output at a higher level than the average luminance (filter output) from the pixel position 1 to the pixel position 4 due to the influence of the high luminance signal after the pixel position 6. It can be seen that as the average luminance increases, the correction gain decreases, and the signal level of the output image after correction at the pixel position 5 (thick solid line in FIG. 18) decreases.

  Further, at the pixel position 6, the average luminance is influenced by the luminance signal before the pixel position 5, and becomes a small value. For this reason, the correction gain is increased, and the corrected output level (thick solid line in FIG. 18) is a large value. In a region (edge region) where the luminance change between pixels is large, the correction gain is increased, so that the contrast can be enhanced by an aperture effect in which the signal at the edge portion is enhanced. Having an aperture effect means that image processing for the purpose of viewing photographs, displays, etc. and recognition devices that can perform recognition processing using images with emphasized edges emphasize the edges and make the image look sharp. An image with high sensitivity and high display quality can be generated.

  On the other hand, an authentication apparatus that detects a feature point from a slight contrast difference needs to output a faithful signal without applying edge enhancement in the preprocessing. Therefore, when there is a region where the luminance changes drastically as described above, it is necessary to consider the luminance change in the average luminance.

このとき、関数ｆ（ｘ）は、ｘを変数とする区間線形関数であり、次式（１３）で与えられる。ｙ（ｎ）は、Ｙ信号のεフィルタの出力の平均輝度を、ｘ（ｎ）は、ｎ画素の輝度を示している。式（１３）は、画素（ｎ）と、平均を求める画素間（±ｋ画素）の差分値を求める。差分値が、ε以内であれば平均値を求めるために差分値を用い、差分値がεを超える場合はα（ここでは、０）を用いる。このような処理を行うことで、急激な輝度変化を伴うエッジやノイズによる特異的に輝度が変化する場合など、輝度の急激な変化量に対して平均値が引きずられることが無く補正利得を求めることができる。

ここで、α＝０とした関数が、図１７に示される区間線形関数である。 Therefore, in the fourth embodiment, by providing the filter means 2 with a non-linear filter characteristic, it is possible to eliminate the above-described problem even when the change in luminance is severe. For example, a non-linear ε filter is used for the filter means 2. In general, the first-order ε filter is defined by the following equations (11) and (12).

At this time, the function f (x) is an interval linear function with x as a variable, and is given by the following equation (13). y (n) represents the average luminance of the output of the ε filter of the Y signal, and x (n) represents the luminance of n pixels. Equation (13) calculates a difference value between the pixel (n) and the average pixel (± k pixels) to be averaged. If the difference value is within ε, the difference value is used to obtain an average value, and if the difference value exceeds ε, α (here, 0) is used. By performing such processing, the correction gain is obtained without the average value being dragged with respect to the amount of sudden change in luminance, such as when the luminance changes specifically due to edges or noise accompanied by a sudden change in luminance. be able to.

Here, the function with α = 0 is the interval linear function shown in FIG.

  FIG. 19 shows the average luminance when a nonlinear ε filter is used as the filter means 2. Also at the pixel position 5 and the pixel position 6, the change (trend) in luminance of the input image is reproduced. Also, the signal level of the output image obtained by calculating the correction gain can reproduce the change (trend) in the luminance of the input image, unlike the characteristics shown in FIG.

  As described above, by using the ε filter, it is possible to realize an optimum dynamic range conversion process while retaining edge information even in an image in which an abrupt luminance change exists in an input image.

  The ε filter can be composed of a comparator and a line memory, and has effects such as a reduction in hardware mounting area and a reduction in software processing time.

  In addition, the configuration using a non-linear filter such as a median filter or a stack filter may be used as long as it is a means capable of detecting the luminance level of the region without being limited to the ε filter.

  Further, in the fourth embodiment, points other than the above are the same as those in any of the first to third embodiments.

  According to the signal processing device 5 and the signal processing device 12 of the fourth embodiment, the output of the filter means can hold the edge information, and the Mach effect in the case of a sudden luminance change can be suppressed.

1 is a block diagram schematically showing a configuration of an image processing apparatus according to Embodiment 1 of the present invention (an apparatus that performs an image processing method according to Embodiment 1). FIG. 4 is a flowchart schematically showing an operation of the image processing apparatus according to the first embodiment (an image processing method according to the first embodiment). 3 is a graph showing a correction gain used in the image processing apparatus according to the first embodiment. 6 is a table showing correction gains used in the image processing apparatus according to Embodiment 1; 6 is a graph showing a value obtained by multiplying a normalized average luminance by a correction gain used in the image processing apparatus according to the first embodiment. 6 is a diagram illustrating an example of an image before correction processing by the image processing apparatus according to Embodiment 1. FIG. 6 is a diagram illustrating an example of an image after correction processing by the image processing apparatus according to Embodiment 1. FIG. (A) shows the normalized luminance (broken line) and the normalized average luminance (solid line) in the high luminance region shown in FIG. 6, and (b) shows the high luminance region shown in FIG. It is a figure which shows the standardized brightness | luminance (broken line) and the standardized output brightness | luminance (solid line) in FIG. (A) shows the normalized luminance (dashed line) and the normalized average luminance (solid line) in the low luminance region shown in FIG. 6, and (b) shows the low luminance region shown in FIG. It is a figure which shows the standardized brightness | luminance (broken line) and the standardized output brightness | luminance (solid line) in FIG. It is a figure which shows the generation frequency (frequency) of each brightness | luminance of an input image with a histogram. It is a figure which shows the generation frequency (frequency) of each brightness | luminance of the improved image with a histogram. It is a block diagram which shows roughly the structure of the image processing apparatus (apparatus which performs the image processing method which concerns on Embodiment 2) which concerns on Embodiment 2 of this invention. It is a graph which shows the correction gain used in the image processing apparatus which concerns on Embodiment 3 of this invention. 10 is a graph illustrating a value obtained by multiplying a normalized average luminance by a correction gain used in the image processing apparatus according to the third embodiment. 10 is a graph showing a value obtained by multiplying a normalized average luminance by a correction gain used in another image processing apparatus according to the third embodiment. 10 is a graph showing a value obtained by multiplying a normalized average luminance by a correction gain used in another image processing apparatus according to the third embodiment. It is a figure which shows an interval linear function in case the filter means of the image processing apparatus which concerns on Embodiment 4 of this invention is comprised with an epsilon filter. It is a figure which shows the luminance signal level after correction | amendment of the image processing apparatus of the comparative example using a linear filter as a filter means. It is a figure which shows the luminance signal level after correction | amendment at the time of using a nonlinear filter as a filter means of the image processing apparatus which concerns on Embodiment 4. FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Luminance detection means, 2 Filter means, 3 Correction gain detection means, 4 Calculation means, 5,12 Image processing means, 6 Delay means, 7 Coefficient means, 8 Addition means, 9 Offset detection means, 10 Offset subtraction means, 11 Offset Addition means.

Claims (5)

In an image processing apparatus that corrects an input image signal for each pixel and generates a corrected image signal,
Filter means for obtaining and outputting the luminance distribution of pixels around the pixel to be corrected and the pixels to be corrected;
Correction gain detection means for obtaining a correction gain of the pixel to be corrected from the output of the filter means;
Using the correction gain obtained by the correction gain detection means, and calculating means for calculating for each pixel of the input image signal,
The output of the filter means is the average luminance of the pixels to be corrected and the peripheral pixels of the pixels to be corrected,
The calculation by the calculation means is multiplication,
The correction gain detecting means has a characteristic that the value obtained by multiplying the average luminance by the correction gain is a monotonically increasing function, and the correction gain decreases when the average luminance is large, the average gain The image processing apparatus according to claim 1, wherein the correction gain is calculated so that the correction gain is increased when luminance is low . The output of the filter means is Yavg,
The maximum value Ymax of the output of the filter means,
The correction gain is G _k ,
When the maximum value of the correction gain is Gmax,

The image processing apparatus according to claim 1, wherein the relationship is satisfied. The output of the filter means is Yavg,
The maximum value Ymax of the output of the filter means,
The correction gain is G _k ,
The maximum value of the correction gain is Gmax,
When the minimum value of the correction gain is Gmin,

The image processing apparatus according to claim 1, wherein the relationship is satisfied. Offset detecting means for detecting a first offset amount indicating the degree of black float of the input image from the input image signal;
Offset subtracting means for subtracting the first offset amount from the input image signal;
Offset addition means for adding a second offset amount equal to or smaller than the first offset amount to the output of the computing means;
Wherein by calculating means calculating the image processing apparatus according to any one of claims 1 to 3, characterized in that to be made to the output of said offset subtraction means. In an image processing method for correcting an input image signal for each pixel and generating a corrected image signal,
Obtaining a luminance distribution of pixels to be corrected and peripheral pixels of the pixels to be corrected;
Obtaining a correction gain of a pixel to be corrected from the luminance distribution;
Performing a calculation for each pixel of the input image signal using the determined correction gain, and
The luminance distribution in the step of obtaining the luminance distribution is an average luminance of a pixel to be corrected and peripheral pixels of the pixel to be corrected,
The operation in the step of performing the operation is multiplication,
The correction gain in the step of obtaining the correction gain is:
The value obtained by multiplying the average luminance by the correction gain is a monotonically increasing function, and when the average luminance is large, the correction gain is reduced.When the average luminance is small, An image processing method, wherein the correction gain is calculated so as to have a characteristic of increasing the correction gain .

Images