JP7262032B2 - Image processing device - Google Patents

Description

The present invention relates to an image processing apparatus, and more particularly, to improvement of an image processing apparatus that converts luminance information of an input image to generate an output image.

As a method of correcting the contrast of an image, there is a tone curve adjustment method of converting luminance information of each pixel using a tone curve. A tone curve represents a correspondence relationship that associates an input value of luminance with an output value, and the degree of correction changes by adjusting the shape of the tone curve. For example, if the contrast is increased, the difference between light and dark becomes large, and a halftone image can be sharpened, but a low gradation image and a high gradation image become unclear. Also, even in a halftone image, if the contrast is made too high, gradation changes in bright and dark areas are lost. Thus, it is not easy to obtain a tone curve suitable for image sharpening.

On the other hand, a histogram flattening method that determines a tone curve from a histogram that represents the luminance distribution of an image is conventionally known. In the histogram equalization method, an image is analyzed to obtain a histogram consisting of the number of pixels for each luminance, and a tone curve is determined from the cumulative frequency of the histogram. According to this histogram equalization method, an image with a wide dynamic range can be obtained for any image.

However, in the image processing as described above, when an image has an area with relatively many low-luminance pixels and an area with many high-luminance pixels, it is possible to sharpen these areas at the same time. I had a difficult problem. For example, a sufficiently clear image cannot be obtained for an image of a subject with a large difference in brightness or an image of a backlight.

The present invention has been made in view of the above circumstances, and image processing capable of obtaining a sufficiently clear image even when a single image includes a low-luminance region and a high-luminance region. The purpose is to provide an apparatus. In particular, it is an object of the present invention to provide an image processing apparatus capable of simultaneously sharpening an area with many low-luminance pixels and an area with many high-luminance pixels.

An image processing apparatus according to a first aspect of the present invention comprises area dividing means for dividing an input image into two or more processing areas, and pixel number distribution for each luminance class based on luminance information of pixels in the processing areas. histogram generating means for generating a histogram representing the histogram, integrated histogram calculating means for obtaining an integrated histogram by accumulating the number of pixels of the histogram in order from the minimum luminance class, and based on the total number of pixels in the processing area, the maximum luminance class normalizing means for normalizing the integrated histogram so that the integrated value matches the upper limit value of luminance; and converting the luminance information of the input image for each processing region based on the normalized integrated histogram, luminance conversion means for generating an output image.

According to such a configuration, the input image is divided into a plurality of processing regions, and the luminance information of the input image is converted for each processing region using the normalized integrated histogram. It is possible to sharpen an area with many high-brightness pixels at the same time.

In the image processing device according to the second aspect of the present invention, in addition to the above configuration, the brightness class consists of n consecutive brightness values (n is an integer equal to or greater than 2), and each brightness value in the brightness class has The histogram generation means divides the luminance range into two or more luminance classes to obtain the histogram, and the luminance conversion means obtains the The luminance information of the input image is converted using the integrated histogram linearly interpolated by the frequency interpolation means as a tone curve for each processing area.

According to such a configuration, since the luminance class has a range and the frequency corresponding to each luminance value in the luminance class is obtained by linear interpolation, excessive correction can be suppressed. For example, when pixels are concentrated at a specific luminance value and the frequency is prominent, it is possible to prevent the luminance change from being overemphasized.

An image processing apparatus according to a third aspect of the present invention, in addition to the above configuration, is a rectangular area including a pixel of interest and having the centers of the above four processing areas adjacent to each other in the horizontal direction and the vertical direction as vertices. is an interpolation area, further comprising weight calculation means for obtaining different two-dimensional weights according to the horizontal and vertical positions of the target pixel in the interpolation area, wherein the luminance conversion means converts the two-dimensional weights to The tone curve of the pixel of interest is obtained by taking a weighted average of the tone curves of the four processing regions using the above-mentioned four processing regions, and the luminance information is converted.

According to such a configuration, the pixels of interest in the interpolation region are weighted according to the position of the pixels of interest. Therefore, the boundaries of the processing regions are conspicuous due to the difference in tone curves between adjacent processing regions. can be suppressed.

In the image processing apparatus according to the fourth aspect of the present invention, in addition to the above configuration, the area dividing means inputs a rectangular area consisting of 2 to the power of m (m is an integer equal to or greater than 1) pixels as the processing area. configured to segment an image; According to such a configuration, division processing for normalizing the integrated histogram can be performed by bit shifting.

According to the present invention, it is possible to sharpen an area with many low-luminance pixels and an area with many high-luminance pixels at the same time. Therefore, even when a single image includes a low-luminance area and a high-luminance area, a sufficiently clear image can be obtained.

1 is a block diagram showing one configuration example of an image processing apparatus 1 according to an embodiment of the present invention; FIG. FIG. 2 is an explanatory diagram schematically showing an example of the operation during contrast correction in the image processing apparatus 1 of FIG. 1, showing an input image 2 and a histogram; FIG. 2 is an explanatory diagram schematically showing an example of the operation during contrast correction in the image processing apparatus 1 of FIG. 1, showing an integrated histogram; FIG. 4 is a diagram showing an example of a tone curve obtained by linear interpolation of integrated histograms; FIG. 2 is an explanatory view schematically showing an example of the operation during contrast correction in the image processing apparatus 1 of FIG. 1, showing an interpolation area 5 including a pixel of interest 4; FIG. 10 is a diagram showing an example of an output image obtained by contrast correction processing; 2 is a flow chart showing an example of operations during contrast correction in the image processing apparatus 1 of FIG. 1;

<Image processing device 1>
FIG. 1 is a block diagram showing one configuration example of an image processing apparatus 1 according to an embodiment of the present invention. This image processing apparatus 1 is an apparatus for correcting the contrast of an input image and generating an output image, and includes an area division section 11, a histogram generation section 12, an integrated histogram calculation section 13, a normalization section 14, a frequency interpolation section 15, It is composed of a tone curve storage unit 16 , a luminance conversion unit 17 and a weight calculation unit 18 .

The image processing apparatus 1 can be used, for example, in a system for monitoring passing vehicles such as a toll gate or a system for monitoring a production line of industrial products. In a monitoring system for passing vehicles, an image captured by a monitoring camera is input as an input image, and the output image is analyzed to identify the vehicle type and vehicle number.

The region division unit 11 divides the input image into two or more processing regions, and outputs luminance information of pixels to the histogram generation unit 12 and the luminance conversion unit 17 for each processing region. The input image is, for example, an image having W pixels in the horizontal direction, H pixels in the vertical direction, and Na=W×H pixels in total, and consists of luminance information for each pixel.

The processing area is a processing unit of the contrast correction process, and consists of a rectangular area having the number of pixels in the horizontal direction of w1, the number of pixels in the vertical direction of h1, and the total number of pixels of n1=w1×h1. The processing area consists of, for example, 2 to the mth power (m is an integer equal to or greater than 1) pixels, where n1= ^2m .

The histogram generation unit 12 generates a histogram representing the pixel number distribution for each luminance class based on the luminance information of the pixels within the processing area. A luminance class is a unit of processing when obtaining a luminance distribution, and consists of n consecutive luminance values (n is an integer equal to or greater than 2). The brightness class is common to each processing area.

When the luminance information consists of luminance values of 8 bits (256 gradations), the class width n is, for example, n=8 and is specified in advance. At this time, the number of classes is 256/8=32. The histogram generator 12 divides the luminance range into 32 luminance classes to obtain histograms.

Since the luminance value is an integer from 0 to 255, the histogram is created by counting the number of pixels whose luminance value belongs to the luminance class for each luminance class. Note that, for the class width n, 2 ⁴ =16 or 2 ⁵ =32, for example, may be used. Moreover, the configuration may be such that the user is allowed to specify an arbitrary class width n.

The integrated histogram calculation unit 13 obtains an integrated histogram by integrating the number of pixels of the histogram in order from the minimum luminance class. The integrated histogram is based on the histogram generated by the histogram generator 12, and by adding the number of pixels in a certain luminance class or the integrated value to the number of pixels in the next luminance class, the integrated value of the luminance class is obtained.

The normalization unit 14 normalizes the integrated histogram based on the total number of pixels n1 in the processing area so that the integrated value of the maximum brightness class matches the upper limit of brightness. The integrated value of the maximum luminance class in the integrated histogram obtained by the integrated histogram calculator 13 matches the total number of pixels n1. When the luminance range is from 0 to 255, the upper limit of luminance is 255. Therefore, the normalization process is performed by multiplying each integrated value of the integrated histogram by 255/n1.

The frequency interpolation unit 15 obtains the frequency corresponding to each luminance value within the luminance class by linear interpolation of the integrated histogram. The frequency interpolation unit 15 uses the integrated histogram normalized by the normalization unit 14 as a base, and performs a weighted average of the frequency of the luminance class of interest and the frequency of the previous luminance class to obtain The frequency corresponding to the luminance value of is obtained.

For example, the frequency of the minimum luminance class (luminance values 0 to 7) is associated with the first luminance value 8 in the second luminance class from the minimum luminance class (luminance values 8 to 15), and the second luminance from the minimum luminance class The frequency of the class (luminance values 8 to 15) is associated with the first luminance value 16 in the third minimum luminance class (luminance values 16 to 23). A brightness value of 0 is associated with a frequency of 0.

At this time, if the frequency of the (k−1)-th (k=2 to 32) luminance class from the minimum luminance class is f _k−1 and the frequency of the k-th luminance class is f _k , then the k-th luminance The frequency f(x) of the x-th (x=1 to n) luminance value in the class is expressed by the following equation (1).

The integrated histogram normalized by the normalization unit 14 and linearly interpolated by the frequency interpolation unit 15 is stored in the tone curve storage unit 16 as a tone curve for correcting the contrast. Instead of normalizing the integrated histogram and then obtaining the frequency of each luminance value in the luminance class by linear interpolation, the frequency of each luminance value in the luminance class is obtained by linear interpolation, and then the integrated histogram is normalized. The configuration may be such that

The luminance conversion unit 17 converts the luminance information of the input image for each processing area based on the integrated histogram normalized by the normalization unit 14, and generates an output image. The luminance conversion unit 17 converts the luminance information of the input image using the integrated histogram linearly interpolated by the frequency interpolation unit 15 as a tone curve for each processing area.

In order to weight the tone curves of four processing regions adjacent to each other in the horizontal direction and the vertical direction, the weight calculation unit 18 calculates the horizontal and vertical positions of the pixel of interest in the interpolation region for the pixel of interest in the interpolation region. Different two-dimensional weights are obtained according to . The interpolation area is a rectangular area including the pixel of interest, and has vertices at the centers of four processing areas adjacent to each other in the horizontal and vertical directions.

The luminance conversion unit 17 uses the two-dimensional weight obtained by the weight calculation unit 18 to perform a weighted average of the tone curves of the four processing regions adjacent to each other, thereby obtaining the tone curve of the pixel of interest and converting the luminance information. do.

FIG. 2 is an explanatory diagram schematically showing an example of the operation during contrast correction in the image processing apparatus 1 of FIG. 1, showing an input image 2 and a histogram. In the figure, (a) shows an input image 2 divided into a large number of processing areas 3, and (b) shows a histogram created from luminance information of pixels in the processing areas 3. .

The input image 2 is, for example, an image in which the number of pixels W in the horizontal direction is W=1920 and the number of pixels H in the vertical direction is H=1080. On the other hand, the processing area 3 is, for example, a rectangular area in which the number of pixels w1 in the horizontal direction is w1=128, the number of pixels h1 in the vertical direction is h1=8, and the total number of pixels n1 is 2 to the 10th power.

In this way, by dividing the input image 2 into a rectangular area consisting of 2 m (where m is an integer equal to or greater than 1) pixels as the processing area 3, division processing for normalizing the integrated histogram is performed by bit shifting. It can be carried out. This input image 2 is evenly divided into a number of processing areas 3 .

A histogram is created by dividing the luminance range into a plurality of luminance classes and counting the number of pixels in each luminance class. In this example, the luminance range is evenly divided into seven luminance classes, and the number of pixels is obtained for each.

When the class width n is 8 and the number of classes is 32, for example, luminance values of 0 to 7 belong to the minimum luminance class, and luminance values of 8 to 15 belong to the next luminance class. . Also, luminance values of 248 to 255 belong to the maximum luminance class.

FIG. 3 is an explanatory diagram schematically showing an example of the operation during contrast correction in the image processing apparatus 1 of FIG. 1, showing an integrated histogram. (a) in the figure shows an integrated histogram obtained by integrating the number of pixels of the histogram in order from the minimum luminance class. The number of pixels in the minimum luminance class of the original histogram is associated with the minimum luminance class of the integrated histogram.

A value obtained by adding the number of pixels in the minimum luminance class to the number of pixels in the next luminance class in the original histogram is associated with the luminance class next to the minimum luminance class as an integrated value. Similarly, the third luminance class from the minimum luminance class is associated as an integrated value by adding the number of pixels in the second luminance class from the minimum luminance class to the number of pixels in the third luminance class in the original histogram. be done.

(b) in the figure shows an integrated histogram normalized so that the integrated value of the maximum brightness class matches the upper limit value of the brightness. Since the integrated value of the maximum brightness class matches the total number of pixels n1 in the processing area 3, a normalized integrated histogram is obtained by multiplying the integrated value of each brightness class by 255/n1.

FIG. 4 is a diagram showing an example of a tone curve obtained by linear interpolation of integrated histograms. In the drawing, (a) shows the tone curve of the processing area 3 with many high-luminance pixels, and (b) shows the tone curve of the processing area 3 with many low-luminance pixels. The tone curve maps luminance input values to output values.

The tone curve created from the histogram of the processing area 3, which has many high-brightness pixels, is a downward convex curve with respect to a straight line passing through the origin and having a slope of 1. While the contrast is reduced on the low-brightness side, Contrast is expanded on the high brightness side.

The tone curve created from the histogram of the processing area 3, which has many low-brightness pixels, is an upward convex curve with respect to a straight line passing through the origin and having a slope of 1. While the contrast is expanded on the low-brightness side, Contrast is reduced on the high brightness side.

In this way, since the tone curve is automatically adjusted according to the luminance distribution of the processing area 3, an image in which both an area with many low-luminance pixels and an area with many high-luminance pixels exist, such as an image shot against backlight. However, these areas can be sharpened simultaneously.

FIG. 5 is an explanatory diagram schematically showing an example of operations during contrast correction in the image processing apparatus 1 of FIG. The pixel of interest 4 is any pixel in the input image 2 .

The interpolation area 5 is a processing unit when interpolating the tone curves between the adjacent processing areas 3 in order to prevent the boundaries of the processing areas 3 from becoming conspicuous due to the difference in tone curves between the processing areas 3. be. This interpolation area 5 is formed of a rectangular area whose vertices are the centers of four processing areas 3a to 3d adjacent to each other in the horizontal and vertical directions among the processing areas 3 in the vicinity of the target pixel 4. FIG.

The position of the pixel of interest 4 in the interpolation area 5 is defined by the center of the processing area 3a as the origin, the vertical straight line passing through the origin and the center of the processing area 3c as the s-axis, and the horizontal straight line passing through the origin and the center of the processing area 3b. It can be expressed using coordinates (s, t) with a straight line in the direction as the t-axis (0≤s≤1, 0≤t≤1).

A two-dimensional weight w(s, t) for interpolating the tone curve is expressed using s and t. The frequency g(x) of the target pixel 4 is f _i,j (x), f _{i, j+1} (x), f _{i+1, j} (x), f Letting _{i+1, j+1} (x) be obtained by forming a bilinear curved surface from these tone curves, it is expressed by the following equation (2).

In the above equation (2), each coefficient of frequencies f _i,j (x), f _i,j+1 (x), f _i+1,j (x), f _i+1,j+1 (x) is a two-dimensional weight w(s , t). If the input image 2 is an image of 1920×1080 pixels and the processing area 3 is an area of 128×8 pixels, the position of the processing area 3 can be expressed using a matrix of 135 rows and 15 columns ( 1≤i≤135, 1≤j≤15).

Since the target pixel 4 in the interpolation region 5 is weighted according to the position coordinates (s, t) of the target pixel 4, the tone curves of the adjacent processing regions 3 are different. It is possible to suppress conspicuous borders.

FIG. 6 is a diagram showing an example of an output image obtained by contrast correction processing. In the figure, (a) shows the input image, (b) shows the output image when the class width n is n=1, and (c) shows the class width n=n= 8, the output image is shown. In the figure, an image of a vehicle in motion is shown.

This input image has low brightness as a whole and is darkened, making it difficult to make out the state of the subject. On the other hand, the output images of (b) and (c) are sharpened by the contrast correction processing, and the state of the subject can be easily confirmed.

In particular, in the output image of (c), the sharp changes in the histogram are smoothed out by linear interpolation due to the width of the luminance class, so excessive correction is reduced and the image has natural gradation changes. is obtained. On the other hand, in the output image of (b), due to excessive correction, the coarseness of image quality due to the influence of camera white noise or the like is emphasized.

Steps S101 to S111 in FIG. 7 are a flow chart showing an example of operations during contrast correction in the image processing apparatus 1 in FIG. First, the image processing apparatus 1 divides the input image 2 into a large number of processing areas 3 (step S101), and based on the luminance information of the pixels in the processing areas 3, generates a histogram consisting of the number of pixels for each luminance class. (Step S102).

Next, the image processing apparatus 1 obtains an integrated histogram by accumulating the number of pixels in the histogram in order from the minimum luminance class (step S103), and normalizes the integrated histogram based on the total number of pixels n1 in the processing area 3. (Step S104). Then, the image processing apparatus 1 generates a tone curve by obtaining the frequency corresponding to each luminance value in the luminance class by linear interpolation of the integrated histogram (step S105).

The image processing apparatus 1 repeats the processing procedure from step S102 to step S105 until tone curves are obtained for all processing regions 3 in the input image 2 (step S106).

Next, the image processing apparatus 1 assigns different two-dimensional weights w(s, t) to the pixel of interest 4 in the interpolation region 5 according to the horizontal and vertical positions of the pixel of interest 4 in the interpolation region 5. (step S107), and the frequencies f _i,j (x), f _i,j+1 (x), f _i+1,j (x), f _i+1,j+1 (x) of the tone curves of the four processing regions 3a to 3d are By weighted averaging, the tone curve is synthesized to obtain the frequency g(x) of the target pixel 4 (step S108).

The image processing apparatus 1 generates an output image by converting the luminance information of the target pixel 4 in the input image 2 using the frequency g(x) (step S109). The image processing apparatus 1 repeats the processing procedure from step S107 to step S109 until the frequency g(x) is obtained for all pixels in the interpolation area 5 (step S110). Further, the image processing apparatus 1 repeats the processing procedure from step S107 to step S110 until the frequency g(x) is obtained for all interpolation regions 5 in the input image 2 (step S111).

According to the present embodiment, the input image 2 is divided into a plurality of processing regions 3, and the luminance information of the input image 2 is converted for each processing region 3 using the normalized integrated histogram. It is possible to sharpen an area with many pixels and an area with many high-brightness pixels at the same time. Moreover, since the luminance class has a width and the frequency corresponding to each luminance value in the luminance class is obtained by linear interpolation, excessive correction can be suppressed.

Furthermore, since the pixel of interest 4 in the interpolation region 5 is weighted according to the position coordinates (s, t) of the pixel of interest 4, the tone curves of adjacent processing regions 3 are different. 3 can be suppressed from being conspicuous. In addition, since the input image is divided into a rectangular area consisting of 2 m pixels as the processing area 3, division processing for normalizing the integrated histogram can be performed by bit shifting.

In the present embodiment, an example of performing interpolation processing between the processing regions 3 using the two-dimensional weight w(s, t) after normalizing the integrated histogram has been described. It is not limited to this. For example, when weighted averaging the frequencies of the four processing regions 3 using the two-dimensional weight w(s, t), the processing for normalizing the frequencies may be collectively performed.

1 image processing device 11 area division unit 12 histogram generation unit 13 integrated histogram calculation unit 14 normalization unit 15 frequency interpolation unit 16 tone curve storage unit 17 luminance conversion unit 18 weight calculation unit 2 input image 3 processing region 4 target pixel 5 interpolation region

Claims (4)

an area dividing means for dividing an input image into two or more processing areas;
Histogram generating means for generating a histogram representing pixel number distribution for each luminance class based on the luminance information of pixels in the processing area;
integrated histogram calculation means for calculating an integrated histogram by integrating the number of pixels of the histogram in order from the minimum luminance class ;
A rectangular area including a pixel of interest and having vertexes at the centers of the four processing areas adjacent to each other in the horizontal direction and the vertical direction is defined as an interpolation area, and the pixel of interest in the horizontal direction and the interpolation area in the interpolation area is defined as an interpolation area. weight calculation means for obtaining different two-dimensional weights according to vertical positions;
Using the integrated histogram as the tone curve for each of the processing regions, and weighted averaging the tone curves of the four processing regions using the two-dimensional weight, when obtaining the tone curve of the target pixel, the frequency is normalized, and the tone curve of the pixel of interest is used to convert the luminance information to generate an output image . The luminance class consists of n consecutive luminance values (where n is an integer of 2 or more),
further comprising frequency interpolation means for obtaining a frequency corresponding to each luminance value in the luminance class by linear interpolation of the integrated histogram;
The histogram generating means obtains the histogram by dividing the luminance range into two or more luminance classes,
2. The image processing apparatus according to claim 1, wherein said luminance conversion means uses said integrated histogram linearly interpolated by said frequency interpolation means as a tone curve for each processing area. an area dividing means for dividing an input image into two or more processing areas;
Histogram generating means for generating a histogram representing pixel number distribution for each luminance class based on the luminance information of all pixels in the processing area;
integrated histogram calculation means for calculating an integrated histogram by integrating the number of pixels of the histogram in order from the minimum luminance class;
a frequency interpolation means for obtaining a frequency corresponding to each luminance value in the luminance class by linear interpolation of the integrated histogram;
normalization means for normalizing the integrated histogram linearly interpolated by the frequency interpolation means so that the integrated value of the maximum brightness class matches the upper limit value of brightness based on the total number of pixels in the processing area;
brightness conversion means for converting the brightness information of the input image for each processing area based on the normalized integrated histogram to generate an output image;
The luminance class consists of n consecutive luminance values (where n is an integer of 2 or more),
The histogram generating means obtains the histogram by dividing the luminance range into two or more luminance classes,
The luminance conversion means converts the luminance information of the input image using the integrated histogram linearly interpolated by the frequency interpolation means and normalized by the normalization means as a tone curve for each processing area. image processing device. 4. The area dividing means according to claim 1, wherein the input image is divided into rectangular areas composed of 2@m pixels (m is an integer equal to or greater than 1) as the processing areas. 10. The image processing device according to claim 1.

Images