JP2021177423A - Image processing apparatus - Google Patents

Abstract

To provide an image processing apparatus that obtains a sufficiently clear image even when a low luminance area and a high luminance area are present in one image.SOLUTION: An image processing apparatus 1 includes: an area dividing unit 11 that divides an input image 2 into two or more processing areas; a histogram creation unit 12 that creates a histogram representing the distribution of the number of pixels for every luminance class based on information on the luminance of pixels in the processing area 3; an integrated histogram calculation unit 13 that integrates the number of pixels of the histogram in order from the minimum luminance class to determine an integrated histogram; a normalization unit 14 that, based on the total number of pixels in the processing area 3, normalizes the integrated histogram so that the integrated value of the number of pixels in the maximum luminance class matches the upper limit of luminance, and a luminance conversion unit 17 that converts information on the luminance of the input image for every processing area 3 based on the normalized integrated histogram to create an output image.SELECTED DRAWING: Figure 1

Description

The present invention relates to an image processing device, and more particularly to an improvement of an image processing device that converts luminance information of an input image to generate an output image.

As a method of correcting the contrast of an image, there is a tone curve adjustment method of converting the luminance information of each pixel using a tone curve. The tone curve represents a correspondence relationship in which the input value of the luminance is associated with the output value, and the degree of correction changes by adjusting the shape of the tone curve. For example, when the contrast is increased, the difference between light and dark becomes large, and a halftone image can be sharpened, but a low gradation image or a high gradation image becomes unclear. Further, even in a halftone image, if the contrast is set too high, the gradation change in the bright part and the dark part is lost. As described above, it is not easy to obtain a tone curve suitable for sharpening an image.

On the other hand, a method of flattening a histogram in which a tone curve is determined from a histogram representing the brightness distribution of an image has been conventionally known. In the histogram flattening method, the image is analyzed to obtain a histogram consisting of the number of pixels for each brightness, and the tone curve is determined from the cumulative frequency of the histogram. According to this histogram flattening method, it is possible to obtain an image having a wide dynamic range regardless of the image.

However, in the image processing as described above, when there is a region having a relatively large number of low-luminance pixels and a region having a large number of high-luminance pixels in one image, these regions can be sharpened at the same time. There was a problem that it was difficult. For example, it was not possible to obtain a sufficiently clear image with respect to an image of a subject having a large difference in brightness or an image taken against the sun.

The present invention has been made in view of the above circumstances, and image processing capable of obtaining a sufficiently clear image even when a low-luminance region and a high-luminance region exist in one image. The purpose is to provide the device. In particular, it is an object of the present invention to provide an image processing apparatus capable of simultaneously sharpening a region having many low-luminance pixels and a region having many high-luminance pixels.

The image processing apparatus according to the first aspect of the present invention determines the distribution of the number of pixels for each luminance class based on the region dividing means for dividing the input image into two or more processing regions and the luminance information of the pixels in the processing region. A histogram generating means for generating a histogram to be represented, an integrated histogram calculation means for calculating an integrated histogram by integrating the number of pixels of the histogram in order from the minimum brightness class, and a maximum brightness class based on the total number of pixels in the processing area. Based on the normalization means for normalizing the integrated histogram so that the integrated value matches the upper limit of the brightness and the normalized integrated histogram, the brightness information of the input image is converted for each processing area. It is provided with a luminance conversion means for generating an output image.

According to such a configuration, the input image is divided into a plurality of processing areas, and the brightness information of the input image is converted for each processing area using the normalized integrated histogram. Areas with many high-brightness pixels can be sharpened at the same time.

In the image processing apparatus according to the second aspect of the present invention, in addition to the above configuration, the above-mentioned luminance class consists of n consecutive luminance values (n is an integer of 2 or more), and each luminance value in the above-mentioned luminance class is included. A frequency interpolation means for obtaining the corresponding frequency by linear interpolation of the integrated histogram is further provided, the histogram generating means divides the brightness range into two or more brightness classes to obtain the histogram, and the brightness conversion means obtains the histogram. It is configured to convert the luminance information of the input image by using the integrated histogram linearly interpolated by the frequency interpolation means as a tone curve for each processing area.

According to such a configuration, since the luminance class has a width and the frequency corresponding to each luminance value in the luminance class is obtained by linear interpolation, excessive correction can be suppressed. For example, when the pixels are concentrated on a specific luminance value and the dioptric power is prominent, it is possible to suppress the change in luminance from being overemphasized.

In addition to the above configuration, the image processing apparatus according to the third aspect of the present invention is a rectangular region including a pixel of interest, and has a rectangular region whose apex is the center of four processing regions adjacent to each other in the horizontal and vertical directions. Is further provided as a weight calculation means for obtaining different two-dimensional weights according to the horizontal and vertical positions of the pixel of interest in the interpolation area, and the brightness conversion means obtains the two-dimensional weights. By weight-averaging the tone curves of the four processing regions using the above, the tone curve of the pixel of interest is obtained and the brightness information is converted.

According to such a configuration, since the attention pixel in the interpolation area is weighted according to the position of the attention pixel, the boundary of the processing area is conspicuous due to the difference in the tone curve between the adjacent processing areas. Can be suppressed.

In the image processing apparatus according to the fourth aspect of the present invention, in addition to the above configuration, the region dividing means inputs a rectangular region composed of 2 to the mth power (m is an integer of 1 or more) as the processing region. It is configured to divide the image. According to such a configuration, the division process for normalizing the integrated histogram can be performed by bit shift.

According to the present invention, it is possible to simultaneously sharpen a region having many low-luminance pixels and a region having many high-luminance pixels. Therefore, even when a low-luminance region and a high-luminance region exist in one image, a sufficiently clear image can be obtained.

It is a block diagram which showed one structural example of the image processing apparatus 1 by embodiment of this invention. It is explanatory drawing which shows typically an example of the operation at the time of contrast correction in the image processing apparatus 1 of FIG. 1, and the input image 2 and a histogram are shown. It is explanatory drawing which shows typically an example of the operation at the time of contrast correction in the image processing apparatus 1 of FIG. 1, and shows the integration histogram. It is a figure which showed an example of the tone curve obtained by the linear interpolation of the integration histogram. It is explanatory drawing which shows typically an example of the operation at the time of contrast correction in the image processing apparatus 1 of FIG. 1, and shows the interpolation area 5 including the attention pixel 4. It is a figure which showed an example of the output image obtained by the contrast correction processing. It is a flowchart which showed an example of the operation at the time of contrast correction in the image processing apparatus 1 of FIG.

<Image processing device 1>
FIG. 1 is a block diagram showing a configuration example of an image processing device 1 according to an embodiment of the present invention. The image processing device 1 is a device that corrects the contrast of the input image to generate an output image, and includes a region division unit 11, a histogram generation unit 12, an integrated histogram calculation unit 13, a normalization unit 14, and a frequency interpolation unit 15. It is composed of a tone curve storage unit 16, a brightness conversion unit 17, and a weight calculation unit 18.

The image processing device 1 can be used, for example, in a system for monitoring a passing vehicle such as a tollhouse gate or a system for monitoring a production line of an industrial product. In the passing vehicle monitoring system, an image taken by a surveillance camera is input as an input image, and the output image is analyzed to identify the vehicle type and vehicle number.

The area division unit 11 divides the input image into two or more processing areas, and outputs pixel brightness information to the histogram generation unit 12 and the brightness conversion unit 17 for each processing area. The input image is, for example, an image in which the number of pixels in the horizontal direction is W, the number of pixels in the vertical direction is H, and the total number of pixels is Na = W × H, and is composed of luminance information for each pixel.

The processing area is a processing unit of the contrast correction processing, and is composed of a rectangular area in which the number of pixels in the horizontal direction is w1, the number of pixels in the vertical direction is h1, and the total number of pixels is n1 = w1 × h1. The processing area is composed of, for example, 2 to the mth power (m is an integer of 1 or more), and n1 = 2 ^m .

The histogram generation unit 12 generates a histogram showing the distribution of the number of pixels for each brightness class based on the brightness information of the pixels in the processing area. The luminance class is a processing unit for obtaining the luminance distribution, and consists of n consecutive luminance values (n is an integer of 2 or more). The brightness class is common to each processing area.

When the luminance information consists of an 8-bit (256 gradations) luminance value, the class width n is, for example, n = 8, and is specified in advance. At this time, the number of classes is 256/8 = 32. The histogram generation unit 12 divides the luminance range into 32 luminance classes and obtains a histogram.

Since the luminance value is an integer of 0 or more and 255 or less, the histogram is created by counting the number of pixels whose luminance value belongs to the luminance class for each luminance class. For the class width n, for example, ²⁴ = 16 or ²⁵ = 32 may be used. Further, the configuration may be such that the user can specify an arbitrary class width n.

The integrated histogram calculation unit 13 integrates the number of pixels of the histogram in order from the minimum luminance class to obtain an integrated histogram. The integrated histogram is based on the histogram generated by the histogram generation unit 12, and the integrated value of the luminance class is obtained by adding the number of pixels or the integrated value of a certain luminance class to the number of pixels of the next luminance class.

The normalization unit 14 normalizes the integrated histogram so that the integrated value of the maximum luminance class matches the upper limit of the luminance based on the total number of pixels n1 in the processing area. The integrated value of the maximum luminance class in the integrated histogram obtained by the integrated histogram calculation unit 13 matches the total number of pixels n1. When the brightness range is 0 or more and 255 or less, the upper limit of the brightness is 255. Therefore, the normalization process is performed by multiplying each integrated value of the integrated histogram by 255 / n1.

The frequency interpolation unit 15 obtains the frequency corresponding to each luminance value in the luminance class by linear interpolation of the integrated histogram. The frequency interpolation unit 15 is based on the integrated histogram normalized by the normalization unit 14, and by weighted averaging the frequency of the luminance class of interest and the frequency of the previous luminance class, the frequency within the luminance class of interest is obtained. The frequency corresponding to the brightness value of is obtained.

For example, the frequency of the minimum luminance class (luminance value 0 to 7) corresponds to the first luminance value 8 in the second luminance class (luminance value 8 to 15) from the minimum luminance class, and the second luminance from the minimum luminance class. The frequency of the class (luminance values 8 to 15) corresponds to the first luminance value 16 in the third luminance class (luminance values 16 to 23) from the minimum luminance class. Further, the brightness value 0 is associated with the frequency 0.

At this time, if the frequency of the (k-1) th (k = 2-32) luminance class from the minimum luminance class is f _k-1 and the frequency of the _kth luminance class is f k, the k-th luminance is concerned. The frequency f (x) of the xth (x = 1 to n) luminance value in the class is expressed by the following equation (1).

The tone curve storage unit 16 holds an integrated histogram that has been normalized by the normalization unit 14 and then linearly interpolated by the frequency interpolation unit 15 as a tone curve for correcting contrast. Instead of normalizing the integrated histogram and then linearly interpolating to obtain the frequency of each luminance value in the luminance class, linear interpolation is used to determine the frequency of each luminance value in the luminance class, and then the integrated histogram is normalized. It may be configured to do so.

The luminance conversion unit 17 converts the luminance information of the input image for each processing area based on the integrated histogram normalized by the normalization unit 14, and generates an output image. The luminance conversion unit 17 converts the luminance information of the input image by using the integrated histogram linearly interpolated by the frequency interpolation unit 15 as a tone curve for each processing area.

In order to weight the tone curves of the four processing regions adjacent to each other in the horizontal and vertical directions, the weight calculation unit 18 positions the pixels of interest in the interpolation region in the horizontal and vertical directions in the interpolation region of the pixels of interest. Two-dimensional weights that differ according to the above are obtained. The interpolation area is a rectangular area including the pixel of interest, and the apex is the center of four processing areas adjacent to each other in the horizontal direction and the vertical direction.

The luminance conversion unit 17 obtains the tone curve of the pixel of interest by weighted averaging the tone curves of the four processing regions adjacent to each other using the two-dimensional weight obtained by the weight calculation unit 18, and converts the luminance information. do.

FIG. 2 is an explanatory diagram schematically showing an example of an operation at the time of contrast correction in the image processing apparatus 1 of FIG. 1, and an input image 2 and a histogram are shown. In the figure, (a) shows an input image 2 divided into a large number of processing areas 3, and (b) shows a histogram created from the luminance information of the pixels in the processing area 3. ..

The input image 2 is, for example, an image in which the number of pixels W in the horizontal direction is W = 1920 and the number of pixels H in the vertical direction is H = 1080. On the other hand, the processing area 3 is, for example, a rectangular area in which the number of pixels w1 in the horizontal direction is w1 = 128, the number of pixels h1 in the vertical direction is h1 = 8, and the total number of pixels n1 is 2.

By dividing the input image 2 with the rectangular area consisting of 2 m-th power (m is an integer of 1 or more) as the processing area 3 in this way, the division process for normalizing the integrated histogram is performed by bit shift. It can be carried out. The input image 2 is evenly divided into a large number of processing areas 3.

The histogram is created by dividing the luminance range into a plurality of luminance classes and counting the number of pixels for each luminance class. In this example, the luminance range is evenly divided into seven luminance classes, and the number of pixels is calculated for each.

When the class width n is n = 8 and the number of classes is 32, for example, the minimum luminance class belongs to the luminance value = 0 to 7, and the next luminance class belongs to the luminance value = 8 to 15. .. Further, the luminance value = 248 to 255 belongs to the maximum luminance class.

FIG. 3 is an explanatory diagram schematically showing an example of an operation at the time of contrast correction in the image processing apparatus 1 of FIG. 1, and an integrated histogram is shown. In (a) in the figure, an integrated histogram in which the number of pixels of the histogram is integrated in order from the minimum luminance class is shown. The minimum luminance class of the integrated histogram is associated with the number of pixels of the minimum luminance class of the original histogram.

The value obtained by adding the number of pixels of the minimum luminance class to the number of pixels of the next luminance class in the original histogram is associated with the next luminance class of the minimum luminance class as an integrated value. Similarly, for the third luminance class from the lowest luminance class, the value obtained by adding the number of pixels of the second luminance class from the lowest luminance class to the number of pixels of the third luminance class in the original histogram is associated as an integrated value. Be done.

In (b) in the figure, an integrated histogram is shown in which the integrated value of the maximum luminance class is normalized so as to match the upper limit of the luminance. Since the integrated value of the maximum luminance class matches the total number of pixels n1 in the processing area 3, a normalized integrated histogram can be obtained by multiplying the integrated value of each luminance class by 255 / n1.

FIG. 4 is a diagram showing an example of a tone curve obtained by linear interpolation of an integrated histogram. In the figure, (a) shows the tone curve of the processing region 3 having many high-luminance pixels, and (b) shows the tone curve of the processing region 3 having many low-luminance pixels. The tone curve associates the luminance input value with the output value.

The tone curve created from the histogram of the processing area 3 having many high-luminance pixels is a curve that is convex downward with respect to a straight line having a slope of 1 through the origin, and the contrast is reduced on the low-luminance side. The contrast is expanded on the high brightness side.

The tone curve created from the histogram of the processing area 3 having many low-luminance pixels is a curve that is convex upward with respect to a straight line having a slope of 1 through the origin, and the contrast is expanded on the low-luminance side. Contrast is reduced on the high brightness side.

Since the tone curve is automatically adjusted according to the brightness distribution of the processing area 3 in this way, an image in which both a region having many low-luminance pixels and a region having many high-luminance pixels exist, such as an image taken against the sun. Even so, those areas can be sharpened at the same time.

FIG. 5 is an explanatory diagram schematically showing an example of an operation at the time of contrast correction in the image processing apparatus 1 of FIG. 1, and shows an interpolation region 5 including a pixel of interest 4. The pixel of interest 4 is an arbitrary pixel in the input image 2.

The interpolation area 5 is a processing unit when the tone curve is interpolated between adjacent processing areas 3 in order to prevent the boundary of the processing area 3 from being conspicuous due to the difference in tone curves between the processing areas 3. be. The interpolation region 5 is composed of a rectangular region having the center of four processing regions 3a to 3d adjacent to each other in the horizontal direction and the vertical direction among the processing regions 3 in the vicinity of the pixel of interest 4.

The position of the pixel of interest 4 in the interpolation region 5 is such that the center of the processing region 3a is the origin, the vertical straight line passing through the origin and the center of the processing region 3c is the s-axis, and the horizontal straight line passing through the origin and the center of the processing region 3b. It can be expressed using coordinates (s, t) with the straight line in the direction as the t-axis (0 ≦ s ≦ 1, 0 ≦ t ≦ 1).

The two-dimensional weights w (s, t) for interpolating the tone curve are represented using s and t. The power g (x) of the pixel of interest is the power of the tone curve obtained from the processing areas 3a to 3d, respectively, of fi _{, j} (x), fi _{, j + 1} (x), fi _{+ 1, j} (x), f. _{If i + 1, j + 1} (x), it is obtained by forming a bilinear curved surface from these tone curves, and is expressed by the following equation (2).

In the above equation (2), _{each coefficient of the frequencies fi, j} (x), fi _{, j + 1} (x), fi _{+ 1, j} (x), fi _{+ 1, j + 1} (x) has a two-dimensional weight w (s). , T). Further, if the input image 2 is an image of 1920 × 1080 pixels and the processing area 3 is an area of 128 × 8 pixels, the position of the processing area 3 can be represented by using a matrix of 135 rows and 15 columns ( 1 ≦ i ≦ 135, 1 ≦ j ≦ 15).

Since the attention pixel 4 in the interpolation area 5 is weighted according to the position coordinates (s, t) of the attention pixel 4, the tone curve of the processing area 3 is different due to the difference in tone curve between the adjacent processing areas 3. It is possible to prevent the boundary from becoming conspicuous.

FIG. 6 is a diagram showing an example of an output image obtained by the contrast correction process. In the figure, (a) shows an input image, (b) shows an output image when the class width n is n = 1, and (c) shows an output image when the class width n is n = 1. The output image in the case of 8 is shown. In the figure, an image taken with a moving vehicle as a subject is shown.

This input image has low brightness as a whole, is darkened, and the appearance of the subject is unclear. On the other hand, the output images of (b) and (c) are sharpened by the contrast correction process, and the state of the subject can be easily confirmed.

In particular, in the output image of (c), since the sudden change in the histogram is smoothed by linear interpolation by giving a width to the luminance class, excessive correction is reduced and the image has a natural gradation change. Has been obtained. On the other hand, in the output image of (b), the roughness of the image quality due to the influence of the white noise of the camera is emphasized by the excessive correction.

Steps S101 to S111 of FIG. 7 are flowcharts showing an example of the operation at the time of contrast correction in the image processing apparatus 1 of FIG. First, the image processing device 1 divides the input image 2 into a large number of processing areas 3 (step S101), and generates a histogram composed of the number of pixels for each brightness class based on the brightness information of the pixels in the processing area 3. (Step S102).

Next, the image processing apparatus 1 integrates the number of pixels of the histogram in order from the minimum luminance class to obtain an integrated histogram (step S103), and normalizes the integrated histogram based on the total number of pixels n1 in the processing area 3. (Step S104). Then, the image processing device 1 generates a tone curve by obtaining the frequency corresponding to each luminance value in the luminance class by linear interpolation of the integrated histogram (step S105).

The image processing device 1 repeats the processing procedure from step S102 to step S105 until a tone curve is obtained for all the processing areas 3 in the input image 2 (step S106).

Next, the image processing device 1 assigns a two-dimensional weight w (s, t) that differs depending on the horizontal and vertical positions of the pixel of interest 4 in the interpolation region 5 of the pixel of interest 5 with respect to the pixel of interest 4 in the interpolation region 5. _{Obtaining (step S107) Finding the frequencies fi, j} (x), fi _{, j + 1} (x), fi _{+ 1, j} (x), fi _{+ 1, j + 1} (x) of the tone curves of the four processing areas 3a to 3d. By weighted average, tone curves are combined to obtain the frequency g (x) of the pixel of interest 4 (step S108).

The image processing device 1 generates an output image by converting the luminance information of the pixel of interest 4 in the input image 2 using the power g (x) (step S109). The image processing device 1 repeats the processing procedure from step S107 to step S109 until the power g (x) is obtained for all the pixels in the interpolation region 5 (step S110). Further, the image processing device 1 repeats the processing procedure from step S107 to step S110 until the frequency g (x) is obtained for all the interpolation regions 5 in the input image 2 (step S111).

According to the present embodiment, since the input image 2 is divided into a plurality of processing areas 3 and the brightness information of the input image 2 is converted for each processing area 3 using the normalized integrated histogram, low-luminance pixels. It is possible to sharpen an area with a large number of pixels and an area with a large number of high-brightness pixels at the same time. Further, since the luminance class has a width and the frequency corresponding to each luminance value in the luminance class is obtained by linear interpolation, excessive correction can be suppressed.

Further, since the attention pixel 4 in the interpolation area 5 is weighted according to the position coordinates (s, t) of the attention pixel 4, the processing area is caused by the difference in tone curve between the adjacent processing areas 3. It is possible to suppress the boundary of 3 from being conspicuous. Further, since the input image is divided by using the rectangular region composed of 2 m-th power pixels as the processing region 3, the division process for normalizing the integrated histogram can be performed by bit shift.

In the present embodiment, an example in which interpolation processing between processing regions 3 is performed using two-dimensional weights w (s, t) after normalizing the integrated histogram has been described. However, the present invention describes the present invention. It is not limited to this. For example, when the frequencies of the four processing regions 3 are weighted and averaged using the two-dimensional weights w (s, t), the processing for normalizing the frequencies may be performed collectively.

1 Image processing device 11 Area division unit 12 Histogram generation unit 13 Integrated histogram calculation unit 14 Normalization unit 15 Frequency interpolation unit 16 Tone curve storage unit 17 Luminance conversion unit 18 Weight calculation unit 2 Input image 3 Processing area 4 Attention pixel 5 Interpolation area

Claims (4)

An area dividing means for dividing an input image into two or more processing areas,
A histogram generation means that generates a histogram showing the distribution of the number of pixels for each luminance class based on the brightness information of the pixels in the processing area.
An integrated histogram calculation means for obtaining an integrated histogram by integrating the number of pixels of the above histogram in order from the minimum brightness class.
A luminance conversion means that converts the luminance information of the input image for each processing region and generates an output image based on the integrated histogram.
A rectangular region including the pixel of interest, the rectangular region having the center of the four processing regions adjacent to each other in the horizontal and vertical directions as vertices is used as the interpolation region, and the pixel of interest is included in the interpolation region in the horizontal and vertical directions. It is equipped with a weight calculation means for obtaining different two-dimensional weights according to the position in the direction.
The luminance conversion means obtains the tone curve of the pixel of interest by using the integrated histogram as a tone curve for each processing region and weighting averaging the tone curves of the four processing regions using the two-dimensional weights. , An image processing device characterized in that the frequency is normalized when converting luminance information. The above-mentioned luminance class consists of n consecutive luminance values (n is an integer of 2 or more).
Further provided with frequency interpolation means for obtaining the frequency corresponding to each luminance value in the luminance class by linear interpolation of the integrated histogram.
The histogram generating means divides the luminance range into two or more luminance classes and obtains the histogram.
The image processing apparatus according to claim 1, wherein the luminance conversion means uses the integrated histogram linearly interpolated by the power interpolation means as a tone curve for each processing region. An area dividing means for dividing an input image into two or more processing areas,
A histogram generation means that generates a histogram showing the distribution of the number of pixels for each luminance class based on the brightness information of the pixels in the processing area.
An integrated histogram calculation means for obtaining an integrated histogram by integrating the number of pixels of the above histogram in order from the minimum brightness class.
A frequency interpolation means for obtaining the frequency corresponding to each luminance value in the luminance class by linear interpolation of the integrated histogram, and
A normalization means for normalizing the integrated histogram linearly interpolated by the frequency interpolation means so that the integrated value of the maximum luminance class matches the upper limit value of the luminance based on the total number of pixels in the processing area.
Based on the normalized integrated histogram, the luminance conversion means for converting the luminance information of the input image for each processing region and generating the output image is provided.
The above-mentioned luminance class consists of n consecutive luminance values (n is an integer of 2 or more).
The histogram generating means divides the luminance range into two or more luminance classes and obtains the histogram.
The luminance conversion means is characterized by linearly interpolating by the frequency interpolation means and converting the luminance information of the input image by using the integrated histogram normalized by the normalization means as a tone curve for each processing region. Image processing device. Any one of claims 1 to 3, wherein the area dividing means divides the input image using a rectangular area composed of 2 m-th power (m is an integer of 1 or more) as the processing area. The image processing apparatus according to the section.

Images