KR101574022B1 - Adaptive image processing method and apparatus using pixel binning - Google Patents

Abstract

Disclosed are an adaptive image processing method using pixel binning and a device thereof. The adaptive image processing method using pixel binning includes the steps of: determining a binning ratio according to the brightness of each pixel in an input image; and amplifying the pixel value of each pixel by using the binning ratio determined for each pixel.

Description

[0001] The present invention relates to an adaptive image processing method and apparatus using pixel binning,

The present invention relates to image processing, and more particularly, to an adaptive image processing method and apparatus using pixel binning.

Pixel binning is a technique for improving the sensitivity of a sensor. It is a technique for increasing the sensitivity of a sensor by bundling a plurality of photodiodes into a single photodiode. Pixel binning, performed in the sensor, can improve sensitivity, but has the disadvantage that spatial resolution is degraded.

As a result, a digital pixel binning has been proposed, which can improve the spatial resolution degradation problem by adding the digital value of the pixels that have undergone sensor processing at the image processor end. Such a digital pixel binning has an advantage of brightening an image by binning a pixel in the form of an average filter without considering the characteristics of the image captured in a low-illuminance environment. Conventional digital pixel binning, however, has the problem that the bright pixels are amplified too brightly by blending both the bright pixels and the dark pixels in the form of an average filter, thereby degrading the sharpness.

An object of the present invention is to provide an adaptive image processing method and apparatus using pixel binning capable of obtaining a noise-suppressed image while minimizing sharpness loss of an image photographed in a low-illuminance environment.

The present invention also provides an adaptive image processing method and apparatus using pixel binning in which no frame memory or iteration operation is required in the calculation process and no additional hardware is required.

According to an aspect of the present invention, there is provided an adaptive image processing apparatus using pixel binning capable of obtaining a noise-suppressed image with minimized sharpness loss for an image photographed in a low-illuminance environment.

According to an embodiment of the present invention, there is provided a display apparatus including: a map generation unit for determining a binning magnification according to brightness of each pixel of an input image; And a pixel binning unit for amplifying a pixel value of each pixel using the binarization ratio of each pixel.

And a preprocessor for converting the input image into a gray channel. The map generator may determine a binarization ratio using pixel values of the pixels of the input image converted into the gray channel.

The map generator may calculate a brightness ratio of each pixel by applying an average filter of nxn size to each pixel of the converted input image, and calculate a brightness ratio of each pixel using a brightness ratio and a user- The binarization ratio of each pixel can be determined.

Wherein the map generation unit determines the binning magnification of each pixel so that the brightness ratio of each pixel is closer to the user specified maximum magnification as the brightness ratio of each pixel is lower, Can be determined.

According to another embodiment of the present invention, there is provided a calculation unit for calculating a difference value between neighboring pixels in an n x n area based on each pixel of an input image, where n is a natural number; An alignment unit for aligning the surrounding pixels using a difference value between each pixel and the neighboring pixels; And a pixel binning unit that amplifies the pixel value of each pixel by applying weights of the pixels and the aligned neighboring pixels according to a designated binning magnification.

The difference value is an absolute value of a difference between each pixel and each of the surrounding pixels.

The sorting unit may sort the neighboring pixels in descending order using the difference value.

Wherein the pixel binning unit is configured to determine a weight of each of the pixels and the aligned neighboring pixels within a maximum value, respectively, until each sum of weights of the pixels and the aligned neighboring pixels is equal to the binning magnification, The pixel value of each pixel can be amplified by adding the weights in the order of neighboring pixels having high alignment order among the aligned neighboring pixels.

According to another embodiment of the present invention, there is provided a calculation unit for calculating a deviation between each pixel of an input image and neighboring pixels of an n x n area, where n is a natural number; A uniform binning unit for calculating a first binning value of each pixel with a summation value obtained by equally dividing a weight of each of the neighboring pixels by a predetermined binning power; A pixel binning unit for calculating a second binning value of each pixel by applying weights differentially considering the similarity between each pixel and neighboring pixels; And a pixel blending unit for amplifying pixel values of each pixel using the deviation, the first binning value, and the second binning value.

The uniforming bin unit may determine the weights of the pixels to be within a maximum value and equally divide the binning ratio obtained by subtracting the weights of the pixels from the binning ratio to uniformly apply weights to the neighboring pixels to sum To calculate the first binning value.

Wherein the pixel binning unit calculates a difference value between neighboring pixels of the nxn region based on the pixels, aligns the neighboring pixels using the calculated difference value, The second binning value of each pixel can be calculated by applying the weight of the aligned neighboring pixels.

The pixel blending unit may multiply the result obtained by multiplying the first binning value by the deviation and a result obtained by multiplying the second binning value by a value obtained by subtracting the deviation from the reference binning value to blend and amplify pixel values of the respective pixels .

And a correction unit that corrects each of the amplified pixels using each pixel of the input image.

Wherein the correction unit divides the result of dividing the pixel value of the amplified pixel by a predetermined binarization ratio and the result obtained by dividing the pixel value of each pixel of the input image by a designated reference value and divides the result by the maximum value of the image, And adjusting the amplified pixel values of the respective pixels by summing the pixel values of the amplified pixels and the pixel values of the pixels of the input image using the correction ratio.

According to another embodiment of the present invention, there is provided a display apparatus comprising: a map generator for determining a binning magnification according to brightness of each pixel of an input image; An alignment unit for aligning the neighboring pixels by calculating a difference value between adjacent pixels of the n x n area based on each pixel of the input image, And a pixel binning unit for applying a weight of each of the pixels and the aligned neighboring pixels according to a binning magnification determined according to each pixel to amplify pixel values of the pixels to calculate a first binning value .

Wherein the pixel binning unit is configured to determine a weight of each of the pixels and the aligned neighboring pixels within a maximum value, respectively, until each sum of weights of the pixels and the aligned neighboring pixels is equal to the binning magnification, And the first binning value for each pixel is a value obtained by adding weights in order of neighboring pixels having high alignment order among the aligned neighboring pixels.

A calculation unit for calculating a deviation between each pixel of the input image and surrounding pixels of the n x n area; A uniform binning unit for calculating a second binning value by amplifying a pixel value of each pixel with a sum value obtained by equally dividing a weight of each of the neighboring pixels by a predetermined binning ratio; And blending a pixel value of each pixel using the deviation and the first binning value and the second binning value.

And a correction unit that corrects each of the amplified pixels using each pixel of the input image.

According to another aspect of the present invention, there is provided an adaptive image processing method using pixel binning that can obtain an image in which noise is suppressed while minimizing sharpness loss for an image photographed in a low-illuminance environment.

According to an embodiment of the present invention, there is provided a method of adjusting a binning magnification according to brightness of each pixel of an input image, And amplifying a pixel value of each pixel using the binarization ratio of each pixel.

The method of claim 1, further comprising the step of converting the input image to a gray channel prior to the step of determining the binarization ratio, wherein the binarization ratio determination step comprises using a pixel value of each pixel of the input image, The binarization ratio can be determined.

According to another embodiment of the present invention, there is provided a method of calculating a difference value between adjacent pixels of an n x n area based on each pixel of an input image, wherein n is a natural number; Aligning the neighboring pixels using a difference value between each pixel and the neighboring pixels; And amplifying a pixel value of each pixel by applying weights of the pixels and the aligned neighboring pixels according to a designated binning magnification, thereby providing an adaptive image processing method using pixel binning.

Wherein amplifying the pixel value of each pixel is performed by determining a weight of each of the pixels and the aligned neighboring pixels within a maximum value, wherein a sum of weights of the pixels and the aligned neighboring pixels is equal to the binning magnification The pixel value of each pixel can be amplified by adding the weights in the order of the neighboring pixels having the highest alignment order among the aligned neighboring pixels from the pixels to the sum of the pixels.

According to still another embodiment of the present invention, there is provided a method of calculating a deviation between each pixel of an input image and surrounding pixels of an n x n area, where n is a natural number; Calculating a first binning value of each pixel with a sum value obtained by equally dividing a weight of each of the neighboring pixels by a predetermined binning power; Calculating a second binning value of each pixel by applying a weighting difference to each pixel in consideration of the similarity between each pixel and neighboring pixels; And blending and amplifying the pixel values of each pixel using the deviation, the first binning value, and the second binning value, and then performing an adaptive image processing method using the pixel binning.

The step of blending and amplifying the pixel value of each pixel may further comprise the step of summing the result obtained by multiplying the first binning value by the deviation and the result obtained by multiplying the second binning value by a value obtained by subtracting the deviation from the reference value, The pixel value of the pixel can be blended and amplified.

According to still another embodiment of the present invention, there is provided a method of adjusting a binning magnification according to brightness of each pixel of an input image, Calculating a difference value between neighboring pixels in an n x n region based on each pixel of the input image, and aligning the neighboring pixels, wherein n is a natural number; And calculating a first binning value by amplifying a pixel value of each pixel by applying a weight value of each pixel and the aligned neighboring pixels using a binning magnification determined according to each pixel, An adaptive image processing method using pixel binning can be provided.

Calculating a deviation between each pixel of the input image and neighboring pixels of the n x n area; Calculating a second binning value by amplifying a pixel value of each pixel with a summation value obtained by equally dividing a weight of each of the neighboring pixels by a predetermined binning power; And blending the pixel value of each pixel using the deviation, the first binning value, and the second binning value.

And correcting the amplified pixel using each pixel of the input image.

Wherein the step of correcting comprises: summing a result obtained by dividing a pixel value of each of the amplified pixels by a predetermined binarization ratio and a result obtained by dividing a pixel value of each pixel of the input image by a specified reference value, Calculating a correction ratio for each pixel; And correcting the amplified pixel value of each pixel by summing the pixel values of the amplified pixels and the pixel values of the pixels of the input image using the correction ratio.

The present invention provides an adaptive image processing method and apparatus using pixel binning according to an exemplary embodiment of the present invention, so that a noise-suppressed image can be obtained while minimizing sharpness loss of an image photographed in a low-illuminance environment.

Further, the present invention does not require a frame memory or an iterative operation in the calculation process, and has the advantage that no additional hardware is required.

1 is a block diagram schematically showing an internal configuration of an image processing apparatus according to a first embodiment of the present invention;
2 is a flowchart illustrating an adaptive image processing method using pixel binning according to a first embodiment of the present invention.
3 is a block diagram schematically showing an internal configuration of an image processing apparatus according to a second embodiment of the present invention;
FIG. 4 is a diagram illustrating an example of arranging neighboring pixels in ascending order using pixel values of respective pixels and absolute values of differences of neighboring pixel values according to a second embodiment of the present invention; FIG.
5 is a flowchart illustrating an adaptive image processing method using pixel binning according to a second embodiment of the present invention.
FIG. 6 is a block diagram schematically showing an internal configuration of an image processing apparatus according to a third embodiment of the present invention; FIG.
7 is a flowchart illustrating an adaptive image processing method using pixel binning according to a third embodiment of the present invention.
FIG. 8 is a block diagram schematically showing an internal configuration of an image processing apparatus using pixel binning according to a fourth embodiment of the present invention; FIG.
9 is a flowchart illustrating an adaptive image processing method using pixel binning according to a fourth embodiment of the present invention.
FIG. 10 and FIG. 11 are diagrams showing a result of comparing amplified images according to the pixel binning of the conventional and the present invention; FIG.

BRIEF DESCRIPTION OF THE DRAWINGS The present invention is capable of various modifications and various embodiments, and specific embodiments are illustrated in the drawings and described in detail in the detailed description. It is to be understood, however, that the invention is not to be limited to the specific embodiments, but includes all modifications, equivalents, and alternatives falling within the spirit and scope of the invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

The terms first, second, etc. may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another.

The terminology used in this application is used only to describe a specific embodiment and is not intended to limit the invention. The singular expressions include plural expressions unless the context clearly dictates otherwise. In the present application, the terms "comprises" or "having" and the like are used to specify that there is a feature, a number, a step, an operation, an element, a component or a combination thereof described in the specification, But do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or combinations thereof.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram schematically showing an internal configuration of an image processing apparatus according to a first embodiment of the present invention.

1, an image processing apparatus 100 according to an exemplary embodiment of the present invention includes an input unit 110, a preprocessor 115, a map generator 120, a pixel beeper 125, a corrector 130 ), A memory 135, and a processor 140.

The input unit 110 is a means for receiving an input image. Here, the input image may be an image photographed in a low-illuminance environment.

The preprocessing unit 115 converts the input image into a gray channel. That is, the preprocessing unit 115 can convert an input image photographed in a low-illuminance environment into an RGB image into a gray image. Since the method of converting the RGB image into the gray image itself is obvious to those skilled in the art, a detailed description thereof will be omitted.

The map generating unit 120 generates a ratio map for determining an amplification factor (hereinafter referred to as a binning ratio) according to the brightness of each pixel in the input image converted into the gray image.

The ratio map is derived using an input image that is converted to a gray image to determine a single value for each channel of R / G / B. In the case of deriving the ratio map as it is without converting each channel of R / G / B into a gray image, each pure R / G / B may be mistaken as black. Accordingly, the map generating unit 120 can generate the ratio map using the input image converted into the gray image by the preprocessing unit 115.

In an embodiment of the present invention, when generating the ratio map for determining the binning magnification according to the brightness of each pixel in the map generator 120, an average filter may be used to minimize the influence of noise. And calculates the brightness ratio for each pixel.

This can be expressed as the following equation.

은 에버리지 커널을 나타내고,

는 입력 영상을 그레이 채널로 변환한 영상을 나타낸다. 또한, max() 함수는 영상의 최대값을 계산하는 함수이다. 따라서, 각 픽셀에 대한 밝기 비율(t(x,y))는 0 내지 1 사이의 값을 가지며, 픽셀값이 밝은값을 가질수록 높은 값을 가지게 되는 것을 알 수 있다.here,

Represents an average kernel,

Represents an image obtained by converting an input image into a gray channel. The max () function is a function for calculating the maximum value of the image. Therefore, it can be seen that the brightness ratio t (x, y) for each pixel has a value between 0 and 1, and the pixel value has a higher value as it has a bright value.

The map generating unit 120 may calculate the binning magnification for the brightness value using the calculated brightness ratio for each pixel.

This is expressed by the following equation (2).

은 사용자가 정해는 최대 비닝 배율을 나타낸다. 또한, 각 픽셀의 밝기값에 대한 비닝 배율은 해당 픽셀을 포함하도록 1 이상의 값을 가지도록 구현하였다. 즉, 각 픽셀의 비닝 배율은 각 픽셀의 밝기 비율이 높을수록 1에 가까운 값을 가지고, 밝기 비율을 낮을수록

에 가까운 값을 가지는 것을 알 수 있다.here,

Represents the maximum binning magnification that the user has set. In addition, the binning magnification for the brightness value of each pixel is implemented to have one or more values to include the pixel. That is, the binning magnification of each pixel has a value close to 1 as the brightness ratio of each pixel is higher, and as the brightness ratio is lower

As shown in Fig.

Accordingly, the map generator 120 can derive a ratio map for the binning magnification so that the image is amplified with a higher binning magnification as the image becomes darker, and closer to 1 as the brighter the image, so that the map is amplified with a lower binning magnification.

As described above, the image is amplified with a higher binning magnification as the image becomes darker, and is amplified with a lower binning magnification so as to be closer to 1, so that the problem of degrading the sharpness can be solved There is an advantage.

The pixel binning unit 125 calculates a first binning value for each pixel according to a binning ratio of each pixel of the input image calculated through the map generating unit 120 and amplifies the pixel value of each pixel do.

For example, the pixel transmitting unit 125 may calculate the first binning value for the first pixel by multiplying the pixel value of the first pixel by 2.5, assuming that the binning magnification of the first pixel is 2.5. On the other hand, assuming that the binning magnification of the second pixel is 1.5, the first binning value for the second pixel can be calculated by multiplying the pixel value of the second pixel by the binning magnification of 1.5.

The pixel voting unit 125 can thus amplify each pixel by calculating the first binning value for each pixel by multiplying the calculated binning magnification for each pixel by the pixel value of each pixel.

The correction unit 130 performs a function of correcting the pixel value (i.e., the first binning value) of each pixel amplified through the pixel binning unit 125 using each pixel value of the input image.

The correction unit 130 calculates a correction ratio using the first binning value for each pixel and each pixel value of the input image to correct the first binning value for each pixel using each pixel value of the input image do.

For example, the correction unit 130 divides the pixel value (i.e., the first binning value) of each amplified pixel by the binning ratio, and divides the pixel value of each pixel of the input image into a predetermined reference value (e.g., 2) And the correction ratio can be calculated by dividing the sum by the maximum value of the image.

Thus, the correction ratio can be calculated so as to have an average property of the input image and the binning image.

As a result, the correction unit 130 decreases the correction ratio as the binning magnification of each pixel increases by dividing the pixel value (i.e., the first binning value) of each amplified pixel by the binning magnification. That is, as the binning magnification of each pixel increases, the correction effect through the correction unit 130 may be reduced so that a large number of bright portions of the amplified image can be adjusted through the pixel binning.

The correction ratio can be calculated using the following equation (3).

는 비닝 배율을 나타내고,

는 픽셀 비닝부를 통해 증폭된 비닝 영상을 나타내고,

는 입력 영상을 나타낸다.here,

Represents the binning magnification,

Represents a binned image amplified through the pixel binning unit,

Represents an input image.

When the correction ratio for each pixel is calculated, the correction unit 130 can correct the pixel value (first binning value) of each amplified pixel using each pixel value of the input image by applying the correction ratio.

This can be expressed by equation (4).

As described above, since the pixel value (first binning value) of each bined pixel is corrected using each pixel value of the input image through the corrector 130, the higher the correction ratio, the brighter the portion. And the first binning value of the amplified binned image can be reflected because the darkening portion is lower as the correction ratio is lower.

As a result, the image finally obtained can be suppressed of noise close to the input image, and an image with well-balanced luminance can be obtained.

The memory 135 stores various algorithms necessary for operating the image processing apparatus 100 according to an embodiment of the present invention, various data required for an image processing process using pixel binning, a function of storing various data mass- .

The processor 140 may include internal components (e.g., an input unit 110, a preprocessor 115, a map generation unit 120, and a pixel transmitting unit) of the image processing apparatus 100 according to an embodiment of the present invention. The controller 125, the corrector 130, the memory 135, and the like).

2 is a flowchart illustrating an adaptive image processing method using pixel binning according to the first embodiment of the present invention.

In step 210, the image processing apparatus 100 converts the input image into a gray channel.

As described above, when the binning magnification according to the brightness of each pixel is determined using the input image, an error represented by the binarization factor for the brightness due to the color component of each pixel is removed. Since this is the same as that described above with reference to FIG. 1, a detailed description will be omitted.

In step 215, the image processing apparatus 100 determines the binning magnification according to the brightness of each pixel of the converted input image. For example, the image processing apparatus 100 may determine the binarization magnification using the pixel values of the respective pixels of the input image converted into the gray image.

As described above with reference to FIG. 1, in order to minimize the influence of noise, the image processing apparatus 100 calculates brightness ratios for each pixel using an nxn average filter, It is possible to determine the binning magnification for the pixel. The detailed description thereof is the same as FIG. 1, so that a duplicate description will be omitted.

In operation 220, the image processing apparatus 100 calculates a first binning value for a pixel value of each pixel using the binning magnification determined according to the brightness of each pixel, and generates a binning image by amplifying each pixel.

In step 225, the image processing apparatus 100 corrects the first binning value of each pixel using each pixel of the input image.

For example, the image processing apparatus 100 divides the pixel value (i.e., the first binning value) of each amplified pixel by the binning magnification, divides the pixel value of each pixel of the input image into a predetermined reference value (e.g., 2) And the correction ratio can be calculated by dividing by the maximum value of the image.

Thus, the correction ratio can be calculated so as to have an average property of the input image and the binning image.

The image processing apparatus 100 divides the pixel value (i.e., the first binning value) of each amplified pixel by the binning magnification, so that the correction ratio becomes lower as the binning magnification of each pixel becomes larger. That is, the image processing apparatus 100 can reduce the correction effect as the binning magnification of each pixel increases, so that a large amount of bright portions of the amplified image can be controlled through the pixel binning.

FIG. 3 is a block diagram schematically illustrating an internal configuration of an image processing apparatus according to a second embodiment of the present invention. FIG. 4 is a block diagram of a pixel value of each pixel and neighboring pixel values Are used to sort neighboring pixels in ascending order.

3, the image processing apparatus 300 according to the second embodiment of the present invention includes an input unit 310, a calculation unit 315, an alignment unit 320, a pixel receiving unit 325, a correction unit 330 ), A memory 335, and a processor 340. [

 Hereinafter, the description of the input unit 310, the correction unit 330, the memory 335, and the processor 340, which are the same as those of the image processing apparatus 100 of FIG. 1, will be omitted. I will explain.

The calculation unit 315 calculates a difference value between neighboring pixels of the n x n region based on each pixel of the input image.

For example, the calculation unit 315 may calculate an absolute value of a difference between neighboring pixels of the n x n region based on each pixel of the input image.

The aligning unit 320 aligns surrounding pixels using the absolute values of the pixel values of the pixels and the neighboring pixel values.

At this time, the aligning unit 320 may align neighboring pixels in ascending order by using the absolute values of the pixel values of the pixels and the neighboring pixel values.

This can be expressed by the following equation (5).

Where d represents the absolute value of the difference between the current pixel and the surrounding pixels, and k represents the n x n block. Also, i, j indicate positions in n x n blocks, respectively, and can have values from -1 to 1 in the current pixel. The sort () function represents an alignment function, and it is assumed that the sort () function is performed in ascending order in an embodiment of the present invention.

In the second embodiment of the present invention, assuming that n is 3 and the area for calculating the difference value between the current pixel and neighboring pixels is 3 x 3 block, it is assumed that i, j has a value of -1 to 1 . Each of i and j is a value for referring to the position of neighboring pixels based on the current pixel. It is of course that the values representing i and j can also be different when the size of the block is not 3 x 3. That is, it is obvious that n may be set to another size such as 2, 4, 5, etc. in addition to 3.

FIG. 4 shows an example of aligning neighboring pixels in ascending order using pixel values of respective pixels and absolute values of differences of neighboring pixel values.

For example, in the nxn area, assuming that n is 3, eight neighboring pixels are selected centering on each pixel (i.e., the current pixel), and the absolute values of the differences between each pixel and each of the neighboring pixels are calculated .

Then, the aligning unit 320 arranges the neighboring pixels as shown in FIG. 3 by arranging them in ascending order using the absolute values of the differences between the pixels and the neighboring pixels.

This can be expressed as the following equation (6).

Here, s (k) represents a pixel value of the n x n region arranged in order of small absolute difference value between each pixel and each surrounding pixel.

The pixel binning unit 325 performs a function of calculating and amplifying a first binning value of each pixel by applying weights of neighboring pixels aligned with each pixel according to a specified binning magnification. In this manner, each pixel is referred to as an image binning image in which each pixel is amplified by the pixel binning unit 325.

For example, as shown in FIG. 4, when neighboring pixels around each pixel (current pixel) are aligned in ascending order using the absolute difference value (i.e., ), The pixel receiving unit 325 applies a weighting order to peripheral pixels having a high alignment order among the neighboring pixels arranged from each pixel until the weight sum of each pixel and aligned peripheral pixels becomes equal to the binning magnification The first binning value for each pixel can be calculated and amplified by the summed value.

At this time, the weights of the pixels and the neighboring pixels may be respectively determined within a maximum value, and the maximum value may be set to one.

The formula for deriving the weight for each pixel and surrounding pixels is:

When the weights for the pixels and neighboring pixels are determined, the pixel binning unit 325 multiplies the pixels of the nxn region, which are arranged in order of small absolute difference between each pixel and each of the neighboring pixels, A first binning value for the pixel can be calculated. This can be expressed by the following equation (8).

와

는 각각 앞에서 구한 1x9의 정렬된 픽셀들의 행렬과 가중치 행렬의 트랜스포즈(transpose)값이다. 결과적으로, 본 발명의 다른 실시예에 따른 각 픽셀에 대한 제1 비닝값은 현재 픽셀과 유사한 주변 픽셀 순으로 높은 가중치를 적용하여 합산된 결과값임을 알 수 있다.here,

Wow

Is the transpose of the matrix of 1x9 aligned pixels and the weight matrix obtained previously. As a result, it can be seen that the first binning value for each pixel according to another embodiment of the present invention is a sum value obtained by applying a high weighting value to neighboring pixels similar to the current pixel.

According to another embodiment of the present invention, the pixel binning unit 325 is not applied to each pixel evenly according to the binning magnification specified at the time of applying the weight, but uses the binning magnification calculated according to the brightness of each pixel according to FIG. It is natural that it is possible.

5 is a flowchart illustrating an adaptive image processing method using pixel binning according to a second embodiment of the present invention.

In operation 510, the image processing apparatus 300 calculates the absolute value of the difference with neighboring pixels of the n x n region based on each pixel of the input image. Here, n may be a natural number.

In another embodiment of the present invention, n is assumed to be 3, but it is natural that n may be a natural number other than 3.

In step 515, the image processing apparatus 300 arranges neighboring pixels in order similar to each pixel using the calculated absolute difference value.

For example, the image processing apparatus 300 aligns neighboring pixels in ascending order using the calculated absolute difference values for each pixel and surrounding pixels. The detailed description thereof is the same as that described with reference to FIG. 3, so that redundant description will be omitted.

In step 520, the image processing apparatus 300 calculates the first binning value by amplifying the pixel value of each pixel by applying the weight of the neighboring pixels aligned with each pixel in consideration of the binning magnification.

In more detail, when the weights are applied to the respective pixels and the surrounding pixels in consideration of the binning magnification, the image processing apparatus 300 sets the maximum value of the weights of the pixels and the surrounding pixels so as not to exceed the self (i.e., 1) Weights can be applied. At this time, the image processing apparatus 300 applies the weighting in descending order of priority, including each pixel (i.e., itself), and weights the weighting value to each pixel or surrounding pixels until the sum of the weights is equal to the binning magnification Can be applied. The image processing apparatus 300 may set the weights of neighboring pixels having low priority such that the sum of weights exceeds the binning magnification to zero so that they are not reflected.

This is the same as that described in detail with reference to FIG. 3, and a detailed description thereof will be omitted.

The image processing apparatus 300 calculates a first binning value for each pixel by multiplying each pixel by a weight determined for each pixel and each neighboring pixel, So that each pixel can be amplified.

In step 525, the image processing apparatus 300 corrects the first binning value of each pixel using each pixel of the input image.

Since this is the same as step 225 in FIG. 2, a duplicate description will be omitted.

6 is a block diagram schematically showing an internal configuration of an image processing apparatus according to a third embodiment of the present invention.

6, the image processing apparatus 600 according to the third exemplary embodiment of the present invention includes an input unit 610, a calculation unit 615, a uniform calling unit 620, a pixel calling unit 625, A correction unit 630, a correction unit 635, a memory 640, and a processor 645. Here, the input unit 610, the pixel receiving unit 625, the correction unit 635, the memory 640, and the processor 645 have the same configurations as those described in FIGS. 1 and 3, respectively, .

The calculation unit 615 calculates a deviation of each pixel of the input image and neighboring pixels of a predetermined size (e.g., n-region).

This can be expressed by the following equation.

는 n x n 영역의 평균값을 나타내고, s(0)는 각 픽셀의 픽셀값을 나타낸다. 여기서,

는 입력 영상의 최대값으로 예를 들어, 8비트 영상인 경우, 255로 설정될 수 있다.here,

Represents the average value of the nxn area, and s (0) represents the pixel value of each pixel. here,

The maximum value of the input image may be set to 255, for example, in the case of an 8-bit image.

The uniforming unit 620 applies a uniform weighting factor to the surrounding pixels of the n x n region based on each pixel of the input image to amplify each pixel to generate a binning image.

For example, if n is 3, the weight of each pixel is set to 1 when the area is 3 x 3, and the surrounding eight pixels are binarized by weighting the binning magnification by subtracting the weight of each pixel from the binning magnification. Can be applied. This can be expressed by the following equation.

When weights are determined for each pixel and neighboring pixels, a weight is applied to each pixel and neighboring pixels, and the weighted values are summed to amplify each pixel to calculate a second binning value for each pixel.

This can be expressed as Equation 11.

는 각 픽셀 및 주변 픽셀에 대한 가중치의 행렬에 대한 트랜스포즈(transpose)값을 나타내고, S는 각 픽셀 및 주변 픽셀의 행렬을 나타낸다.here,

Represents a transpose value for a matrix of weights for each pixel and surrounding pixels, and S represents a matrix of each pixel and surrounding pixels.

The pixel binning unit 625 calculates a first binning value for each pixel by applying weights to neighboring pixels arranged in accordance with the similarity between each pixel and each pixel according to the binning magnification, Function.

The pixel binning unit 625 can perform the same function as the pixel binning described with reference to FIG. 1 and FIG. 3, which is the same as that described above, so that a duplicate description will be omitted.

However, in order to facilitate understanding and explanation, the binning image calculated by the pixel binning unit 625 is referred to as a first binning image or a binning value of each pixel is referred to as a first binning value, The binning value of each pixel calculated through the above will be collectively referred to as a second binning value.

The pixel blending unit 630 performs a function of blending each pixel using a first binning value, a second binning value, and a deviation of each pixel. That is, the pixel blending unit 630 may blend each pixel by summing the result obtained by multiplying the deviation and the second binning value by the result obtained by multiplying the result obtained by subtracting the deviation from the reference value from the first binning value. This can be expressed by the following equation (12).

Each pixel amplified through such pixel blending is judged to be noise as much as the average of nxn blocks, so that it is determined that noise is not a noise, and the influence of the first binning value is large. If the difference between the average value and each pixel is large, The second binning value calculated by the second binning value calculation unit 620 can be greatly affected.

7 is a flowchart illustrating an adaptive image processing method using pixel binning according to a third embodiment of the present invention.

In operation 710, the image processing apparatus 600 calculates a deviation between each pixel of the input image and surrounding pixels of the n x n region. Here, n may be a natural number.

In another embodiment of the present invention, n is assumed to be 3, but it is natural that n may be a natural number other than 3.

In operation 715, the image processing apparatus 600 divides the binarization power equally to calculate weights for each of the surrounding pixels. At this time, the image processing apparatus 600 may calculate the weight for each of the surrounding pixels by dividing the binned power by subtracting the binning power by setting the weight of each pixel to a maximum value.

In step 720, the image processing apparatus 600 calculates a first binning value of each pixel by applying a weight to each pixel and a surrounding pixel, respectively, and adding the values. Hereinafter, the convenience of understanding and explanation will be referred to as uniform vining.

In step 725, the image processing apparatus 600 calculates a second binning value of each pixel by applying weights differentially according to the binning magnification in consideration of the similarity between each pixel and neighboring pixels.

This is the same as that described with reference to Figs. 2 to 5, and will be collectively referred to as pixel binning in order to facilitate understanding and explanation in this specification.

In the case of pixel binning, the binarization ratio may be specified by the user, or the binarization ratio may be determined according to the brightness of each pixel, as described with reference to FIG.

In step 730, the image processing apparatus 600 blends and amplifies pixel values of each pixel using the deviation, the first binning value, and the second binning value.

For example, the image processing apparatus 600 blends the pixel value of each pixel by summing the result obtained by multiplying the first binning value by the deviation and the result obtained by multiplying the second binning value by a value obtained by subtracting the deviation from the reference value Can be amplified. This is the same as that described in detail with reference to FIG. 6, and a duplicate description will be omitted.

In step 735, the image processing apparatus 600 corrects the amplified binning value of each pixel using the pixel value of each pixel of the input image. This is the same as that described with reference to FIG. 6, so duplicate descriptions are omitted.

FIG. 8 is a block diagram schematically illustrating an internal configuration of an image processing apparatus using pixel binning according to a fourth embodiment of the present invention. 1, 3, and 6, it is assumed that each pixel binning is performed individually. However, pixel binning may be performed adaptively (stepwise) in an image processing apparatus as shown in FIG.

8, an image processing apparatus 800 according to an embodiment of the present invention includes an input unit 810, a preprocessing unit 815, a map generation unit 820, a first calculation unit 825, 830, a pixel binning unit 835, a second calculation unit 840, a uniforming unit 845, a pixel blending unit 850, a correction unit 855, a memory 860, and a processor 865 .

8 are the same as the respective elements included in FIG. 1, FIG. 3, and FIG. 6, so that duplicate descriptions will be omitted and only different differences will be described.

According to another embodiment of the present invention, the pixel binning unit 835 applies weights of neighboring pixels aligned with each pixel using the binning magnification according to the brightness of each pixel calculated by the map generating unit 820 The first binning value of each pixel can be calculated and amplified.

The uniforming unit 845 calculates the weights for the neighboring pixels of the nxn region based on each pixel, and uses the binning magnification calculated according to the brightness of each pixel described in Fig. 1, instead of the designated binning magnification, The weights of the pixels can be applied in an equal ratio.

Other configurations and functions are the same as those described with reference to Figs. 1, 3 and 6, and a duplicate description will be omitted.

FIG. 9 is a flowchart illustrating an adaptive image processing method using pixel binning according to a fourth embodiment of the present invention, and FIGS. 10 and 11 illustrate a result of comparing amplified images according to the conventional pixel binning method of the present invention FIG.

In operation 910, the image processing apparatus 800 determines the binning magnification according to the brightness of each pixel of the input image.

For example, the image processing apparatus 800 converts an input image into a gray image, and applies an nxn-size average filter to each pixel of the input image converted into the gray image to calculate brightness ratios for the respective pixels , The binarization magnification for each pixel brightness can be calculated using the calculated brightness ratio of each pixel and the user-specified maximum magnification. This is the same as that described in detail with reference to FIG. 1, so that redundant description will be omitted.

In step 915, the image processing apparatus 800 calculates an absolute value of a difference between each pixel of the input image and each of the neighboring pixels of the nxn area, and then calculates the neighboring pixels in ascending order using the calculated absolute value of the difference do. This is the same as that described above with reference to FIG. 3 and FIG. 4, so a duplicate description will be omitted.

In step 920, the image processing apparatus 800 applies a weight to neighboring pixels arranged according to the similarity degree between each pixel and the corresponding pixel using the binning magnification determined for each pixel, And calculates the first binning value. Accordingly, the image processing apparatus 800 can generate the first binning image by amplifying each pixel in consideration of the brightness of each pixel and the similarity of neighboring pixels. This is the same as that described in detail with reference to FIG. 1 and FIG. 3, so a duplicate description will be omitted.

Next, in step 925, the image processing apparatus 800 calculates a deviation between each pixel of the input image and surrounding pixels of the n x n area.

In operation 930, the image processing apparatus 800 equally divides the binning magnification of each pixel to calculate a weight for each of the surrounding pixels. At this time, the image processing apparatus 800 sets the weight to a maximum value (1) in order to reflect each pixel as it is at the time of pixel binning and divides the weight of neighboring pixels equally by a value obtained by subtracting the weight of each pixel from the binning magnification have. This is the same as that described with reference to FIG. 6, so duplicate descriptions are omitted.

In step 935, the image processing apparatus 800 calculates the second binning value of each pixel by summing the result of applying the calculated weight to each pixel and the surrounding pixels, and amplifies each pixel.

When the second binning value according to uniform binning is calculated in this manner, the image processing apparatus 800 blends each pixel using the first binning value and the second binning value of each pixel in step 935 using the deviation. This has the advantage of minimizing the effect of noise when each pixel corresponds to noise. This is the same as that described with reference to FIG. 6, so duplicate descriptions are omitted.

In step 940, the image processing apparatus 800 blends binning values binned in different ways using the calculated deviation, the first binning value, and the second binning value for each pixel to amplify each pixel. This is the same as that described with reference to FIG. 6, so duplicate descriptions are omitted.

In operation 945, the image processing apparatus 800 corrects the amplified pixel value of each pixel using the pixel value of each pixel of the input image. This is the same as that described with reference to FIG. 1, so duplicate descriptions will be omitted.

In FIG. 10, reference numeral 1010 denotes an image taken at an underground parking lot of about 8 lux, and an image taken at ISO 3200 of EV +1. The result of pixel binning using 1010 of FIG. 10 as an input image in the conventional method is shown in 1030, and a resultant image according to the fourth embodiment of the present invention is shown in 1040. It can be seen that the sharpness of the bright portion according to the pixel binning is further improved in the resultant image according to the fourth embodiment of the present invention.

In FIG. 11, reference numeral 1110 denotes an image taken at an underground parking lot of about 8 lux and an image taken at ISO 3200 for an EV +1. A result of pixel binning according to a conventional method using 1110 of FIG. 11 as an input image is shown at 1130, and a result of pixel binning according to a fourth embodiment of the present invention is shown at 1140. As shown in FIG. 11, it can be seen that the pixel binning according to the fourth embodiment of the present invention is adaptive to brightness, and the detail of the list portion is well maintained by the effect of WDR, thereby improving the sharpness.

Meanwhile, the adaptive image processing method using pixel binning according to an embodiment of the present invention can be implemented in a form of a program command that can be executed through a variety of means for processing information electronically and recorded in a storage medium. The storage medium may include program instructions, data files, data structures, and the like, alone or in combination.

Program instructions to be recorded on the storage medium may be those specially designed and constructed for the present invention or may be available to those skilled in the art of software. Examples of storage media include magnetic media such as hard disks, floppy disks and magnetic tape, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, magneto-optical media and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as devices for processing information electronically using an interpreter or the like, for example, a high-level language code that can be executed by a computer.

The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the invention as defined in the appended claims. It will be understood that the invention may be varied and varied without departing from the scope of the invention.

100: image processing device
110: input unit
115:
120: map generating unit
125: pixel non-
130:
135: Memory
140: Processor

Claims (29)

A preprocessing unit for converting an input image into a gray channel;
A map generator for determining a binarization ratio using pixel values of the pixels of the input image converted into the gray channel; And
And a pixel binning unit for amplifying a pixel value of each pixel by using a binning magnification of each pixel,
The map generator may calculate a brightness ratio of each pixel by applying an average filter of nxn size to each pixel of the input image,
Wherein the binarization ratio determining unit determines the binarization ratio of each pixel by using the brightness ratio of each pixel and the user-specified maximum magnification. delete delete The method according to claim 1,
The map generation unit generates,
Determining a binning magnification of each pixel so as to approach the user-specified maximum magnification as the brightness ratio of each pixel is lower,
Wherein the binarization ratio determining unit determines the binarization ratio of each pixel so that the ratio of brightness of each pixel is closer to the maximum brightness ratio. Converting an input image to a gray channel;
Determining a binarization magnification using pixel values of each pixel of the input image transformed into the gray channel; And
Amplifying a pixel value of each pixel by using a binning ratio of each pixel,
The binarization ratio may be determined by calculating a brightness ratio of each pixel by applying an average filter of nxn size to each pixel of the converted input image, Wherein the binarization ratio of each pixel is determined using the maximum magnification. delete A calculation unit for calculating a difference value between neighboring pixels of the nxn area based on each pixel of the input image, where n is a natural number;
An alignment unit for aligning the neighboring pixels in descending order using a difference value between each pixel and the surrounding pixels; And
And a pixel binning unit for amplifying pixel values of each pixel by applying weights of the pixels and the aligned neighboring pixels according to a specified binning ratio. 8. The method of claim 7,
Wherein the difference value is an absolute value of a difference between each pixel and each of the neighboring pixels. delete 8. The method of claim 7,
The pixel-
Determining a weight of each of the pixels and the aligned neighboring pixels within a maximum value,
Wherein each of the plurality of pixels includes a plurality of pixels arranged in the order of a plurality of pixels arranged in a matrix, And amplifies the pixel value of the pixel. Calculating a difference value between neighboring pixels of the nxn region based on each pixel of the input image, where n is a natural number;
Aligning the neighboring pixels in descending order using a difference value between each pixel and neighboring pixels; And
And amplifying a pixel value of each pixel by applying weights of the pixels and the aligned neighboring pixels according to a specified binning magnification. The method of claim 11, wherein
Wherein the step of amplifying the pixel value of each pixel comprises:
Determining a weight of each of the pixels and the aligned neighboring pixels within a maximum value,
Wherein each of the plurality of pixels includes a plurality of pixels arranged in the order of a plurality of pixels arranged in a matrix, Wherein the pixel value of the pixel is amplified. A calculation unit for calculating a deviation between each pixel of the input image and surrounding pixels of the nxn area, where n is a natural number;
A uniform binning unit for calculating a first binning value of each pixel with a sum value obtained by equally dividing a weight of each of the neighboring pixels by a predetermined binning power;
A pixel binning unit for calculating a second binning value of each pixel by applying weights differentially considering the similarity between each pixel and neighboring pixels; And
And a pixel blending unit for amplifying a pixel value of each pixel using the deviation, the first binning value, and the second binning value. 14. The method of claim 13,
Wherein the uniforming-
Wherein the binarization unit divides the binarization factor obtained by subtracting the weights of the respective pixels from the binarization factor to uniformly divide the binarization ratio of the respective pixels and applies the weights to each of the neighboring pixels uniformly, And calculates a binning value. 14. The method of claim 13,
The pixel-
Calculating a difference value between neighboring pixels of the nxn region based on each of the pixels, aligning the neighboring pixels using the calculated difference value,
And calculates a second binning value of each pixel by applying weights of the pixels and the aligned neighboring pixels according to a specified binning ratio. 14. The method of claim 13,
Wherein the pixel blending unit comprises:
Wherein a sum of a result obtained by multiplying the first binning value by the deviation and a result obtained by multiplying the second binning value by a value obtained by subtracting the deviation from a reference value are added to each other to blend and amplify pixel values of the pixels. Processing device. Calculating a deviation between each pixel of the input image and surrounding pixels of the nxn region, where n is a natural number;
Calculating a first binning value of each pixel with a sum value obtained by equally dividing a weight of each of the neighboring pixels by a predetermined binning power;
Calculating a second binning value of each pixel by applying a weighting difference to each pixel in consideration of the similarity between each pixel and neighboring pixels; And
And blending and amplifying pixel values of each pixel using the deviation, the first binning value, and the second binning value. 18. The method of claim 17,
Wherein the step of blending and amplifying the pixel values of each pixel comprises:
And multiplying the result of multiplying the first binning value by the deviation and a result obtained by multiplying the second binning value by a value obtained by subtracting the deviation from the reference binarization value to blend and amplify the pixel value of each pixel. Adaptive Image Processing Method Using. 14. A method according to any one of claims 1, 7 or 13,
And a correction unit for correcting each of the amplified pixels using each pixel of the input image. 20. The method of claim 19,
Wherein,
A result obtained by dividing a pixel value of each of the amplified pixels by a predetermined binarization ratio and a result obtained by dividing a pixel value of each pixel of the input image by a designated reference value is summed and divided by a maximum value of the image, Calculating,
Wherein the controller adjusts the amplified pixel value of each pixel by summing the pixel value of each pixel and the pixel value of each pixel of the input image using the correction ratio. A map generating unit for determining a binning magnification according to the brightness of each pixel of the input image;
An alignment unit for aligning the neighboring pixels by calculating a difference value between adjacent pixels of the nxn area based on each pixel of the input image, And
And a pixel binning unit for calculating a first binning value by amplifying a pixel value of each pixel by applying weights of the pixels and the aligned neighboring pixels according to a binning magnification determined according to each pixel,
The pixel-
Determining a weight of each of the pixels and the aligned neighboring pixels within a maximum value,
Wherein each of the plurality of pixels includes a plurality of pixels arranged in the order of a plurality of pixels arranged in a matrix, And calculates the first binning value for the pixel. delete 22. The method of claim 21,
A calculation unit for calculating a deviation between each pixel of the input image and surrounding pixels of the nxn area;
A uniform binning unit for calculating a second binning value by amplifying a pixel value of each pixel with a sum value obtained by equally dividing a weight of each of the neighboring pixels by a predetermined binning ratio; And
And a pixel blending unit for blending pixel values of each pixel using the deviation, the first binning value, and the second binning value. 24. The method of claim 23,
And a correction unit for correcting each of the amplified pixels using each pixel of the input image. Determining a binning magnification according to brightness of each pixel of the input image;
Calculating a difference value between neighboring pixels of the nxn region based on each pixel of the input image, and aligning the neighboring pixels, wherein n is a natural number;
Calculating a first binning value by amplifying a pixel value of each pixel by applying a weight value of each of the pixels and the aligned neighboring pixels using a binning magnification determined according to each pixel;
Calculating a deviation between each pixel of the input image and neighboring pixels of the nxn area;
Calculating a second binning value by amplifying a pixel value of each pixel with a summation value obtained by equally dividing a weight of each of the neighboring pixels by a predetermined binning power; And
And blending the pixel values of each pixel using the deviation, the first binning value, and the second binning value. delete The method according to any one of claims 5, 11, 17, and 25,
And correcting the amplified pixel using each pixel of the input image using the pixel binning. 28. The method of claim 27,
Wherein the correcting comprises:
A result obtained by dividing the pixel value of the amplified pixel by the specified binning magnification and a result obtained by dividing the pixel value of each pixel of the input image by the designated reference value are summed and divided by the maximum value of the image, Respectively; And
And correcting the amplified pixel value of each pixel by summing the pixel values of the amplified pixels and the pixel values of the pixels of the input image using the correction ratio. Adaptive Image Processing Method. A recording medium on which program codes for carrying out the method according to any one of claims 5, 11, 17 and 25 are recorded.

Images