KR102301924B1 - Shadow reconstruction method using multi-scale gamma correction - Google Patents

Abstract

The shadow restoration method using multi-scale gamma correction provided by the present invention includes the steps of receiving an image including a shadow region and a normal region, correcting the shadow region with reference to the gamma value of the normal region, the shadow region and the general region and correcting the boundary area of the area.

Description

Shadow restoration method using multi-scale gamma correction {SHADOW RECONSTRUCTION METHOD USING MULTI-SCALE GAMMA CORRECTION}

The present invention relates to a shadow restoration method using multi-scale gamma correction. More particularly, it relates to a shadow restoration method using multi-scale gamma correction capable of restoring a more natural shadow region by adding an image restoration process.

A shadow is a common phenomenon observed in natural landscapes and is effective in understanding the structure and meaning of natural scene images. However, it can complicate recognition such as feature detection, object recognition, and scene analysis.

The shadow included in the digital image is one of the most prominent effects in the lighting of the scene, and its main characteristics are the deformability such as intensity, the maximum value of local blocks, and skewness, and the illumination-invariant of adjacent pixels. , have immutability characteristics such as gradient and texture similarity.

Shadows provide information about the direction of objects and light sources, but they also reduce the performance and stability of various computer vision algorithms, such as object detection, tracking, and recognition.

For this reason, previous studies of shadow detection have been conducted, and these are largely a method using the intensity of an image, a method using the chromaticity of the image, a method using geometry, and the texture of the image. ), and the performance of shadow detection is variously proposed depending on the field of application.

Although shadow detection is accelerating the advancement of technology in terms of accuracy, it is finding its application in computer vision or image processing fields such as intelligent video surveillance, traffic surveillance, and aerial imagery analysis. can't see

In particular, in 2016, there was a case where the shadow of a vehicle crossed the lane on the highway and a fine was notified. In the image processing of object recognition, it is very important to detect and restore the shadow region, and it must be considered essential in the preprocessing of computer vision.

Therefore, shadow detection must be considered essential in the digital image analysis procedure, and depending on the accuracy of the detected shadow, the image from which the shadow is reconstructed (or shadow removed) can be interpreted in various ways, so that the shadow is removed. It is necessary to develop a method to increase the reliability of the restored image.

The present invention is to solve the above problems, and to provide a shadow restoration method using multi-scale gamma correction.

-

According to the present invention as described above, the correction of the gamma value for each color channel is performed, so that the correction of the shadow region is performed more accurately.

In addition, since the gamma value correction process of the shadow region and the correction of the boundary region are secondary, there is an advantage in that the corrected image for the shadow region becomes more sophisticated.

1 is a flowchart illustrating a shadow restoration method according to an embodiment of the present invention.
FIG. 2 is pixel coordinates for explaining the concept of a shadow restoration method according to the embodiment of FIG. 1 of the present invention.
FIG. 3 is a graph comparing brightness values of images to be corrected according to the exemplary embodiment of FIG. 1 .
FIG. 4 is pixel coordinates for explaining the concept of the shadow restoration method of FIG. 2 .
5A and 5B are exemplary views for explaining shadow region restoration according to the embodiment of FIG. 1 .
6 is a diagram illustrating a process of restoring a shadow region according to the embodiment of FIG. 1 .
FIG. 7 is a view for explaining a process of shadow correction according to the embodiment of FIG. 1 .
8A and 8B are diagrams for explaining a result of shadow correction according to the embodiment of FIG. 1 .

Hereinafter, the embodiments disclosed in the present specification will be described in detail with reference to the accompanying drawings, but the same or similar components are assigned the same reference numerals regardless of reference numerals, and overlapping descriptions thereof will be omitted. The suffixes "module" and "part" for the components used in the following description are given or mixed in consideration of only the ease of writing the specification, and do not have a meaning or role distinct from each other by themselves. In addition, in describing the embodiments disclosed in the present specification, if it is determined that detailed descriptions of related known technologies may obscure the gist of the embodiments disclosed in the present specification, the detailed description thereof will be omitted. In addition, the accompanying drawings are only for easy understanding of the embodiments disclosed in the present specification, and the technical spirit disclosed herein is not limited by the accompanying drawings, and all changes included in the spirit and scope of the present invention , should be understood to include equivalents or substitutes.

When a component is referred to as being “connected” or “connected” to another component, it is understood that the other component may be directly connected or connected to the other component, but other components may exist in between. it should be On the other hand, when it is mentioned that a certain element is "directly connected" or "directly connected" to another element, it should be understood that the other element does not exist in the middle.

The singular expression includes the plural expression unless the context clearly dictates otherwise.

In addition, throughout this specification, when a part "includes" a certain component, it means that other components may be further included, rather than excluding other components, unless otherwise stated.

It will be apparent to those skilled in the art that the present invention may be embodied in other specific forms without departing from the spirit and essential characteristics of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the drawings, like reference numerals refer to like elements.

How to restore shadows

1 is a flowchart illustrating a shadow restoration method according to an embodiment of the present invention. FIG. 2 is pixel coordinates for explaining the concept of a shadow restoration method according to the embodiment of FIG. 1 of the present invention.

Referring to FIG. 1 , the shadow restoration method of the present invention includes the steps of receiving an image including a shadow region and a normal region, correcting the shadow region with reference to the gamma value of the normal region, the shadow region and the normal region calibrating the boundary region. In particular, the shadow restoration method of the present invention does not correct only the shadow region, but additionally corrects the boundary region between the shadow region and the normal region, thereby enabling more accurate correction of the shadow region.

Referring to FIG. 2 , shadow restoration requires a process of changing a pixel value from a shadow region to a non-shadow region. All pixels in the shadow area should be changed to a normal area. This is done by adjusting the brightness value of the image. In the present embodiment, brightness adjustment of an image is basically performed by gamma correction, and in particular, shadows are restored based on multi-scale gamma correction (MSGC). Structurally, the pixel distribution in the shadow area is raised to a level similar to the pixel distribution in the normal area. Specific details about this will be described.

Correction Step 1 : Correcting the Shadow Area

FIG. 3 is a graph comparing brightness values of images to be corrected according to the exemplary embodiment of FIG. 1 . FIG. 4 is pixel coordinates for explaining the concept of the shadow restoration method of FIG. 2 . 5A and 5B are exemplary views for explaining shadow region restoration according to the embodiment of FIG. 1 . 6 is a diagram illustrating a process of restoring a shadow region according to the embodiment of FIG. 1 .

Referring to FIG. 3 , in general, a digital image is composed of an RGB color model composed of red, green, and blue channels, which are three elements of light. Each group of pixels is classified into red, green, and blue color channels, and different corrections are performed according to the color channels.

The content of the correction of the shadow region is to correct the distribution of gamma values for each color channel of the corrected shadow region to be similar to the distribution of gamma values for each color channel of the normal region.

Referring back to FIG. 3 , pixel values for each color channel in the general area have different distributions. Looking at the distribution for each color channel, Red is at 250, Green is at 110, and Blue is at level 90. In the case of the red channel, the normal area is at level 250, and the shadow area is at level 165. Therefore, this difference is about 85, and the expected correction ratio of the gamma value can be 1.515. Therefore, if a correction ratio of gamma value of 1.515 is applied at the 165 level in the shadow area, it is corrected to the 250 level in the normal area.

Similarly, in the case of the green channel, the normal area is level 110 and the shadow area is level 20. Therefore, this difference is about 90, and the expected correction ratio of the gamma value can be about 5.5. In the case of the blue channel, the normal area is level 90, and the shadow area is level 20. This difference is about 70, and the expected correction ratio of the gamma value can be 4.5.

Therefore, since the difference between the normal area and the shadow area is not the same in each color channel, when correcting at the level of the shadow area and the normal area is performed by correcting one gamma value, proper correction cannot be made, and correction is performed for each color channel. Apply the gamma value correction ratio for

The gamma correction ratio for each channel is inversely calculated so that the representative value of the distribution of gamma values for each color channel in the corrected shadow region is the same as the representative value for the distribution of gamma values for each color channel in the normal region, and the gamma correction ratio for each color channel is applied to correct the shadow area.

은 각각 적, 녹, 청 채널을 의미한다. 식 2에서

는 감마값으로 각 채널별로

,

,

을 각각 추정한다. 또한, 그림자 영역을

, 그림자 영역이 복원된 영역이

라고 정의하여, 그림자가 복원된 영상이

라고 할 때 log를 취하면 식 3과 같이 각 채널별 감마값

를 추정할 수 있다.If the gamma value for each channel is set in Equation 1, it is the same as Equation 2, where

denotes red, green, and blue channels, respectively. in Equation 2

is the gamma value for each channel.

,

,

are estimated respectively. Also, the shadow area

, the area where the shadow area is restored is

By definition, the image in which the shadow is restored is

If we take the log, gamma value for each channel as shown in Equation 3

can be estimated.

를 추정하는데, 식 4와 같다. 여기서,

는 도 5a 및 도 5b의 그림자 영역 복원 전 영상에서 각 채널별로 분포된 화소값들에 대한 표준편차이다.Next, the area where the shadow area was restored

To estimate , it is the same as Equation 4. here,

is the standard deviation of pixel values distributed for each channel in the image before shadow region restoration of FIGS. 5A and 5B .

의 값들을 두배로 취하면 도 5a 및 도 5b의 그림자 복원 후와 같은 결과를 얻을 수 있으며, 이는 식 3의 감마값

를 추정하기 위한 입력값인 그림자 영역이 복원된 영역

를 획득할 수 있다. Therefore, the calculated standard deviation

If the values of are doubled, the same result as after shadow restoration of FIGS. 5A and 5B can be obtained, which is the gamma value of Equation 3

The area in which the shadow area, which is the input value for estimating

can be obtained.

Referring to FIG. 6 , in the original image (a), a shadow area and a normal area are expressed. In the case of processing as one gamma value (b), information on pixel values of each area is unnaturally corrected around the edge of the shadow area and the normal area.

In the case of correction through multiple channels (c), since the pixel levels of the shadow region and the normal region for each channel are different, it can be seen that a more natural correction is made when the gamma value correction is performed for each color channel.

Correction 2nd step: Boundary area correction

FIG. 7 is a view for explaining a process of shadow correction according to the embodiment of FIG. 1 . 8A and 8B are diagrams for explaining a result of shadow correction according to the embodiment of FIG. 1 .

Referring to FIG. 7 , the step of correcting the boundary area between the shadow area and the normal area includes defining a boundary area (Bimage) that is a boundary between the shadow area and the normal area and setting the boundary area with reference to surrounding image pixels. including restoring. If this process is described in detail, it is divided into a step of correcting the shadow area (S200) and a step of largely correcting the boundary area (Bimage).

First, looking at the process of correcting the shadow region, an image or image (RGB) including the shadow region and the normal region is provided ( S100 ). Then, the boundary edge image (cMask) of the shadow region and the normal region is extracted, and the standard deviation on the boundary edge image (cMask) is obtained. Using this, a multiscale gamma correction value is calculated. Using this, the shadow area is finally corrected (S200).

Here, the boundary edge (cMask) means a crossed edge mask, and an image may be input based on a user.

Now, looking at the process of correcting the area, after forming the shadow area image (sMask), the boundary area (Bimage) of the shadow area is extracted. Using this, the boundary area Bimage is corrected based on the image in which the shadow area is corrected (S300). Through this, the image (RGB') in which the shadow is finally restored can be obtained.

Referring to FIGS. 8A and 8B , the image change step for each step may be described. Each image goes from left to right to correct the shadows.

Referring to FIG. 8A , the first step is an image representing an original image (RGB) including a shadow. Using this, a shadow area image (sMask) defining a shadow area can be extracted. The boundary area image (Bimage) is extracted using the shadow area image (sMask). Through this, the boundary area between the shadow area and the normal area can be distinguished.

Referring to FIG. 8B , the boundary edge image cMask may acquire information about the boundary edge between the shadow area and the normal area. Thereafter, the multi-channel gamma correction (MSGC) image or image may identify an image in which the shadow region is corrected. It can be seen that an unnatural image boundary appears in the boundary region. This multi-channel gamma correction (MSGC) image or image is corrected using the boundary region image (Bimage). Afterwards, it is possible to finally implement an image or image (RGB') in which the shadow is completely corrected.

user-based interface

A plurality of shadow regions may exist in one image. Rather than correcting it as a whole, if users can select and specify a shadow to be corrected, a more desired method of correcting the image can be provided.

Therefore, in this case, in addition to the previously mentioned shadow correction method, a step of receiving an input of a shadow region to be corrected may be added to implement this.

The present invention is not limited by the above embodiments and the accompanying drawings. For those of ordinary skill in the art to which the present invention pertains, it will be apparent that the components according to the present invention can be substituted, modified and changed without departing from the technical spirit of the present invention.

-

Claims (5)

receiving an image including a shadow area and a normal area;
correcting the shadow region with reference to the gamma value of the normal region;
correcting a boundary area between the shadow area and the normal area;
A shadow restoration method that includes. According to claim 1,
The correction of the shadow region comprises correcting a distribution of gamma values for each color channel of the corrected shadow region to be similar to a distribution of gamma values for each color channel of the normal region. 3. The method of claim 2,
The step of correcting the shadow area is
The gamma value correction ratio for each channel is inversely calculated so that the representative value of the distribution of gamma values for each color channel in the corrected shadow region is the same as the representative value for the distribution of gamma values for each color channel in the normal region. A shadow restoration method, characterized in that correcting the shadow area by applying . According to claim 1,
The step of correcting the boundary area between the shadow area and the normal area includes:
defining a boundary region that is a boundary between the shadow region and the normal region; and
and restoring the boundary region with reference to surrounding image pixels. receiving an image including a normal area and a plurality of shadow areas;
receiving an input of a shadow region to be corrected among the plurality of shadow regions;
correcting the selected shadow region with reference to the gamma value of the normal region;
correcting a boundary area between the selected shadow area and the normal area;
A shadow restoration method that includes.

Images