KR101684990B1

KR101684990B1 - Method for deblurring vehicle image using sigma variation of Bilateral Filter

Info

Publication number: KR101684990B1
Application number: KR1020150110061A
Authority: KR
Inventors: 김희석; 손휘곤
Original assignee: 청주대학교 산학협력단
Priority date: 2015-08-04
Filing date: 2015-08-04
Publication date: 2016-12-12

Abstract

The present invention relates to a vehicle using a bidirectional filter for correcting a Blur image of ADAS (Advanced Driver Assistance Systems) by using a bilateral filter using histogram equalization and deviation A vehicle image debloring method using a sigma parameter of a bidirectional filter according to the present invention is composed of five steps in the order of image scale, smoothing, noise filter, and contour extraction.

and

Value is extracted in accordance with the running phenomenon of the road in ADAS, and the blur of 10 pixels or less is processed faster than the conventional methods.

Description

[0001] The present invention relates to a vehicle image deblurring method using a sigma parameter of a bidirectional filter,

The present invention relates to an image processing method, and more particularly, to a blur image of ADAS (Advanced Driver Assistance Systems) using a bilateral filter using histogram equalization and deviation. The present invention relates to a vehicle image deblurring method using a sigma parameter of a bidirectional filter to be corrected.

ADAS (Advanced Driver Assistance Systems) is an advanced driver assistant system (LDAS) that displays LDW (Lane Departure Warning System) to indicate lane departure and travel path, AVM (Around View (Tire Pressure Monitoring System), PAS (Parking Assistance System), ACC (Advenced Smart Cruse Control), NVS (Night Vision System, Night Vision System), BSD (Blind Spot Detection), and so on, to help drivers to drive safely.

Such applications are mainly performed through image processing through a camera and sensor values. When an image is recognized through a camera, a blur occurs in which the image is blurred due to camera shake, out-focusing, and fast motion of the subject . In most cases, when blurring occurs, clear images are obtained by re-shooting. However, in the case of a vehicle image requiring fast processing by recording a specific moment, sharp and accurate images without blur are required every moment Deblurring techniques are essential.

The deblur method using image processing is divided into frequency domain correction using FT (Fourier transform), correction using a motion vector, and correction using a sensor.

First, in the frequency domain, the non-blind deconvolution with the blur kernel and the blind deconvolution (blur kernel), in which the blur kernel is not given, Blind deconvolution).

Since the blur kernel is not given at the time of actual image capturing, a large amount of computation is required because a PSF (Point Spread Function) is restored by using a blind deconvolution method. The processing speed does not follow the video transmission speed.

Next, a correction method using a motion vector is a method of finding and recovering a shaky direction through one or several sample windows. Since the method of locating the blur direction is partially obtained and stored in a small number of bins, it is faster than the method of obtaining the PSF in the frequency domain. However, .

Finally, the sensor-based method is very accurate in the correction of physical shaking such as camera shake, because of low computational complexity. However, since it is necessary to provide additional sensor in order to cope with occasional motion blur in out-focusing or low-light condition, There are disadvantages.

KR 10-2014-0017000 A (2014.02.10) KR 10-2014-0034825E (2014.03.20) KR 10-2014-0141392 A (2014.12.10)

Mey Chen, Todd Jochem, and Dean Pomerleau, "AURORA: A Vision-Based Roadway Departure Warning System," Proceedings of the IEEE, Vol. 1, pp. 243-248, 1995 Y. C. Liu, K. Y. Lin and Y. Chen, "Bird's-eye view vision system for vehicle surrounding monitoring," RobVis 2008, LNCS 4931, pp. 207-218, 2008 Ho Jin Jo, Seung Yong Lee, "Research Trends of Image Deblurring," Journal of IEEK, 39 (10), pp.25-35, 2012 Jan Kotera, Filip Sroubek, Peyman Milanfar, "Blind deconvolution using alternating maximum a posteriori estimation with heavy-tailed priors," Computer Analysis of Images and Patterns, Vol.8048, pp. 59-66, 2013 Shengyang Dai, Ying Wu, "Motion from blur," Proceedings of the IEEE, pp. 1-8, 2008 Neel Joshi, Shing Bing Kang, C. Lawrence Zitnick, Richard Szeliski, "Image deblurring using inertial measurement sensors," Journal of the ACM Transactions on Graphics, Vol 29, 2010 Jae-Yeong Lee, Wonpil Yu, "Robust Estimation of Edge Density in Blurred Images," International Conference on Ubiquitous Robots and Ambient Intelligence, pp.521-524, 2012 Senthikumaran N, Thimmiaraja J, "Histogram Equalization for Image Enhancement Using MRI brain images," Computing and Communication Technologies, pp80-83, 2014 N. Ravia Shabnam Parveen, Dr.M. Mohammed Sathik, "Enhancement of Bone Fracture Images by Equalization Methods," Computer Technology and Development, pp391-394, 2009 Yadav, G, Maheshwary, S, Agarwal, A., "A Contrast limited adaptive histogram equalization based enhancement for real time video system," Advances in Computing, Communications and Informatics, pp.2392-2397, 2014 Chin-Chen Chang, Ju-Yuan Hsiao, Chih-Ping Hsieh, "An Adaptive Median Filter for Image Denoising," Intelligent Information Technology Application, pp.346-350, 2008 Archana H. Sable, K. C. Jonadhale, "Modified Double Bilateral Filter For Sharpness Enhancement And Noise Removal," Advances in Computer Engineering, pp. 295-297, 2010 Bing Wang, ShaoSheng Fan, "An Improved CANNY Edge Detection Algorithm," Computer Science and Engineering, pp.497-500, 2009 Chunxi Ma, Wenshuo Gao, "An improved Sobel algorithm based on median filter," Mechanical and Electronics Engineering, Vol.1, pp. 88-92, 2010

As described above, the deblurer method using the image processing according to the related art has advantages, but it is difficult to process the vehicle image because the calculation is complicated because the specific time is recorded and the vehicle image requires fast processing.

Therefore, an object of the present invention to solve the above-mentioned problems is to provide a vehicular image de-interlacing system using sigma parameters of bidirectional filters capable of detecting only important feature points necessary for ADAS image recognition such as signs, pedestrians, Method.

According to an aspect of the present invention, there is provided a vehicle image debloring method using a sigma parameter of a bidirectional filter, the method comprising: a scale down step of reducing a scale of an image taken by a camera to a nearest neighborhood; Performing HE (histogram equalization) processing on the scaled-down image; Determining an amount of blur included in the HE-processed image according to Equation (13); If the amount of the blur determined according to the determination of the amount of blur is larger than the constant value, the weight for removing noise of the Gaussian distribution along the distance

And contour preservation weight

Performing bilateral filtering to remove noise while preserving contours of the image using the image; And extracting a contour of the image using a Sobel filter (Sobel Filter).

(13)

Where X is the amount of blur in the kth frame,

Is a constant value that determines the amount of blur.

In the present invention, when the values of Peak Signal to Noise Ratio (PSNR) are compared with respect to four image situations such as parking, city driving, highway and stagnant road, image processing is processed 8 times faster There is an advantage to be able to do.

Accordingly, the present invention has the effect of enhancing the performance of the vehicle ADAS requiring a high-speed processing by recording a specific moment and ensuring more safe operation for the driver.

1 is a flowchart of a vehicle image debloring method using a sigma parameter of a bidirectional filter according to the present invention,
FIGS. 2A to 2C are graphs and images obtained by comparing a blurred image and a corrected blurred image according to the present invention,
3A to 3C are diagrams showing an example image obtained by comparing a pre-correction contour line of a running image with a post-correction contour line according to the present invention,
FIGS. 4A to 4C show an example image obtained by comparing a pre-correction contour of a parking image with a contour after correction according to the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of a vehicle image debloring method using a sigma parameter of a bidirectional filter according to the present invention will be described in detail with reference to the drawings. It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are intended to provide further explanation of the invention as claimed.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are intended to provide further explanation of the invention as claimed. , &Lt; / RTI > equivalents, and alternatives. In the following description of the present invention, detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the relevant art and are to be interpreted in an ideal or overly formal sense unless explicitly defined in the present application Do not.

In the present invention, HE (Histogram Equalization) and a bilateral filter are used to improve image quality.

First, with respect to the HE, since the range of the color recognition is narrower than that of the human eye, when the recognition range of the color is narrow, the width of the histogram of the image is narrowed and eventually the low color contrast is obtained. In other words, the expression of color is not uniform, resulting in undesirable results in contour extraction. To overcome this problem, we use HE as a method to fix the intensity value of the image and remove the spike shape of the histogram.

HE is an ideal model in which the distribution of the cumulative sum is linear, and improves the image quality by linearly correcting the offset cumulative sum. The expression for HE is expressed by Equation (1) below, where r is the intensity in the pre-smoothed image and s is the smoothed intensity.

However, the HE method can enhance the image in the low color recognition range, but the color is not natural in some cases. Several algorithms have been proposed to overcome this problem. Among them, AHE (Adaptive Histogram Equalization) is a method of dividing an image into windows and applying HE to each window. Unlike the general HE, which applies the contrast value as a whole, it is a good correction method for expressing details such as natural or medical images because it applies local smoothing. However, there are cases where the amplification is over-geographically so that erroneous corrections can be made in some areas.

Another method, Contrast Limited Adaptive Histogram Equalization (CLAHE), is an image enhancement method that compensates for the disadvantages of AHE, which misinterprets the principal brightness value as a high level. CLAHE sets the limit value of brightness contrast as a clip parameter, and if the cumulative value is higher than the clip value, the clip value is cut off and accumulated in the other level. CLAHE provides a more natural image than AHE, but can generate distortion such as ringing. Therefore, HE is used mainly when distortion noise is generated.

Next, a bilateral filter will be described. In general, the nonlinear filter used for noise reduction is a median filter. The noise elimination of the median filter can improve the image post-processing result such as contour extraction by outputting the middle value when the pixel values in the mask are sorted in order of magnitude. However, noise can be removed and contour preservation is possible, but the shape of the object changes due to strong color spreading of the image. The median filter is expressed by the following Equation 2, w denotes a neighboring pixel, and y denotes a median value in [m, n].

In the present invention, a bidirectional filter using a Gaussian distribution is used to compensate for color bleeding of such a median filter. The bidirectional filter effectively preserves the contours through the weights using neighboring pixels and also removes noise. Therefore, in the present invention, the image recognition rate can be improved by using the bidirectional filter.

(5) of the bidirectional filter can be divided into a space D having a Gaussian distribution and a range R part of Equation (4) given below, and accordingly, a deviation value D of a contour

And noise removal

To determine the diffusion of the distribution. of (x _1, y ₁₎ represents the position of the current pixel to be corrected through the two-way filter N denotes a near neighbor pixel set of _{(x 1, y 1) (} x 2, y 2) is the pixel belonging to the N Position. I (x ₂ , y ₂ ) represents the pixel value when the pixel position is (x ₂ , y ₂ ).

In Equation (5), C is expressed by Equation (6) below and normalization of Equation (5) represents a new correction value I (x ₁ , y ₁ ).

As described above, since the existing deblurring methods focus on restoring images to a fine portion, there is a large amount of computation for real-time use. Therefore, in order to reduce the amount of computation when a blur occurs, such as a camera image shake or a subject movement, which occurs during driving of a vehicle, the blur below a reference value based on 10 pixels is reduced to 5 It is corrected through the process of STEP. FIG. 1 is a flowchart of a vehicle image debloring method using a sigma parameter of a bidirectional filter according to the present invention.

As shown in FIG. 1, when an image obtained from the camera is input in step 111, the scale size is reduced in step 113 (step 1: scale down). The scale size reduction in step 113 uses the nearest neighborhood in the image recognition. Although the image quality is deteriorated due to the image staircase phenomenon at the time of scale measurement, the calculation amount is reduced. In contour extraction, the image is reduced to a scale of 0.5 times higher than that of the original image.

Next, histogram smoothing is performed in step 115 of FIG. 1 (step 2: histogram equalization). In the histogram smoothing method, there are AHE and CLAHE based on the basic HE, and CLAHE, which is generally the most advanced form, is mainly used. However, in the present invention, the CLAHE method for performing smoothing for each sub- Apply the HE method to extract feature traces.

In step 117 of FIG. 1, bidirectional filtering is selectively performed according to the amount of blur included in the image processed up to step 115. The smoothed image in step 115 selectively performs a bidirectional filter according to the amount of blur through Equation (7).

In Equation (7), X is the amount of blur in the k-th frame, which is a constant value for determining the amount of blur

If less, it is determined as 'less blur'

If it is bigger, it is judged as 'much blur'. The amount of blur may vary depending on the running speed of the vehicle. Generally, in the case of a road environment where the vehicle must travel at a low speed, many nearby buildings and objects have a lot of noise. Accordingly, in this case, the flow advances to step 119, and bidirectional filtering is performed according to the determination of step 117 (step 3: Blur Decision).

On the other hand, in the case of a highway that needs to travel at high speed, since a lot of noise is removed by the speed, a constant value

The edge detection operation of step 121 is performed without performing step 119 (step 5: edge detection).

In step 119, bilateral filtering (STEP4: Bilayer Filtering) is a filter that removes noise while preserving the outline of the image. In order to accurately recognize the road environment, it is necessary to increase the contrast of colors at the boundary between the objects of the image, It is necessary to reduce unnecessary noise by lowering the color contrast. Therefore, you must specify a high deviation value for a clear extraction of the contour and a low deviation value to reduce color noise.

However, obtaining an appropriate deviation for the corresponding image for each frame of the image has a high calculation amount because it repeats complicated calculations.

Therefore, in the vehicle image deblurring method using the sigma parameter of the bidirectional filter according to the present invention, in the bidirectional filtering step,

Wow

To correct the contour and remove noise. Deviation over distance

Is a weight for removing noise of a Gaussian distribution,

Is a weight for preserving contours.

In the present invention,

Wow

And the result of Fig. 2 is used in consideration of the relevance of the image

The amount of computation can be reduced by fixing the value.

Wow

Can be obtained from the PSNR (Peak Signal to Noise Ratio) used as a method for comparing the degree of similarity of images. Equation (8) below expresses a formula when the PSNR is the maximum.

Where MSE is Mean Square Error and L is the maximum value of the intensity level.

MSE (Mean Square Error) in Equation (8) can be expressed by Equation (9) below. In Equation (9), A represents the original image and B represents the corrected image of the blur image.

In Equation (9), the MSE value is

Can be approximated by

The value can be expressed by Equation (10) below.

Here, N and M are the horizontal and vertical sizes of the image, respectively.

Here, m represents an average value.

Therefore, in order to maximize the PSNR value

The value should be minimized. However,

If the value is minimized, a contour due to noise is generated.

Value about contour

, For noise reduction

Respectively.

The value is appropriately determined to maximize the PSNR.

Wow

To specify a range of values, T (Threshold) must be specified, where T is a standard normal distribution (m = 0,

= 1), which is a deviation value of

= 1 is used.

Therefore,

Wow

Can be expressed by Equation (12) below. &Quot; (12) "

The bar graphs of Figures 2a-2c

The value of 0.1 to 5,

The PSNR value can be clearly distinguished from the data obtained by changing the value of 0.1 to 10 in increments of 0.1

and

As a result, the horizontal axis indicates

Values are 0.1, 1.1, 2.1, 3.1 5

1.1, 3.1, 5.1 7.1, and 9.1.

FIGS. 2A, 2B, and 2C are corrected image results obtained by performing steps 1, 2, 3, 4, and 5 described above. The PSNR values are plotted on the vertical axis, and within the specified range

= 0.1,

= 1.1, the maximum PSNR result was obtained. 2A, 2B and 2C are respectively shown for comparing the blurred image and the corrected de-blurred image for the vehicle image, the in-stream image and the road image, respectively.

In step 121, the edge detection (STEP 5) is performed after performing the bidirectional filtering in step 119 (step 4) or in step 117 without performing bidirectional filtering in step 117 according to the amount of blur identified in step 117. In step 121, . Canny filter is mainly used for contour extraction. The canny filter first removes the noise using the Gaussian filter, extracts the contour, extracts the minimum contour by applying non-maximum suppression using the direction and intensity of the tilt, and finally extracts the hysteresis The outline is determined. The Canny filter is an optimal contour extraction technique because it is determined by the trend of the contour line. However, in the Gaussian filter that removes the noise, not only the noise but also the contour is not preserved. In some cases, the strong noise that sometimes occurs is judged as the contour line, which causes a decrease in the recognition rate due to the connection with the normal contour line. Also, because the slope is used to determine the contour, the computation speed is significantly slower than other contour extraction filters.

Therefore, in the present invention, contour lines are extracted by using a Sobel filter which is easy to extract diagonal contour lines used mainly for vehicle image recognition processing and has a high calculation speed.

In order to verify the efficiency of the vehicle image deblurring method using the sigma parameters of the bidirectional filter according to the present invention, MATLAB R2012a tool was used in intel i5-3.1Ghz environment. Table 1 below shows the result of comparing the PSNR and the execution time using the blind deconvolution method according to the present invention and the conventional vehicle image classified by the situation that can occur on the road.

As shown in Table 1, when the values of PSNR are compared in the four image situations (parking, city driving, highway, stagnant road), the conventional blind deconvolution is compared with the de- And the processing time of the deblurring method according to the present invention is about 8 times faster at the execution time.

FIGS. 3A to 3C are exemplary images comparing the pre-correction contours of the highway image with the post-correction contours according to the present invention. FIGS. 4A to 4C show the comparison between the pre-correction contours of the city road images and the post- 3A and FIG. 4A are images formed by blurring on a highway and a city road, respectively. When the images are contour-extracted, contour extraction results as shown in FIGS. 3B and 4B are obtained, By applying the blur method and correcting it, improved outline extraction results can be obtained as shown in FIGS. 3C and 4C, respectively.

As described above, the vehicle image deblurring method using the sigma parameter of the bidirectional filter according to the present invention for correcting the blur image with five steps for image processing and recognition of the vehicle, However, it is possible to obtain a very fast processing result as compared with the conventional blind deconvolution method using a blind deconvolution, and to extend the scope of applying the deblur method by designing a hardware acceleration system and the like .

Although the preferred embodiments of the present invention have been disclosed for illustrative purposes, those skilled in the art will appreciate that various modifications, additions and substitutions are possible, without departing from the scope and spirit of the invention as disclosed in the accompanying claims. Therefore, the scope of the present invention should not be limited by the described embodiments, but should be determined by the scope of the appended claims and equivalents thereof.

Claims

A scale down step of reducing a scale of an image photographed by a camera to a nearest neighborhood;
Performing HE (histogram equalization) processing on the scaled-down image;
Determining an amount of blur included in the HE-processed image according to Equation (13);
If the amount of blur determined according to the determination of the amount of blur is larger than a constant value, a weight for removing noise of a Gaussian distribution according to the distance

And contour preservation weight

Performing bilateral filtering to remove noise while preserving contours of the image using the image; And
And extracting a contour of the image using a Sobel filter. The vehicle image deblurer according to claim 1,
(13)

Where X is the amount of blur in the kth frame,

Is a constant value that determines the amount of blur.
(14)

Where T is the standard normal distribution (m = 0,

= 1), which is a deviation value of

= 1.
(15)

2. The method of claim 1, wherein the scaling-
Wherein the scale of the original image is reduced to 0.5 times the scale of the original image.

delete

The method according to claim 1,

And

Quot;
In order to maximize the value of the PSNR (Peak Signal to Noise Ratio) expressed by Equation (16), which is used as a method for comparing the degree of similarity of images, the following Equations (17) and

Wherein the step of determining the vehicle image is performed based on the sigma parameter of the bidirectional filter.
(16)

Where MSE is Mean Square Error and L is the maximum value of the intensity level.
(17)

Here, A is the original image, B is the corrected image of the blurred image, and N and M are the width and height of the image, respectively.
(18)

Here, m represents an average value.