KR101186153B1 - Thermal- infrared based detection of the bad weather real-time vehicle method - Google Patents

Abstract

PURPOSE: A method for classifying detection of cars in real time based on a thermal infrared ray during bad weather is provided to easily detect cars by analysis of a thermal infrared ray camera. CONSTITUTION: The average of each pixel is obtained in a frame unit of a space time image of a detection area. A standard deviation is obtained about each pixel. A standard deviation smoothening value of a different image of t-1 time and a standard deviation smoothening value of the space time image are calculated by smoothening the standard deviation. A technique for recognizing a car of the space time image is applied to a thermal image camera. A car is recognized using the standard deviation smoothening value of the car image and the standard deviation smoothening value of the space time image. The car is recognized under bad weather.

Description

Thermal-infrared based detection of the bad weather real-time vehicle method}

The present invention relates to a vehicle recognition technology, and more particularly, statistics of a spatiotemporal image obtained by a thermal infrared camera for constructing a system capable of solving a problem of false detection and having a more accurate traffic detection performance. The present invention relates to a real-time vehicle detection classification method based on an applied and improved algorithm.

Spatio-temporal image analysis techniques acquired by CCD cameras are difficult to detect with many analysis processes in order to prevent the detection of the next lane or to extract the exact shape of the vehicle under the influence of shadow over time even in normal weather.

On the other hand, thermal infrared cameras do not have much more constraints on weather conditions than CCTV cameras (e.g. shadow effects, etc.), but the shape of the vehicle does not appear clearly under heavy weather conditions such as fog, rain, and snow. Recognition of two to three, or appear so faint that it can not be distinguished from the road image is a problem that can not be detected incorrect detection.

On the other hand, the Spatio-Temporal Image Analysis technique draws a horizontal virtual detection line for each lane on the road and separates only the image passing through the detection line according to time to detect the detection line according to time (t). The shape of the vehicle is analyzed by accumulating the image of the x-axis) and the image generated for each frame as an image (car image) distinguishing the vehicle from the road.

Since the spatiotemporal image analysis technique detects and analyzes the color characteristics of the pixel data of the road and vehicle images over time, the vehicle is intact in general weather conditions where there is a large difference in luminance between the road and the vehicle. The detection rate is high, but it shows a high detection rate, but it is difficult to extract the shape of an intact vehicle because there is not much difference in luminance under poor weather conditions.

FIG. 1 is a diagram illustrating a virtual detection line setting of a road. Referring to FIG. 1, a yellow detection line is virtually set based on a second left lane.

As an example, the width of the detection line is set to W = 105 and the height of height is set to H = 1. For example, when the yellow detection line is viewed in pixels, it can be seen that the vehicle over the current time (t) in the above figure becomes as shown in the upper part of FIG. 2.

FIG. 2 is a diagram for explaining space-time image matching over time, that is, in the upper table of FIG. 2, it is assumed that a gray part is a pixel of a vehicle and a white part is a pixel of a road.

When the yellow detection line pixel (1 x 105) is placed vertically, the 0th pixel is considered as the top, and if this pixel is matched to any space-time image data pixel (H x W = 104 xn), the space-time image in frame unit is displayed. Can be generated.

The time is directed from the left pixel to the right pixel and the current time is the rightmost pixel.

If there are more than n spatio-temporal image data, it is deleted and updated by pushing one frame to the left, and the rightmost pixel always fills the pixel data of the current time.

If the image reflects this, it is possible to generate the spatiotemporal image as shown in Figure 3 (space-time image generation screen).

A method of generating a space-time image (reference background image) of a road without a vehicle will be described below. Based on the yellow detection line, the standard deviation of the pixel (1 ㅧ 105) of the pure road image can be seen that the difference is not large when comparing the difference between the pixel of the current frame and the immediately preceding frame.

In addition, even if the average value of the pixel data is compared, the difference is not large.

However, as the vehicle passes by, the pixel deviations and averages are much different from those of the previous frame. Therefore, it can be judged as a road image if the pixel has a uniform deviation and average of 20 to 30 frames or more.

In Equations 1 and 2

Is available,

Generating a background image may be performed as in Equations 3 and 4 below. (However, the formula applied here is H = 1.)

와

처럼 값을 비교하여 각각 그 값이 차이를 보이지 않으면 현재시간(t)의 프레임은 배경영상(B_f(x,y))으로 판단한다. 20~30 프레임의 연속된 배경영상(B_f(x,y))들을 평균하면 배경 영상인 B_avg(x,y)을 얻을 수 있다. 이것을 H=105이고 W=n 인 시공간 배경 영상 데이터 픽셀에 도 2의 방법으로 매칭을 하면 도 4와 같은 시공간 배경영상을 생성할 수 있다. 도 4는, 시공간 배경영상 생성 화면을 나타낸다.

Wow

If the values are not different from each other, the frame of the current time t is determined as the background image B _f (x, y). By averaging successive background images B _f (x, y) of 20 to 30 frames, B _avg (x, y) can be obtained. If this is matched to the space-time background image data pixels having H = 105 and W = n by the method of FIG. 2, the space-time background image as shown in FIG. 4 may be generated. 4 shows a space-time background image generation screen.

Meanwhile, the comparison between time-space image and thermal infrared camera image for each weather will be described. 5 illustrates a space-time image and a thermal infrared camera image of a general weather condition, and in the case of a general weather, the shape of the vehicle is clearly shown regardless of day or night.

6 shows a space-time image and a thermal infrared camera image of a snowy weather condition, it is difficult to distinguish a road from a vehicle even if the image is acquired by a thermal infrared camera in a snowy weather.

FIG. 7 shows a spatiotemporal image and a thermal infrared camera image of a weather condition in which rain falls, it is easier to distinguish a road from a vehicle than snow in case of rain, but heat is reflected by standing rainwater on a road surface.

FIG. 8 shows a space-time image and a thermal infrared camera image of a misty weather condition. The misted image is faint than rain but is easier to distinguish a vehicle than a snowy image.

As shown in the above figures, when it is difficult to distinguish between the road and the vehicle, a method of analyzing and detecting the characteristics that can distinguish the vehicle should be used.

The present invention proposes a technique to reduce the occurrence of false detections by extracting the shape of a vehicle by improving the current spatiotemporal image analysis technique, and to describe based on the image of snow falling which is the worst condition for detection.

[Related Technical Literature]

1. Traffic information prediction with complex detection system Complex detector and traffic information prediction method (Patent application number: 10-2008-0050264)

The present invention is to solve the above problems, generally when the analysis using a thermal infrared camera rather than a time-space image analysis using a widely used CCTV camera is not affected by the shadow that has been a problem so that the detection of the vehicle easily It is to provide a real-time vehicle detection classification method in case of bad weather based on thermal infrared rays to improve the detection performance faster by reducing the number of analysis processing elements.

In addition, the present invention is a system that can have a more accurate traffic detection performance by solving the problem of false detection caused by the thermal infrared camera is independent of the domestic four seasons and bad weather environment such as snow, rain, fog, etc. It is to provide a real-time vehicle detection classification method in bad weather based on thermal infrared to build.

In addition, the present invention further extracts the exact shape of the vehicle for bad weather conditions to detect real-time vehicle in case of bad weather based on heat infrared ray to provide more accurate data in detecting not only traffic detection but also detection of intersection. It is to provide a classification method.

In order to achieve the above object, a real-time vehicle detection classification method based on thermal infrared rays according to an embodiment of the present invention includes: a first step of calculating a standard deviation smoothing equation of each pixel for a difference image; A second step of performing vehicle recognition by applying a vehicle recognition technique of a space-time image to a thermal imaging camera; And a third step of performing vehicle recognition in bad weather conditions. Characterized in that it comprises a.

)과 시공간 영상의 표준편차 평활화값(

)을 연산하는 것을 특징으로 한다.According to another embodiment of the present invention, in the thermal infrared based bad weather real-time vehicle detection classification method, in the first step, the standard deviation smoothing equation is a standard deviation and smoothing equation for the difference image, the spatiotemporal image of the detection area Calculate the average (avg _r (t)) of each pixel (WH) in units of frames, and again obtain the standard deviation (σ _r (t)) for each pixel, and smooth the standard deviation (σ _r (t)). Standard deviation smoothing value of the difference image

) And standard deviation smoothing values of spatiotemporal images

) Is calculated.

)과 시공간 영상의 표준편차 평활화값(

)을 이용해 차량인식을 수행하는 것을 특징으로 한다.In the real-time vehicle detection classification method based on thermal infrared rays according to another embodiment of the present invention, the second step is the standard deviation smoothing value (

) And standard deviation smoothing values of spatiotemporal images

It is characterized by performing a vehicle recognition using).

According to another embodiment of the present invention, in the heat-infrared-based bad weather real-time vehicle detection classification method, in the second step, vehicle detection start,

에 의해 연산하며,

Computed by

에 의해 검지하는 것을 특징으로 한다.Vehicle detection tip,

It is characterized by detecting by.

인지를 계속 확인하고

의 차이가 ± 0.05의 차이를 보일 때까지 검지하는 것을 특징으로 한다.
In a real-time vehicle detection classification method based on thermal infrared rays according to another embodiment of the present invention, the third step may include the next frame even if the detection end is found to prevent the false detection of the end of the vehicle detection.

Continue to check

It is characterized by detecting until a difference of ± 0.05 shows a difference.

In the real-time vehicle detection classification method according to an embodiment of the present invention, when the analysis is performed using a thermal infrared camera rather than a spatiotemporal image analysis using a widely used CCTV camera, the detection of the vehicle is not affected by the shadow that has been a problem. It facilitates and reduces the number of analysis processing elements, thus improving the detection performance faster.

In addition, the real-time vehicle detection classification method according to another embodiment of the present invention, the characteristics of the thermal infrared camera is independent of the domestic four seasons and bad weather environment such as snow, rain, fog, but to solve the problem of false detection It provides the effect of having more accurate traffic detection performance.

In addition, the real-time vehicle detection classification method according to another embodiment of the present invention further extracts the exact shape of the vehicle for bad weather conditions, and more accurate data for not only traffic detection but also detecting the notice of intersection. Provides the effect that can be provided.

1 is a diagram showing a virtual detection line setting of a road.
2 illustrates space-time image matching over time.
3 is a space-time image generation screen.
4 is a space-time background image creation screen.
5 is a space-time image and a thermal infrared camera image of general weather conditions.
6 is a space-time image and a thermal infrared camera image of snowy weather conditions.
7 is a space-time image and a thermal infrared camera image of the weather conditions of the rain.
8 is a space-time image and a thermal infrared camera image of the foggy weather conditions.
9 is a vehicle image of a general weather condition.
10 is a background image.
11 is a space-time image.
12 is a view for explaining a standard deviation smoothing of a time difference image (time-space image-background image) over time.
FIG. 13 is a diagram for describing vehicle detection using a difference image of a space-time image in a simple general weather condition using FIG. 12 as an example.
14 is a vehicle image of poor weather conditions.
15 is a reference background image.
16 is a space-time image.
FIG. 17 is a diagram illustrating a standard deviation smoothing of a time difference image (spatial space image-background image) over time. FIG.
FIG. 18 is a diagram for describing vehicle detection using a difference image of a spatiotemporal image by using FIG. 17 as an example.
19 is a diagram illustrating vehicle detection using a difference image of an improved space-time image.
20 illustrates vehicle region extraction.

Hereinafter, a detailed description of a preferred embodiment of the present invention will be described with reference to the accompanying drawings. In the following description of the present invention, detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear.

In the present specification, when any one element 'transmits' data or signals to another element, the element can transmit the data or signal directly to the other element, and through at least one other element Data or signal can be transmitted to another component.

FIG. 9 is a diagram illustrating a real-time vehicle detection classification method based on thermal infrared rays in bad weather according to an exemplary embodiment of the present invention.

The vehicle extraction method by statistical characteristic analysis of a general space-time image can be analyzed as follows from the space-time image extracted by the yellow detection line based on the image of FIG. 3.

First, a standard deviation smoothing calculation formula of each pixel with respect to the difference image is calculated. If the standard deviation of the pixel value for the car image is obtained as described below, the deviation of the change in the number of pixels of the vehicle and the road surface may be known, and the vehicle and the road may be distinguished based on this.

However, the calculated standard deviation value may cause an error if there is a low standard deviation value in the vehicle. Thus, in order to reduce the error, the current time t, the previous time t-1, and the next time are reduced. Standard deviation smoothing is performed for (t + 1).

을 구하여 적용한다.
Since Equation 3 does not know the value of the next time (t + 1), it finds the time (t-2) like Equation 4

Obtain and apply

Equations 5 to 6 are formulas-a standard deviation and smoothing calculation formula for the difference image, and the average (avg _r (t)) of each pixel WH is obtained from the space-time image of the detection area in units of frames, and again for each pixel. Find the standard deviation (σ _r (t)).

)과 시공간 영상의 표준편차 평활화값(

)을 비교하면 시공간 영상의

값의 변위가 더 높다는 것을 알 수 있다.When smoothing this value, it is calculated as Equation 7 or Equation 8. The standard deviation smoothing value of the difference image of t-1 time (

) And standard deviation smoothing values of spatiotemporal images

) Compares the time-space image

It can be seen that the displacement of the value is higher.

This does not affect much under normal weather conditions.

Secondly, vehicle recognition is performed by applying a conventional vehicle recognition method of a spatiotemporal image using Equation 7 or Equation 8 to a thermal imaging camera.

FIG. 10 is a diagram illustrating a background image. The background image creates an image by a method of generating a space-time image (reference background image) of a road without a vehicle as described above.

FIG. 11 is a diagram illustrating a space-time image. In general weather conditions, a distinction between a road and a vehicle is clearly shown. Also, the shadow effect of the CCD camera does not appear.

), 녹색선은, 기준 배경 영상의 각 화소에 대한 평균값의 표준편차 avg(σ_back)를 나타낸다.12 is a diagram illustrating standard deviation smoothing of a time difference image (spatial space image-background image) over time, and yellow lines represent standard deviation smoothing of the difference image (

), The green line represents the standard deviation avg (σ _back ) of the mean value for each pixel of the reference background image.

When the start and end of the vehicle is a pure road image (reference background image), the rate of change of the standard deviation is set as the threshold of the start (S _th ) and end (E _th ) of the vehicle by the following equation.

The vehicle detection start is calculated by the following equation (9).

The vehicle detection end is calculated by the following equation (10).

은 배경영상의 화소값에 대한 표준편차를 나타내는 것이다.In the above formula

Denotes a standard deviation of pixel values of a background image.

When analyzing the smoothing graph of the difference image standard deviation, the front part of the vehicle suddenly rose to the highest value due to the radiant heat, and the background average standard deviation and the slight deviation were found at the rear part of the vehicle. It shows a convergence phenomenon.

The middle part of the vehicle shows a lot of deviation from the radiant heat part of the dashboard, and the middle part of the vehicle shows a lower value than the background average standard deviation. Therefore, the threadhold value should be 3 to 5 from the background average standard deviation value.

FIG. 13 is a graph illustrating vehicle detection using a difference image of a space-time image in a simple general weather condition using FIG. 12 as an example. In general general weather conditions, as shown in Figure 13, the distinction between the vehicle and the road can be clear so that the detection of the vehicle is not difficult.

Next, vehicle recognition by applying a conventional vehicle recognition technique of a spatiotemporal image in poor weather conditions will be described.

14 shows a vehicle image in poor weather conditions. As shown in FIG. 14, the vehicle and the road in snowy weather cannot be clearly distinguished.

) 를 나타내면, 녹색선은, 기준 배경 영상의 각 화소에 대한 평균값의 표준편차 avg(σ_back)을 나타낸다.15 illustrates a reference background image, and FIG. 16 illustrates a space-time image. As shown in the spatio-temporal image of FIG. 15, it is difficult to distinguish the vehicle because the luminance difference between the road and the vehicle is not large. Here, the yellow line is the standard deviation smoothing of the difference image (

), The green line represents the standard deviation avg (σ _back ) of the mean value for each pixel of the reference background image.

17 illustrates standard deviation smoothing of a time difference image (spatial space image-background image) over time.

Meanwhile, looking at the improved vehicle extraction technology of the spatiotemporal image, in the snowy weather condition as shown in FIG. 17, three pairs of the start and the end of the vehicle exist for the first vehicle compared to the general weather condition. This causes a false detection because there is a section where the standard deviation in the middle of the vehicle having no luminance difference with the background is lower than or equal to E _th of the standard deviation of the mean value of the background. Conventional detection techniques using spatio-temporal image analysis are difficult to apply to such weather conditions.

인 시점이며, 검지 끝은

이 되는 시점까지 검지한다.FIG. 18 is a view illustrating a vehicle detection using a difference image of a spatiotemporal image by using FIG. 17 as an example. The above description is taken as an example of the simple graph of FIG. 17.

At the end of the index finger

It detects until this time.

은 이전프레임 차량의 화소값에 대한 표준편차를 나타낸 것이다.In the above formula

Is the standard deviation of the pixel values of the previous frame vehicle.

S (t-n ') and E (t-n') were detected again after the start of the vehicle (S (tn)) and the end of the vehicle (E (tn)). The end of the vehicle cannot be determined only by setting the threshold value E _th as the section is a false detection section.

For this section, another threshold setting range for distinguishing the vehicle is required.

FIG. 19 is a graph illustrating an improved vehicle detection using a differential image of a space-time image. As shown in FIG. 19, the average value of the reference background standard deviation of a road image without a normal vehicle has almost no change in displacement over time. Even when compared with the deviation smoothing value, the standard deviation smoothing value of the misdetection section (middle of the vehicle) is always larger than the mean value of the reference background standard deviation.

For this section, another threshold setting range for distinguishing the vehicle is required.

인지를 계속 확인하고

의 차이가 ± 0.05의 차이를 보일 때 까지(파란색 검지 시간) 검지한다. Even if the end of detection is found, the next frame

Continue to check

Detect until the difference of ± 0.05 shows (blue detection time).

은 배경영상의 화소값에 대한 표준편차를 나타내며,

는 이전프레임 차량의 화소값에 대한 표준편차를 나타낸 것이다.In the above formula

Indicates the standard deviation of the pixel values of the background image.

Is the standard deviation of the pixel values of the previous frame vehicle.

At that time, the second end time is determined by following the previous frame in the direction of the red arrow of FIG. 19 to determine the end of the vehicle.

If detected based on the above content analysis can be detected with one vehicle as shown in FIG.

As described above, the specification and the drawings have been described with respect to the preferred embodiments of the present invention, although specific terms are used, it is only used in a general sense to easily explain the technical contents of the present invention and to help the understanding of the invention. It is not intended to limit the scope of the present invention. It will be apparent to those skilled in the art that other modifications based on the technical idea of the present invention can be carried out in addition to the embodiments disclosed herein.

Claims (3)

The standard deviation smoothing equation of each pixel for the difference image according to the pixel change between the current frame and the previous frame is a standard deviation and smoothing equation for the difference image. The average (avg _r (t)) of each pixel WH is calculated, and the standard deviation (σ _r (t)) is obtained for each pixel, and the standard deviation (σ _r (t)) is smoothed. Standard deviation smoothing value of the difference image of 1 hour

) And standard deviation smoothing values of spatiotemporal images

Calculating a first step; A standard deviation smoothing value of the difference image is applied to a thermal imaging camera using a vehicle recognition technique of a space-time image.

) And standard deviation smoothing values of spatiotemporal images

A second step of performing vehicle recognition using; And a third step of performing vehicle recognition in bad weather conditions. In the thermal infrared based real-time vehicle detection classification method that comprises a,
In the second step,
Vehicle detection starts,

Computed by
Vehicle detection tip,

Is detected by (σback is the standard deviation of the pixel value of the background image).
In the third step,
To prevent false detection of the end of vehicle detection, the next frame is detected even if the end of detection is found.

Continue to check

Real-time vehicle detection classification method based on thermal infrared rays, characterized in that the detection until the difference between and within ± 0.05. delete delete

Images