KR101647370B1 - road traffic information management system for g using camera and radar - Google Patents

Abstract

The present invention includes a camera and a radar detector and can detect various types of accurate traffic information by detecting object and object feature information from image information and radar signals, coordinate and time-synchronize image objects and radar objects, the accuracy of object detection can be significantly improved by matching a synchronized image object and a radar object using a homography matrix using a sample consensus algorithm, The HOG descriptor with different patterns is pre-set and stored. By detecting the pattern difference of the HOG descriptor using the nonlinear SVM (Support Vector Machine) classifier using the RBF kernel, the accuracy of the object detection is increased and the detected It is possible to detect the type of object, and the traffic information user interface And more particularly, to a traffic information management system using a camera and a radar that allows a user to easily and easily view traffic information by displaying an actual image, an incident tracking image, and a virtual map.

Description

[0001] The present invention relates to a traffic information management system using a camera and a radar,

The present invention relates to a traffic information management system using a camera and a radar. More specifically, the present invention relates to a traffic information management system using a camera and a radar, And a traffic information management system capable of dramatically increasing the accuracy of object recognition and providing various traffic information.

As the popularization of vehicles and the rate of road penetration increase exponentially, vehicle information is collected using vehicle detection means, and traffic information such as vehicle density, congestion rate, vehicle speed, and offending vehicle is generated based on collected vehicle information Intelligent Transportation System (ITS) is widely used for comprehensive management and monitoring of generated traffic information.

As a vehicle detection method for collecting such vehicle information, a method using a radar signal and a method using a camera are usually used.

A method using a radar signal is a method of collecting vehicle information by analyzing a collected reflected signal after collecting a reflected signal of a radar signal transmitted to a predetermined sensing area, But it is less affected by the external environment and has an advantage of object detection in the longitudinal direction. However, this method using radar has disadvantages of detection of position and velocity in the lateral direction, accuracy of classification and information detection for objects, and the like.

A method using a camera is a device for collecting vehicle information by analyzing image information obtained by photographing a camera. The device is excellent in object classification, has excellent accuracy of object classification and information detection, Detection is advantageous. However, the method using the camera is easily affected by the external environment, and has a disadvantage that the detection accuracy with respect to the distance and the speed is relatively low as compared with the radar signal.

Accordingly, various studies have been conducted to detect vehicle information by analyzing image information and radar signals by integrally manufacturing a camera and a radar transceiver.

An unexpected detection system disclosed in Korean Patent No. 10-1343975 filed by the applicant of the present invention includes a camera and a radar detector at the same time, and the image information and the radar signal The detection accuracy and the reliability of the vehicle information can be increased.

However, since the unexpected detection system is configured to use only the image information when the image information obtained by the camera is in a state in which the vehicle can be detected, and to use the radar signal in the case where the obtained image information is in a state in which the vehicle can not be detected The advantages of the radar signal and the image information can not be maximized and can not be used. Therefore, the accuracy of the detection is lower than when the radar signal and the image information are used in combination. That is, in the case where the obtained image information is clear, since the vehicle information is detected only through the image analysis, the detection accuracy of the distance and the speed is deteriorated. If the obtained image information is not clear, The accuracy of object classification and object feature information detection is low.

1 is a block diagram showing a vehicle detector disclosed in Korean Patent No. 10-1446546 filed by the applicant of the present invention and entitled " Position Based Real Time Vehicle Information Display System "; FIG. 2 is a block diagram of a vehicle detector FIG. 3 is a view illustrating a traffic information user interface in which traffic information collected by the user is displayed.

1 is a block diagram showing a vehicle detector 100 of the first embodiment of the present invention. The vehicle sensor 100 includes a database 102 for storing data, a data transmitting and receiving unit 103 for transmitting and receiving data, a radar signal receiving unit 110 for receiving data from the radar transmitting and receiving unit 110, A vehicle locus detection unit 104 for detecting the locus of each of the vehicles based on the vehicle locus information detected by the locus detection unit 104, A vehicle information detecting unit 105 for detecting vehicle information including the vehicle information and the vehicle speed detected by the vehicle information detecting unit 105; An accident situation generating unit 106 for generating an unexpected situation, a vehicle speed detecting unit 105 for detecting a position, a speed, a running direction and an anteroposterior distance detected by the vehicle information detecting unit 105, Based vehicle information generating unit 107 for generating position-based vehicle information by matching the information on the position of the sensing region S with the position information of the sensing region S and the position information of the sensing region S, A photographing means driving section 109 for controlling the photographing means 120 and a control section 102 for controlling these elements 102, 103, 104, 105, 106, 107 ), (108), (109).

The image information obtained by the photographing of the photographing means 120 is transmitted to the traffic management server. The traffic management server analyzes the image information to analyze the vehicle location information, and then transmits the vehicle location information to the location-based vehicle information generating unit 107 Based vehicle information generated by the location-based vehicle information.

The data obtained by the conventional art 100 constructed as above is displayed on the traffic information user interface 200 of FIG. 2 when displayed on the terminal, and the traffic information user interface 200 displays the image corresponding to the actual road shape A virtual map active window 720 for mapping and displaying the location-based traffic information on the created virtual map 211 on which the road image 212 is displayed, a virtual map active window 720 for displaying the image transmitted from the traffic management server, (230).

As described above, the conventional art 100 collects information by performing the radar detector 110 and the photographing means 120 at the same time, and then matches the collected vehicle information and displays it on the terminal through the traffic information user interface 200, And more accurate traffic information can be provided.

However, in the conventional art 100, object feature information such as object detection, detected object position, detected object velocity, and the like is detected using only a radar signal, image information is not utilized for object detection, The accuracy of detection of position and velocity information about the lateral direction, the size of the detected object, lane information, etc. is limited.

In addition, since the conventional art 100 does not have a separate constituent means for classifying whether the detected object is a vehicle, a pedestrian, or a fixed object, all the detected objects are judged to be the same object, and the accuracy of the information is lowered.

In order to acquire an image corresponding to the detection area of the radar detector 110 in consideration of the characteristics of the radar sensor 110 having a relatively large sensing area as compared with the photographing device 120, However, since only one photographing means 120 is provided, the area of the photographing area, which can recognize the object (vehicle) compared to the detection area of the radar detector, is excessively small, There is a problem that the efficiency is low.

In order to solve such a problem, a plurality of cameras divide the detection area of the radar detector into a plurality of cameras, and at the same time, the panorama images acquired by these cameras are used in combination. When these panorama images are combined, It is necessary to combine the distortion after the correction of the image due to use. However, even if it is supposed that a plurality of imaging means are provided in the conventional technique 100, a method and means for correcting the distortion in the image combination are described in the related art 100 There is no accuracy limit.

That is, 1) it is possible to increase the accuracy of the object feature information and to secure diversity by detecting the object information in consideration of the advantages of the image information and the advantages of the radar signal, 2) to detect the size and type of the detected object And 3) it is possible to easily detect the object type by using the object type model including the object feature information for each kind, 4) the traffic management which can accurately detect the object feature information using the image information through the image preprocessing process System and method are urgently needed.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and an object of the present invention is to provide a camera and a radar detector, which can generate various and accurate traffic information by detecting object and object characteristic information from image information and a radar signal, And to provide a traffic information management system using radar.

Another object of the present invention is to provide a method and apparatus for synchronizing a video object and a radar object by coordinate and time synchronization and then matching a synchronized video object and a radar object using a homography matrix using a ransac (random sample consensus) And to provide a traffic information management system using a camera and a radar capable of remarkably increasing the accuracy of detection.

Another object of the present invention is to provide a HOG descriptor having a different pattern for each type of vehicle, pedestrian, and background during the detection of video object information and storing the pattern difference of the HOG descriptor in a nonlinear SVM The present invention is to provide a traffic information management system using a camera and a radar capable of detecting an object by using a support vector machine classifier.

Further, another object of the present invention is to provide a traffic information management system using a camera and a radar which can easily and easily view traffic information by displaying a real image, an unexpected situation tracking image and a virtual map through a traffic information user interface .

According to an aspect of the present invention, there is provided a radar apparatus comprising: a radar detector for transmitting / receiving a radar to a predetermined sensing area; An array camera configured to capture an image of the sensing area to acquire an image, and to capture the sensing area and photograph the closest area; A radar object detection unit for detecting a radar object that is an object by analyzing a radar signal collected by the radar detector; a video object detection unit for detecting a video object that is an object in the image by analyzing the image acquired by the photographing unit; And a matching unit for matching the video object and the radar object by coordinate and time synchronization, wherein the controller corrects radial distotion of the array camera when an image is input from the array cameras, A binary image transformation module for transforming the mosaic image into a binary image of a foreground using a Gaussian mixture model; and a binary image transformation module for transforming the binary image into a foreground binary image using a Gaussian mixture model, A binary image transformed by the image transformation module, or An object extraction module for extracting an object by performing a blob processing on a noise-removed binary image by the noise elimination module; and an object extraction module for extracting the noise from the binocular HOG descriptor And a SVM classification module for recognizing at least one object type of a vehicle, a pedestrian, and a vehicle color through a learning process using a nonlinear SVM (Support Vector Machine) classifier using an RBF kernel.

In the present invention, the matching unit maps a position of a video object and a position of a radar object in the same frame, obtains a homography matrix using a ransac (random sample consensus) algorithm, It is preferable to detect the composite composite object information by matching the image object information and the radar object information.

The controller may further include a video object feature information detector for detecting video object feature information related to the video object and a radar object feature information detector for detecting radar object feature information related to the radar object, And the video object feature information may generate traffic information by matching the video object feature information and the radar object feature information to the fused composite object information.

The controller may further include an interface management unit for generating a traffic information user interface that is a graphic user interface in which the traffic information is displayed according to a predetermined graphic procedure, The traffic information user interface includes a virtual map active window for displaying traffic information mapped to a virtual map corresponding to an actual road shape and a virtual map active window for displaying the image, Wherein the virtual map displays a road image produced in correspondence with a shape of an actual road and a vehicle image displayed on the road image corresponding to a position of the fusion compound object, The size of the image is set corresponding to the size information of the feature information Is recommended.

The controller may further include a motion estimator, wherein the motion estimator determines velocity information included in the radar object feature information as a motion estimation value when radar object information is included in the fusion composite object information, When the radar object information is not included in the object information, an optical flow of a continuous frame is repeatedly calculated for a continuous frame object for the image object information using a Lucas Canade method, and the calculated optical flow is used as a motion estimation value The traffic information includes the motion estimation value, and the vehicle image of the virtual map of the traffic information user interface moves according to the motion estimation value.

In addition, the traffic information management apparatus may further include a tracking camera that tracks a target object that is a cause of an unexpected situation when an unexpected situation occurs, and performs shooting, and the controller compares the traffic information with a preset threshold value And an abrupt situation judging unit for judging whether or not an erroneous situation has occurred.

Also, in the present invention, the traffic information user interface displays an image obtained by the tracking camera when an unexpected situation occurs in the image active window, and displays an object of an unexpected situation in a color different from the vehicle images .

delete

delete

delete

According to the present invention having the above-mentioned problems and solutions, a camera and a radar detector can be provided, and various accurate traffic information can be generated by detecting object and object characteristic information from image information and a radar signal.

According to the present invention, after synchronizing the video object and the radar object with each other, the synchronized image object and the radar object are matched by using a homography matrix using a ransac (random sample consensus) algorithm, Can be significantly increased.

According to the present invention, a HOG descriptor having a different pattern for each type of vehicle, pedestrian, background, and the like is previously set and stored at the time of detecting the video object information, and the pattern difference of the HOG descriptor is stored in a Support Vector Machine (SVM) ) Classifier, it is possible to increase the accuracy of object detection and to detect the kinds of detected objects.

In addition, according to the present invention, a real image, an unexpected situation tracking image, and a virtual map are displayed through a traffic information user interface, so that a user can easily and easily view traffic information.

1 is a block diagram illustrating a vehicle detector disclosed in Korean Patent No. 10-1446546 (entitled " Location Based Real Time Vehicle Information Display System ") filed by the applicant of the present invention.
FIG. 2 is an exemplary view showing a traffic information user interface on which traffic information collected by the vehicle detector of FIG. 1 is displayed.
3 is a configuration diagram illustrating a traffic management system according to an embodiment of the present invention.
4 is a block diagram showing the controller of Fig.
FIG. 5 is an exemplary real world view for explaining the radial distortion caused by the preprocessing unit of FIG. 4;
6 is an illustrative real-time image for explaining a mosaic image by the preprocessing unit of FIG.
FIG. 7 is a block diagram illustrating a video object detection and classification unit of FIG. 4;
8 is an exemplary diagram for explaining the binary image transformation module of FIG.
FIG. 9 is an exemplary diagram illustrating a HOG descriptor applied to the SVM classification module of FIG. 8. FIG.
10 is a block diagram showing the matching unit of Fig.
11 is a diagram illustrating an example of a traffic information user interface generated by the traffic information user interface management unit of FIG.
FIG. 12 is an exemplary view showing the traffic information user interface of FIG. 11 when an unexpected situation occurs.

Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings.

3 is a configuration diagram illustrating a traffic management system according to an embodiment of the present invention.

The traffic management system 1 shown in Fig. 3 includes an array camera 33, a radar detector 35 and a tracking camera 37 and is provided with image information by the array camera 33 and radar An integrated detector 3 for generating a traffic information user interface, which is a graphical user interface displayed on the basis of the generated tools, and a traffic information management server A traffic control server 7 for managing and monitoring traffic information received from the integrated sensor 3, a client 8 for displaying a traffic information user interface transmitted from the integrated sensor 3, and a data movement path between them And a data communication network (9).

Although the integrated detector 3 itself generates the traffic information and the traffic information user interface for the sake of convenience in the present invention, the integrated detector 3 may simply generate the object feature information And transmits the detected object feature information and image information to a separate management server. The management server analyzes the object feature information and the image information received from the integrated detector 3 to produce a traffic information and traffic information user interface .

The data communication network 9 provides a connection path and a data movement path between the integrated detector 3, the client 8 and the traffic control center server 7 and more specifically a local area network (LAN), a wide area network ), A mobile communication network, a wired communication network, and the like can be applied.

The client 8 is a digital device connected to the data communication network 9 to request a traffic information user interface to the integrated detector 3 or to display a traffic information user interface received from the integrated detector 3 on the monitor.

The traffic control server 7 manages and controls the integrated detectors 3, processes and manages traffic information and image information received from the controller 31, and monitors the roads.

The integrated detector 3 includes a radar detector 35 for receiving a reflection signal of a radar signal transmitted to a predetermined sensing area S and a controller 35 for acquiring an image by photographing each of the imaging areas in which the sensing area S is divided Array cameras 33, a tracking camera 37 for tracking and photographing the object in the event of an unexpected event, the radar signal and image information obtained by the radar detector 35 and the array camera 33 are analyzed And a controller 31 for detecting an object and generating a traffic information and traffic information user interface.

At this time, the camera for photographing the nearest shooting area among the array cameras 33 is composed of a fish-eye lens, and the other cameras are composed of a high magnification lens corresponding to the focal distance.

4 is a block diagram showing the controller of Fig.

The controller 31 shown in FIG. 4 may be configured as either an internal or an external device and analyzes the image information and the radar signal input from the array camera 33 and the radar detector 35 to generate a traffic information and traffic information user interface do.

The controller 31 includes a database unit 312 for storing data, a data transmitting and receiving unit 313 for transmitting and receiving data to and from the traffic control server 7 and the client 8, A preprocessing unit 314 for performing a preprocessing process on the acquired image, and an image processing unit 314 for analyzing the image subjected to the distortion correction and the noise removal by the preprocessing unit 314 to detect an object (hereinafter referred to as a video object) A motion estimation target list unit 316 for generating a list of image objects to which movement estimation is to be performed and a motion estimation target list unit 316 for detecting A radar object feature detecting unit 317 for detecting an object (hereinafter referred to as a radar object) by analyzing the radar signal and detecting feature information of the detected radar object, A motion object feature detecting unit 319 for detecting feature information of a video object by analyzing the video object data, a motion estimating unit 320 for estimating motion of the object, A Kalman filtering unit 321 for performing prediction and correction by performing a Kalman filter algorithm for the object, a fusion complex data for performing the Kalman filtering unit 321, image object characteristic information and a radar object characteristic A traffic information user interface for storing the object information, the object feature information, the motion estimation value, the unexpected situation information, and the like in the designed traffic information user interface, for analyzing the traffic information including the traffic information, 313, 314, 315, 316, 317, 318, 319, 320, 321, 321, 322, 323, And a control unit 311 for managing and controlling the control units 322 and 323.

313, 314, 315, 316, 317, 318, and 319, which are operating systems (OS) of the controller 31, , (320), (321), (322), and (323).

The control unit 311 periodically searches the data received from the management server 7 and the client 8 by crawling the data transmitting and receiving unit 313. [

The control unit 311 periodically controls the data transmitting and receiving unit 313 to transmit the acquired image information, the object virtual map, the traffic information user interface, the traffic information (the video object information, the radar object information, Information, incident situation information, and the like) to the traffic control server 7.

The control unit 311 also inputs the image information obtained by the imaging of the array camera 33 to the preprocessing unit 314 and inputs the radar signal collected by the radar detector 35 to the radar object characteristic detection unit 317 do.

The controller 311 controls the tracking camera 37 so that the tracking camera 37 tracks and tracks the unexpected object, if an unexpected situation is detected by the unexpected state determiner 322. [

When the Kalman filtering unit 321 performs prediction and correction, the control unit 311 inputs the fusion composite data to the motion estimation target list unit 316. [

The database unit 312 stores the manufactured traffic information user interface 500, image information, tracking image information, and object map.

The database unit 312 stores image object information, image object characteristic information, radar object information, radar object characteristic information, and fusion composite data.

The data transmission / reception unit 313 transmits / receives data to / from the traffic control server 7 and the client 8 connected to the data communication network 9.

FIG. 5 is an exemplary real world view for explaining the radial distortion by the preprocessing unit of FIG. 4, and FIG. 6 is an exemplary real world view for explaining a mosaic image by the preprocessing unit of FIG.

The preprocessing unit 314 preprocesses the input image information so that the video object detection and classification unit 315 accurately detects and classifies the video object.

Since the camera for photographing the nearest distance among the array cameras 33 is composed of a fisheye lens, as shown in FIG. 5, the acquired image 410 of the fisheye lens has a radial distortion radial distotion phenomenon occurs.

Accordingly, the preprocessing unit 314 acquires the corrected image 420 by correcting the radial distortion phenomenon, and the correction method of the radial distortion phenomenon is defined by the following equation (1).

는 어안렌즈의 획득영상의 왜곡된 픽셀을 나타내고,

는 보정된 픽셀을 나타낸다. At this time

Represents the distorted pixel of the acquired image of the fisheye lens,

Represents the corrected pixel.

In addition, even if the image of the fish-eye lens is corrected, the array cameras 33 are formed in multiple channels and a different perspective is applied to each channel. Therefore, as shown in FIG. 6, To obtain a mosaic image 440. The mosaic image 440 is obtained by performing a perspective projective transformation. At this time, the perspective projection transformation is a technique for correcting a perspective image to a plane image and is commonly used in such image correction, and thus a detailed description thereof will be omitted.

The image corrected by the preprocessing unit 314 is input to the video object detection and classification unit 315.

FIG. 7 is a block diagram showing a video object detection and classification unit of FIG. 4, FIG. 8 is an exemplary view for explaining the binary image transformation module of FIG. 7, and FIG. 9 is a diagram illustrating a HOG descriptor applied to the SVM classification module of FIG. Fig.

The image object detection and classification unit 315 is driven when the mosaic image 440 is inputted from the preprocessing unit 314 and analyzes the inputted mosaic image 440 to detect the image object which is an object in the image, Classify object types.

7 and 8, the image object detection and classification unit 315 extracts the mosaic image 440 from the binary image of the foreground 451 using a known Gaussian mixture model for the input mosaic image 440, A noise removal module 350 for removing noise by a median filter or a morphology operation of the binary image changed by the binary image conversion module 351, And an object extraction module 355 for extracting the object 451 by blob processing the binary image from which noise has been removed by the noise removal module 353. [

In the present invention, the object extraction module 355 uses the block processing method for the convenience of description. However, the object extraction method is not limited thereto and various known methods such as a method using a foreground histogram may be applied Of course.

In addition, the video object detection and classification unit 315 further includes an SVM classification module 357. Since the SVM classification module 357 has different HOG descriptors for each type of vehicle, pedestrian, and background, the pattern of the HOG descriptor The difference can be detected by using a nonlinear SVM (Support Vector Machine) classifier using the RBF kernel and at the same time detecting the type of object.

9, the HOG descriptor is a method for detecting a feature vector through a histogram of the direction component and size of an edge as shown in FIG. 9. Since the object recognition is excellent, the SVM classification of the video object detection and classification unit 315 After the module 357 detects the HOG descriptor, it has a process of recognizing at least one object type among the quantities, pedestrians, and vehicle colors through a learning process using SVM (Support Vector Machine).

In this way, the video object detection and classification unit 315 detects the video object from the corrected video and detects the type information of the video object. The detected video object information and the type information are input to the motion estimation target list unit 316 do.

The motion estimation target list unit 316 compares the video object information input from the video object detection and classification unit 315 with the video object information of the previous frame, and outputs the video of the current frame located nearest to the video object of the previous frame If an object exists, the ID of the previous frame is given to the image object of the current frame to generate a list of the image object, which is the object of motion estimation.

The video object information and the object type information to which the identification ID is assigned by the motion estimation subject list unit 316 are input to the matching unit 318. [

The radar object characteristic detecting unit 317 is driven when a radar signal collected by the radar detector 35 is input. The radar object characteristic detecting unit 317 detects an object (radar object) by analyzing the inputted radar signal, .

Also, the radar object characteristic detection unit 317 analyzes the radar signal to detect the position, velocity, distance between the target and movement (trajectory) of the detected radar object. At this time, the position information, the velocity information, the distance between the front and the back, and the motion information of the radar object are referred to as radar object feature information.

The ID information of the radar object detected by the radar object characteristic detection unit 317 and the radar object characteristic information are input to the matching unit 318. [

10 is a block diagram showing the matching unit of Fig.

The matching unit 318 of FIG. 10 includes a time synchronization module 381 for synchronizing the time of the video object information and the radar object information, and a display unit 380 for converting the image coordinates of the corrected image time-synchronized by the time synchronization module 381 into a distance coordinate system A position synchronization module 383 for synchronizing the positions of the images, and a matching module 385 for matching the image object information and the radar object information to generate fusion composite object information.

The time synchronization module 381 synchronizes the time of the video object information and the radar object information. At this time, since the detection speed of the radar detection data and the video data is more than 30 frames / sec per second, the time difference is small for each frame, but when the array camera 33 uses the encoder or the network camera that streams the video by RTSP A time difference (t) of about 1 second occurs. Accordingly, the radar object information is buffered by t and synchronized with the time corresponding to the video object information.

The position synchronization module 383 converts the image coordinates of the time-synchronized corrected image by the time synchronization module 381 into a distance coordinate system. At this time, since the distance data is accurate using the Doppler effect, radar object information can be easily converted to the distance coordinate system.

The matching module 385 obtains a homography matrix using a ransac (random sample consensus) algorithm by mapping the position of the image object and the position of the radar object in the same frame, It is possible to detect the composite composite object information by matching the video object information and the radar object information.

The video object feature detection unit 319 analyzes video object information synchronized in time and position by the matching unit 318 and detects video object feature information such as a video object size, lane information, and lateral velocity information. At this time, the video object feature information further includes the object type information detected by the video object detection and classification unit 315 described above.

The motion estimation unit 320 estimates the movement of the object using the continuous frame for the fusion composite object information detected by the matching unit 318. If the composite object information is matched with the radar object information, The motion information of the information is selected as the true value, and when the radar object information is not matched to the fusion composite object information (only the image detection is performed), the following motion information is selected as the true value.

That is, the motion estimator 320 calculates the optical flow of the object of the continuous frame using the Lucas Canadet method when only the image detection is performed.

와

에서 잔류 함수(residual function)

를 최소화하는 움직임

를 구하는 것이 목표이나, 특징 추적(feature tracking) 시 윈도우 크기인

에 따라 정확도와 강인성에 영향을 받는다. 이때 옵티컬 플로의 계산 시 윈도우 사이즈를 선택할 때 지역적 정확성 및 강인성은 상호 교환 및 희생관계이 있기 때문에 피라미드 형태의 루카스 카나데 옵티컬 플로우 계산을 반복적으로 구하면 충분한 지역의 추적 정확도를 제공받을 수 있게 된다.The optical flow is expressed by the following equation (2)

Wow

The residual function in Eq.

Movement to minimize

, But the size of the window at the time of feature tracking

Are affected by accuracy and robustness. Since the local accuracy and robustness have mutual exchange and sacrifice relations in selecting the window size when calculating the optical flow, it is possible to obtain sufficient local tracking accuracy by repeatedly calculating the pyramid-shaped Lucas Canade optical flow calculation.

는 잔류함수이고,

는 잔류함수

를 최소화하는 움직임이고,

와

는 연속프레임 상에서 객체의 위치이다.At this time

Is a residual function,

The residual function

, ≪ / RTI >

Wow

Is the position of the object on consecutive frames.

The Kalman filtering unit 321 performs a Kalman filter algorithm on each frame of the motion information for each object detected by the motion estimation unit 320 to predict and correct the object, The accuracy of the process can be increased.

In this case, the fusion complex data, the video object characteristic information, and the radar object characteristic information, which are performed by the Kalman filtering unit 321, are referred to as traffic information.

The unexpected situation determination unit 322 analyzes the traffic information including the fusion complex data, the video object characteristic information, and the radar object characteristic information performed by the Kalman filtering unit 321 to determine an unexpected situation.

The unexpected situation may be various situations such as speed, ahead distance, emergency road driving, illegal parking, excessive stop, etc., and the unexpected situation determination unit 322 may be configured to process the operation corresponding to the applied situation through the traffic information do.

Also, the unexpected situation determination unit 322 tracks the unexpected situation object when an unexpected situation occurs. At this time, the control unit 311 controls the tracking camera 37 so that the tracking camera 37 tracks and tracks the unexpected object.

FIG. 11 is an exemplary view illustrating a traffic information user interface generated by the traffic information user interface management unit of FIG. 4, and FIG. 12 is an exemplary view illustrating the traffic information user interface of FIG. 11 when an unexpected situation occurs.

The traffic information user interface management unit 323 generates a traffic information user interface 500 that displays the corrected image, the tracked image, and the traffic information according to a predetermined graphical procedure. When a connected client 8 requests to view the traffic information, And transmits the traffic information user interface 500 generated by the traffic information management unit 8.

11 and 12, the traffic information user interface 500 includes a virtual map active window 510 for mapping traffic information to a virtual map 511 corresponding to an actual road shape, a mosaic image 440 An image active window 520 for displaying the tracking image 521, and other information display window 540 for displaying other information.

The other information display window 540 displays the operation state information of the radar detector 35, the array camera 33 and the tracking camera 37, and other information including road surface state information, environmental information, and the like.

The virtual map 511 is displayed on the virtual map active window 510. The virtual map 511 displays the road image 512 produced in correspondence with the shape of the actual road and the position of the vehicle running on the actual road And a vehicle image 513 displayed on the road image 512.

In addition, speed information 514 of the vehicle is indicated in the vehicle image 512 of the virtual map 511. [

In addition, the size of the vehicle image 512 is reflected in the virtual map 511 in correspondence to the traffic information, specifically, the vehicle size information of the video object feature information.

Also, the virtual map 511 displays the unexpected situation object 517 in a color different from that of the other vehicle image 512 when an unexpected situation occurs. At this time, the image 522 of the unexpected situation object photographed by the tracking camera is displayed on the image active window 520.

1: Traffic management system 3: Integrated sensor
7: Traffic control server 8: Client 9: Data communication network
311: Control section 312: Database section 313: Data transmission /
314: preprocessing unit 315: video object detection and classification unit
316: Motion estimation target list unit 317: Radar object feature detection unit
318: Matching unit 319: Video object feature detecting unit
320: motion estimation unit 321: Kalman filtering unit
322: an abrupt situation determination unit 323: a traffic information user interface management unit

Claims (9)

A radar detector for transmitting and receiving a radar to a predetermined sensing area S;
An array camera configured to capture an image of the sensing area to acquire an image, and to capture the sensing area and photograph the closest area;
A radar object detection unit for detecting a radar object that is an object by analyzing a radar signal collected by the radar detector; a video object detection unit for detecting a video object that is an object in the image by analyzing the image acquired by the photographing unit; And a matching unit for matching the video object and the radar object by coordinate and time synchronization,
The controller further includes a preprocessor for correcting radial distor- tion of the array camera when an image is input from the array cameras and acquiring a mosaic image by subjecting the image of each channel to perspective projection transformation,
The image object detection unit may include a binary image transformation module for transforming the mosaic image into a binary image of a foreground using a Gaussian mixture model and a binaural image filter or morphology operation transformed by the binary image transformation module to remove noise An object extraction module for extracting an object by blob processing the binary image from which the noise is removed by the noise removal module; and a RBF kernel detecting module for detecting the HOG descriptor of the noise- And a SVM classification module for recognizing at least one object type among a vehicle, a pedestrian, and a vehicle color through a learning process using a nonlinear SVM (Support Vector Machine) classifier using the SVM. [Claim 4] The method according to claim 1, wherein the matching unit maps a position of a video object and a position of a radar object in the same frame, obtains a homography matrix using a ransac (random sample consensus) algorithm, And detects a composite compound object in which the video object information and the radar object information are matched by obtaining a matrix having a large degree of fitness. [Claim 4] The method of claim 2, wherein the controller comprises: a video object feature information detector for detecting video object feature information that is related to the video object; and a radar object feature detector for detecting radar object feature information, Further comprising an information detecting unit,
Wherein the video object feature information generates traffic information by matching the video object feature information and the radar object feature information with the fused composite object information. The method of claim 3, wherein the controller further comprises an interface management unit for generating a traffic information user interface, which is a graphical user interface in which the traffic information is displayed according to a predetermined graphical procedure,
The video object characteristic information detection unit detects video object characteristic information including size information of the video object,
Wherein the traffic information user interface includes a virtual map active window for displaying traffic information mapped to a virtual road map corresponding to an actual road shape and an image active window for displaying the image, A vehicle image displayed on the road image corresponding to the position of the fusion composite object is displayed and the size of the image is set in correspondence with the size information of the video object feature information The traffic information management apparatus comprising: 5. The apparatus of claim 4, wherein the controller further comprises a motion estimator,
The motion estimator
Determining velocity information included in the radar object feature information as a motion estimation value when the composite compound object information includes radar object information, and determining, when the radar object information is not included in the composite composite object information, An optical flow of a continuous frame is repeatedly calculated for a frame object using a Lucas Canade system, the calculated optical flow is determined as a motion estimation value,
Wherein the traffic information includes the motion estimation value and the vehicle image of the virtual map of the traffic information user interface moves according to the motion estimation value. The traffic information management apparatus of claim 5, wherein the traffic information management apparatus further comprises a tracking camera that tracks a target object, which is a cause of an unexpected situation when an unexpected situation occurs,
Wherein the controller further comprises an unexpected situation determiner for comparing the traffic information with predetermined threshold values to determine whether an unexpected situation has occurred. The method of claim 6, wherein the traffic information user interface
Wherein the image captured by the tracking camera is displayed on the image active window when an unexpected event occurs and the object causing the unexpected event is displayed in a color different from the vehicle images. delete delete

Images