KR20200106810A - System for sensing and control unexpected situation using data fusion - Google Patents

Abstract

According to the present invention, provided is an accidental situation monitoring and controlling system through data fusion comprising: a detection device including at least one of an image camera, a radar sensor, or a tracking camera; and an accidental recognition control device analyzing data collected from the detection device, fusing image data of the image camera with radar data of the radar sensor, and recognizing traffic information and accidental situations on the road. The accidental recognition control device recognizes the accidental situation by analyzing the fusion data, and tracks the accidental situation by controlling the tracking camera when detecting the accidental situation on the road. Through the same, provided has the effect of improving the resolution of data, and efficiently recognizing and monitoring the accidental situation.

Description

Sudden situation monitoring control system through data fusion {SYSTEM FOR SENSING AND CONTROL UNEXPECTED SITUATION USING DATA FUSION}

The present invention relates to an emergency situation monitoring control system through fusion of image data and radar data.

With the development of the Intelligent Transport System (ITS), roads are gradually becoming more intelligent. In addition, cameras for road monitoring are widely installed and spread, such as monitoring vehicles and pedestrians through cameras such as CCTV and video detectors. In addition, recently, methods such as providing information to a driver by tracking or classifying a vehicle using a laser sensor or supporting unmanned autonomous driving have been developed.

In addition, in recent years, accidents such as traffic accidents, stopped vehicles, falling objects, pedestrians, low-speed vehicles, etc. on the road or inside the tunnel are highly likely to lead to large-scale disasters by causing a chain collision or secondary accident. A technology that promptly detects and informs the rear vehicle through electric road signs, roadside broadcasting, and vehicle-to-vehicle communication is being developed.

In addition, as a prior art document related to the present invention to be described later, Korean Patent Laid-Open Publication No. 10-2013-0127822 (published on Nov. 25, 2013)'a heterogeneous sensor fusion processing apparatus and method for classifying and detecting an object on the road 'There is.

However, in the related art, resolution is deteriorated due to limitations of hardware and time constraints required for updating information, and thus, it is difficult to accurately detect a moving object or to accurately recognize an unexpected situation.

In addition, there is a difficulty in maintaining and maintaining millions of CCTVs and controllers in the past, and there is a difficulty in effectively coping with unexpected situations due to the inability to collect traffic information from CCTVs arranged in shaded areas.

The matters described in this background are prepared to enhance an understanding of the background of the invention, and may include matters not known in the prior art to those of ordinary skill in the field to which this technology belongs.

The present invention improves resolution by processing image data and radar data, and proposes an emergency monitoring control system through data fusion capable of effectively recognizing and monitoring an emergency situation.

In addition, the present invention removes the blind spot of traffic information by easily collecting the maintenance information of the detection device and the sudden recognition control device arranged in the shaded area, the traffic information and the sudden situation in the shaded area, and We intend to propose an emergency monitoring and control system through data fusion that can improve quality.

A detection device including at least one of a video camera, a radar sensor, or a tracking camera through data fusion of the present invention, and an analysis of the data collected from the detection device, and the video data of the video camera and the radar It includes an accidental recognition control device that recognizes traffic information and unexpected situations on the road by fusion of the radar data of the sensor, and the accidental recognition control device recognizes the unexpected situation by analyzing the fusion data, but detects the unexpected situation on the road. Then, the tracking camera is controlled to track an unexpected situation.

The accidental recognition control device detects an abnormal moving vehicle with respect to the normal moving direction of the lane using a Doppler effect, but detects the moving speed of the moving object by analyzing the wavelength of the transmission signal and the reception signal of the radar sensor. It may include a reverse driving determination unit to determine whether to move in the reverse direction or the left and right direction compared to the normal movement direction of.

The sudden recognition control device removes noise or diffuse reflection by processing the radar data periodically collected from the radar sensor, and the signal of the time axis is set to a zero value for a time corresponding to an integer multiple after the input signal of a certain period with respect to the periodic signal. It may further include a noise improving unit for improving the moving speed resolution of the moving object by applying a zero padding (Zero Padding) algorithm for signal processing.

The accidental recognition control device recognizes object information by analyzing the received signal reflected from the moving object for the transmission signal of the radar sensor, but a side lobe attenuation algorithm to minimize the noise ratio of the received signal to the transmitted signal. It may further include a feature extraction unit for extracting object information and object features by applying.

The sudden recognition control device fuses the analysis result of the image data and the analysis result of the radar data, samples each data in a time period, converts it from an analog signal to a digital signal, and fuses it, but an over-sampling algorithm The data fusion unit may further include a data fusion unit configured to generate fusion data by applying and fuse object features of the moving object through spatial matching between the image data and the radar data.

The accidental recognition control device includes a transmission/reception unit that is connected to an adjacent accidental recognition control device by wire or wirelessly to collect information, and transmits the collected information to another adjacent accidental recognition control device or to a traffic information center. can do.

The transmission/reception unit includes a repeater unit for controlling transmission/reception time information to be allocated and transmitting/receiving a signal by being synchronized with the allocated transmission/reception time information, and to transmit information to a neighboring accidental recognition control device only at a specified time according to the transmission/reception time information I can.

It is mounted on the mobile vehicle, further comprises an information collection terminal that is connected to the plurality of accidental recognition control devices and wireless communication to collect information from the plurality of accidental recognition control devices, the information collection terminal, wherein the plurality of accidental recognition of the meter reading signal A meter reading signal transmitter to be transmitted to a control device, a carrier wave detection unit that detects a carrier wave transmitted from at least one of the plurality of accidental recognition control devices that has received the meter reading signal, and an accidental recognition control device that transmits the carrier wave It may include an information collection unit that determines whether there is an abnormality in the detection device connected to the accidental recognition control device that transmits the carrier wave, and transmits it to the traffic information center.

The information collection terminal may further include a meter reading signal retransmission control unit for controlling to retransmit the meter reading signal after a predetermined time delay when a plurality of carriers are detected in the same frequency band.

The accidental recognition control device determines the road surface condition of the road through a road surface condition determination algorithm that determines the road surface condition by using polarization of light or through machine learning, and transmits the determined road surface condition information to the surroundings through wireless communication. It can be transmitted to an adjacent accident recognition control device or to a traffic information center.

According to the present invention, image data and radar data are signal-processed and fused, thereby improving data resolution and providing an environment in which unexpected situations can be recognized and monitored effectively.

In addition, the present invention provides an environment capable of effectively detecting an unexpected situation and actively responding to an unexpected situation by detecting an abnormally moving vehicle in a reverse driving direction or a left-right direction with respect to a normal moving direction of a lane using the Doppler effect.

In addition, the present invention provides an environment in which, by applying a zero padding algorithm, a moving speed resolution of a moving object can be improved, and a limitation of resolution improvement due to time constraints required for information update can be overcome.

In addition, the present invention minimizes the noise ratio of the received signal to the transmitted signal by applying a side lobe attenuation algorithm, and provides an environment in which object information and object features can be effectively extracted.

In addition, the present invention generates fusion data of image data and radar data by applying an over-sampling algorithm, thereby minimizing limitations due to hardware limitations and time limitations that occur when disparate data is fused, It provides an environment to improve the resolution of data.

In addition, the present invention provides an environment in which a blind spot of traffic information is removed by collecting traffic information and an unexpected situation in a shadow area through a repeater function, and traffic safety and quality of traffic information can be improved.

In addition, the present invention collects information of the detection device and the sudden recognition control device arranged in the shaded area through the data collection function of the information collection terminal, effectively grasping the presence or absence of an abnormality in the detection device or the sudden recognition control device, and the detection device and It provides an environment in which the accidental recognition control device can be easily maintained.

1 is a diagram showing a schematic configuration of an unexpected situation monitoring control system through data fusion according to an embodiment of the present invention.
2 is a diagram schematically showing the configuration of an accidental recognition control device according to an embodiment of the present invention.
3 is a diagram illustrating data before and after zero padding according to an embodiment of the present invention.
4 is a diagram illustrating an example of detecting an abnormal moving vehicle using the Doppler effect according to an embodiment of the present invention.
5 is a diagram illustrating an example in which an accidental recognition control device transmits data to a traffic information center according to an embodiment of the present invention.
6 is a diagram schematically showing the configuration of an information collection terminal according to an embodiment of the present invention.
FIG. 7 is a diagram illustrating an example in which an information collection terminal wirelessly collects information from an accidental recognition control device and transmits it to a traffic information center according to an embodiment of the present invention.
8 is a flowchart schematically illustrating a process of generating traffic information by fusion of image data and radar data and recognizing/tracking an unexpected situation according to an embodiment of the present invention.
9 is a flowchart illustrating a process of extracting object features by fusion of image data and radar data, and detecting an unexpected situation based on machine learning according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art can easily implement the embodiments of the present invention. However, the present invention may be implemented in various different forms and is not limited to the embodiments described herein. In the drawings, parts irrelevant to the description are omitted in order to clearly describe the present invention, and similar reference numerals are assigned to similar parts throughout the specification.

Throughout the specification, when a part "includes" a certain component, it means that other components may be further included rather than excluding other components unless otherwise stated. In addition, terms such as "... unit", "... group", and "module" described in the specification mean units that process at least one function or operation, which can be implemented by hardware or software or a combination of hardware and software. have.

Now with reference to FIGS. 1 to 9 will be described in detail an emergency situation monitoring control system through data fusion according to an embodiment of the present invention.

1 is a diagram showing a schematic configuration of an unexpected situation monitoring control system through data fusion according to an embodiment of the present invention. At this time, the emergency monitoring control system 10 through data fusion only shows a schematic configuration necessary for explanation according to an embodiment of the present invention, but is not limited to this configuration.

Referring to FIG. 1, an emergency monitoring control system 10 through data fusion according to an embodiment of the present invention includes a detection device 100, an accidental recognition control device 200, an information collection terminal 300, and a traffic It includes an information center 400.

In the unexpected situation monitoring control system 10 through data fusion according to an embodiment of the present invention, the accidental recognition control device 200 collects data from the detection device 100 in which an image camera and a radar device are integrated.

The detection device 100 is installed on roads such as curved sections, intersections, and tunnels, and collects traffic-related data and transmits it to the accidental recognition control device 200. Here, the detection device 100 is installed by integrating at least one of the image camera 110, the radar sensor 120, and the tracking camera 130. In addition, the tracking camera 130 may detect a road under the control of the accidental recognition control device 200 and photograph an unexpected situation. For example, the accidental recognition control device 200 may collect traffic-related data from CCTV, etc., and track the accidental situation.

In addition, the sudden situation monitoring control system 10 through data fusion according to an embodiment of the present invention, the accident recognition control device 200 is a traffic information center 400 and Ethernet (ehthernet)-based TCP / IP communication method or It can be connected and controlled by wireless communication. In this case, the accidental recognition control device 200 may perform functions of an image camera and tracking camera (zoom lens, pan/tilt) and a radar sensor by analyzing a control signal of the traffic information center 400.

In addition, the sudden situation monitoring control system 10 through data fusion according to an embodiment of the present invention applies a Gaussian mixed model to the image data acquired from the video camera 110 according to whether or not the high-speed mode, and a tracking object list algorithm. Apply.

In addition, the emergency situation monitoring control system 10 through data fusion according to an embodiment of the present invention acquires radar data from the radar sensor 120 using a K-band beam, and processes noise/diffuse reflection signals and 2D/ An algorithm that extracts object features through 3D image processing is applied.

In addition, the sudden situation monitoring control system 10 through data fusion according to an embodiment of the present invention extracts highly reliable object information by matching features of objects extracted from image cameras and radar sensors. In addition, the unexpected situation monitoring control system 10 through data fusion according to an embodiment of the present invention tracks/corrects the unexpected situation or predicts the unexpected situation through the Kalman technique, and detects information related to the unexpected situation. ) Or to the accidental recognition control device 200 to provide an environment in which the accidental situation can be effectively monitored.

In addition, the emergency situation monitoring control system 10 through data fusion according to an embodiment of the present invention includes the information collection terminal 300 installed in the maintenance vehicle and the accidental recognition control device 200 equipped with wireless communication technology and wireless communication. It communicates by monitoring the current state of the detection device 100 or the sudden recognition control device 200, and transmits the monitored information to the traffic information center 400.

In addition, the emergency monitoring control system 10 through data fusion according to an embodiment of the present invention transmits traffic information and maintenance information between the accidental recognition control device 200, or the information collection terminal 300 transmits traffic information. And maintenance information is collected and delivered to the traffic information center 400 to remove blind spots of public and private traffic information, improve traffic safety and quality of traffic information, and overcome the limitations of the existing detection system. Provide an environment that can be used.

2 is a diagram schematically showing the configuration of an accidental recognition control device according to an embodiment of the present invention. At this time, the accidental recognition control device 200 only shows a schematic configuration necessary for description according to an embodiment of the present invention, but is not limited to this configuration.

Referring to FIG. 2, an accidental recognition control device 200 according to an embodiment of the present invention includes an accidental recognition control module 210, a data collection unit 220, a signal processing unit 230, a determination unit 240, and It includes a transmitting and receiving unit 250.

The accidental recognition control module 210 analyzes the data collected from the detection device 100 and controls the tracking camera 130 to monitor the accidental situation. In addition, the sudden recognition control module 210 analyzes the data of the image camera 110 and the radar sensor 120, and combines the image data of the image camera 110 and the radar data of the radar sensor 120 to Recognize information and unexpected situations. In addition, when an unexpected situation is detected on the road, the accidental recognition control module 210 may control the tracking camera 130 to track the accidental situation.

The data collection unit 220 collects data from the image camera 110 and the radar sensor 120. In addition, the data collection unit 220 includes an image data collection unit 222, a radar data collection unit 224, and a road surface data collection unit 226 according to an embodiment of the present invention.

The image data collection unit 222 collects image data from the image camera 110. For example, the image data collection unit 222 may collect 2D image data captured by the image camera 110.

The radar data collection unit 224 collects radar data from the radar sensor 120. For example, the radar data collection unit 224 may collect 2D radar data detected by the radar sensor 120.

The road surface data collection unit 226 collects information related to the road surface condition. For example, the road surface data collection unit 226 uses vertical polarization data and horizontal polarization of the road surface captured by the image camera 110 and tracking camera 130 to determine the road surface condition of the road using the polarization of light. Data can be collected.

The signal processing unit 230 processes the data collected by the data collection unit 220 to improve the resolution of data or improve the reliability of object information extraction through data fusion.

The signal processing unit 230 includes a noise improving unit 232, a feature extracting unit 234, and a data fusion unit 236 according to an embodiment of the present invention.

The noise improving unit 232 processes radar data periodically collected from the radar sensor 120 to remove noise or diffuse reflection. The noise improving unit 232 may improve the resolution of the moving speed of the moving object by applying a zero padding algorithm.

For example, the noise improvement unit 232 is an integer multiple (2 times, 4 times, etc.) after a certain period input signal with respect to the periodic signal of the signal detected from the radar sensor 120 through a zero padding algorithm. During the corresponding time, the signal on the time axis can be signal-processed with a zero value ("0").

3 is a diagram illustrating data before and after zero padding according to an embodiment of the present invention.

Referring to FIG. 3, the emergency monitoring control system 10 through data fusion according to an embodiment of the present invention a) zero padding than data before zero padding to which a zero padding algorithm is not applied. ) It can be seen that the resolution of the data after b) zero padding with the algorithm applied is remarkably improved.

Through this, the sudden situation monitoring control system 10 through data fusion according to an embodiment of the present invention improves the resolution of the moving speed of the moving object and overcomes the limitation of improving the resolution due to the time constraint for updating information. It provides an environment in which to do it.

The feature extraction unit 234 analyzes the image data of the image camera 110 and the radar data of the radar sensor 120 to extract and recognize object information of the moving object. In addition, the feature extraction unit 234 analyzes the received signal reflected from the moving object for the transmission signal of the radar sensor 120 to recognize object information.

In addition, the feature extraction unit 234 controls to increase the output of the transmission signal of the radar sensor 120 to a predetermined value or more, and uses a side lobe attenuation algorithm to minimize the noise ratio of the reception signal to the transmission signal. By applying, object information and object features can be effectively extracted.

For example, the feature extraction unit 234 is the phase and frequency between the received signal received by the radio wave transmitted from the transmitting unit (not shown) of the radar sensor 120 reflected by the object to be detected and the transmitted signal transmitted from the radar. The distance or shape of an object can be detected based on the shift. At this time, the shape of the object is roughly displayed according to the resolution of the received signal reflected on the object, and data on the shape of the object is extracted based on the measured result.

However, if the main divergence signal of the radar as well as the surrounding parasitic signal is generated or the side lobe is large, the power distribution of the main divergence signal is lowered and the transmitted detection signal is reduced, and the peripheral object signal detected by the side lobe signal is a noise signal. As a result, the signal-to-noise ratio of the original signal deteriorates.

Accordingly, the emergency situation monitoring control system 10 through data fusion according to an embodiment of the present invention increases the output of the radar transmission signal to a predetermined value or more, and minimizes the noise ratio of the received signal to the transmitted signal. Lobe) By applying the attenuation algorithm, it provides an environment that can effectively extract object information and object features.

The data fusion unit 236 fuses the analysis result of the image data acquired from the image camera 110 and the analysis result of the radar data acquired from the radar sensor 120.

In addition, the data fusion unit 236 samples each data in a time period, converts the analog signal to a digital signal, and fuses it. At this time, in order to extract accurate data during signal processing for data fusion, it is good to sample a lot of signals, but limitations arise due to limitations such as hardware and time limitations.

Accordingly, the data fusion unit 236 generates fusion data by applying an over-sampling algorithm that samples a signal with a bandwidth of twice or more or the highest sampling frequency that can be sampled. In addition, the data fusion unit 236 fuses the object features of the moving object through spatial matching between image data and radar data. In addition, the emergency situation monitoring control system 10 through data fusion according to an embodiment of the present invention effectively monitors the location, distance, size, and shape of a fixed or moving object through fusion data and object information with improved resolution. Can be identified.

As described above, the emergency situation monitoring control system 10 through data fusion according to an embodiment of the present invention generates fusion data of image data and radar data by applying an over-sampling algorithm, thereby fusion of heterogeneous data. It minimizes the limitations of hardware and time that occur during the time, and provides an environment to improve the resolution of data.

The determination unit 240 recognizes traffic information and an unexpected situation on the road based on the analysis result of the data collected from the detection device 100. For example, the determination unit 240 may analyze the fusion data to detect an abnormal moving vehicle traveling in the reverse driving direction. Further, the determination unit 240 may determine the road surface condition of the road based on the analysis result of the data collected from the detection device 100.

The determination unit 240 includes an accidental situation recognition unit 242, a reverse driving determination unit 244, an accidental situation tracking unit 246, and a road surface condition determination unit 248 according to an embodiment of the present invention.

The unexpected situation recognition unit 242 recognizes the unexpected situation by analyzing the data and fusion data collected from the detection device 100. For example, the unexpected situation recognition unit 242 may recognize a traffic accident and an unexpected situation caused by a stopped vehicle, a low-speed moving object, a rockfall, or an object stopped on the shoulder.

The reverse driving determination unit 244 recognizes unexpected information by analyzing the fusion data or radar data, and detects an abnormally moving vehicle in the reverse driving direction or the left and right directions with respect to the normal movement direction of the lane using the Doppler effect.

In addition, the reverse driving determination unit 244 may analyze the wavelength of the transmission signal and the reception signal of the radar sensor to detect the moving speed of the moving object, and then compare it with the normal movement direction of a specific lane to determine whether to travel in the reverse direction or the left and right directions . For example, the reverse driving determination unit 244 may recognize whether or not a movement direction of an object according to an absolute value of a Doppler shift frequency is abnormally moved according to a difference from an average movement direction.

In addition, the reverse driving determination unit 244 may detect a speed in the left-right direction or in the reverse driving direction of the abnormal moving vehicle with respect to the average moving direction of the lane using the Doppler effect.

4 is a diagram illustrating an example of detecting an abnormal moving vehicle using the Doppler effect according to an embodiment of the present invention.

For example, the accidental recognition control apparatus 200 according to an embodiment of the present invention calculates a Doppler shift using a wavelength of a radar signal used for a speed difference between a radar sensor and a moving object. In addition, the sudden recognition control device 200 according to an embodiment of the present invention assumes the standard of the speed difference as a radar sensor, and divides the speed shift between the distant object and the closer object into positive/negative signs, and based on this After detecting the moving speed of the object, it can be compared with the normal moving direction of the corresponding lane to determine whether to run in reverse or to run left and right.

4, the accidental recognition control device 200 according to an embodiment of the present invention compares a transmission signal x of a radar and a reception signal y1, y2 reflected from a moving object to determine whether to run in reverse or left and right. Can judge.

For example, in the case of the received signal (y1), the frequency of the wave increases as the distance between the radar sensor and the moving object decreases, and the received signal (y1) is compared with the normal moving direction of the corresponding lane to a reverse driving vehicle. I can judge.

When an unexpected situation is detected on the road, the unexpected situation tracking unit 246 controls the tracking camera 130 to track the unexpected situation.

The road surface state determination unit 248 determines the road surface state of the road using a polarization phenomenon of light or through a road surface state determination algorithm that determines the road surface state through machine learning. In addition, the sudden situation monitoring control system 10 through data fusion according to an embodiment of the present invention transmits the determined road surface condition information of the road to the surrounding accident recognition control device or information collection terminal through wireless communication, or Road surface condition information can be transmitted to the traffic information center.

The transmission/reception unit 250 is connected to an adjacent accidental recognition control device by wire or wirelessly to collect information. Then, the transmission/reception unit 250 transmits the collected information to another accidental recognition control device 200, the information collection terminal 300, and the traffic information center 400 adjacent thereto.

The transmitting and receiving unit 250 includes a repeater unit 252 according to an embodiment of the present invention.

The repeater unit 252 controls the transmission/reception unit 250 to transmit its own information to an adjacent accidental recognition control device. In addition, when the transmission/reception time information is allocated, the repeater unit 252 may control the transmission/reception unit 250 to automatically transmit the information only at a specified time according to the allocated transmission/reception time information.

For example, the repeater unit 252 is assigned ID and transmission/reception time information for each sudden recognition control device, and controls the transmission/reception unit 250 to transmit data at a time when it must respond based on the allocated transmission/reception time information. can do.

In addition, when the transmission/reception time information is allocated to the data received from the adjacent sudden recognition control device, the repeater unit 252 synchronizes the time information to be responded based on the time information of the received data. The transmission/reception unit 250 may be controlled to transmit information to another adjacent accidental recognition control device based on the set time.

5 is a diagram illustrating an example in which an accidental recognition control device transmits data to a traffic information center according to an embodiment of the present invention.

Referring to FIG. 5, in the emergency situation monitoring control system 10 through data fusion according to an embodiment of the present invention, the second accidental recognition control device 200-A2 in the shaded area is in the shaded area. Information of the recognition control device 200-A1 is collected. Here, the shaded area includes an area where the accidental recognition control device is not connected to the traffic information center 400 by wire or wirelessly, or out of the control area of the traffic information center 400.

And, the second accidental recognition control device (200-A2) transmits its own information together with the information collected by the first accidental recognition control device (200-A1) to the adjacent third accidental recognition control device (200-A3). do. In addition, the third accidental recognition control device 200-A3 communicates its own information together with the information of the first accidental recognition control device 200-A1 and the second accidental recognition control device 200-A2 in the shaded area. It is transmitted to the information center 400.

Through this, the traffic information center 400 maintains not only the third accidental recognition control device 200-A3 directly connected by wire or wirelessly, but also the accidental recognition control devices 200-A1 and 200-A2 in the shaded area. Information, information on a detection device connected thereto, and various traffic information and unexpected situation information detected from the detection device can be effectively collected.

Accordingly, the sudden situation monitoring control system 10 through data fusion according to an embodiment of the present invention provides information on devices in a shadow area, traffic information, and unexpected situations in a shadow area through the repeater function of the accident recognition control device. It provides an environment that can be effectively collected and easily delivered to the traffic information center.

6 is a diagram schematically showing the configuration of an information collection terminal according to an embodiment of the present invention. At this time, the information collection terminal 300 only shows a schematic configuration necessary for description according to an embodiment of the present invention, but is not limited to this configuration.

The information collection terminal 300 according to an embodiment of the present invention is mounted on a mobile vehicle A and is connected to a plurality of accidental recognition control devices through wireless communication to provide maintenance information and road traffic of a plurality of accidental recognition control devices. Gather information. In addition, the information collection terminal 300 according to an embodiment of the present invention provides the collected information to the traffic information center 400.

6, the information collection terminal 300 according to an embodiment of the present invention includes a meter reading signal transmission unit 310, a carrier wave detection unit 320, a meter reading signal retransmission control unit 330, and an information collection unit 340. Include.

The meter reading signal transmitting unit 310 transmits the meter reading signal to a plurality of sudden recognition control devices 200-C1 to 200-CX.

The carrier wave detection unit 320 detects a carrier wave transmitted from at least one sudden recognition control device that has received the meter reading signal transmitted from the meter reading signal transmission unit 310 among a plurality of accidental recognition control devices 200-C1 to 200-CX. do.

The meter reading signal retransmission control unit 330 controls the meter reading signal transmission unit 310 to retransmit the meter reading signal after a predetermined time delay when a plurality of carriers are detected in the same frequency band. For example, when a plurality of carriers are detected in the same frequency band, data loss and breakage may occur due to RF collision, so the meter reading signal retransmission control unit 330 controls to retransmit the meter reading signal after a predetermined delay. .

The information collection unit 340 recognizes the accidental recognition control device that has transmitted the corresponding carrier based on the carrier detected by the carrier wave detection unit 320, and whether there is an abnormality in the accidental recognition control devices or the accidental recognition control device that has transmitted the carrier. It determines whether there is an abnormality in the connected detection device and transmits it to the traffic information center 400.

FIG. 7 is a diagram illustrating an example in which an information collection terminal wirelessly collects information from an accidental recognition control device and transmits it to a traffic information center according to an embodiment of the present invention.

Referring to FIG. 5, the emergency situation monitoring control system 10 through data fusion according to an embodiment of the present invention includes a plurality of accidental recognition control devices through the information collection terminal 300 mounted on the mobile vehicle A. 200-C1 to 200-CX) easily collects information on the shadow area, effectively grasps the presence or absence of an abnormality in the detection device or accidental recognition control device, and through this, a faulty detection device or malfunctioning accidental recognition control device It provides an environment that can be easily maintained.

8 is a flowchart schematically illustrating a process of generating traffic information by fusing image data and radar data and recognizing/tracking an unexpected situation according to an embodiment of the present invention. In this case, the following flowchart will be described using the same reference numerals in connection with the configurations of FIGS. 1 to 7.

Referring to FIG. 8, the emergency monitoring control system 10 through data fusion according to an embodiment of the present invention acquires image data from the image camera 110 (S102).

In addition, the emergency situation monitoring control system 10 through data fusion according to an embodiment of the present invention acquires radar data from the radar sensor 120 (S104).

In addition, the emergency situation monitoring control system 10 through data fusion according to an embodiment of the present invention fuses the analysis result of the image data and the image data, and the analysis result of the radar data and the radar data. Object information and object features are extracted (S106, S108).

Further, the emergency situation monitoring control system 10 through data fusion according to an embodiment of the present invention generates traffic information based on the data analysis result, fusion data, and object information, and recognizes the unexpected situation (S110).

In addition, the unexpected situation monitoring control system 10 through data fusion according to an embodiment of the present invention photographs/tracks and monitors the unexpected situation through the tracking camera 130 when the unexpected situation is recognized (S112).

9 is a flowchart illustrating a process of extracting object features by fusion of image data and radar data and detecting an unexpected situation based on machine learning according to an embodiment of the present invention. In this case, the following flowchart will be described using the same reference numerals in connection with the configurations of FIGS. 1 to 7.

Referring to FIG. 9, the emergency monitoring control system 10 through data fusion according to an embodiment of the present invention acquires image data from the image camera 110 and generates a list of tracking objects by applying a Gaussian mixed model. Do (S202 to S208).

In addition, the emergency situation monitoring control system 10 through data fusion according to an embodiment of the present invention acquires radar data from the radar sensor 120, and undergoes noise/diffuse reflection signal processing and 2D/3D image processing. The object features of are extracted (S210 to S216).

In addition, the sudden situation monitoring control system 10 through data fusion according to an embodiment of the present invention includes an object list based on an analysis result of image data and an image based on object characteristics based on the analysis result of radar data. The data and radar data are fused, and image object features are extracted (S218 and S220).

In addition, the emergency situation monitoring control system 10 through data fusion according to an embodiment of the present invention learns and classifies the fused features based on machine learning to detect the unexpected situation (S222 to S226).

In addition, the unexpected situation monitoring control system 10 through data fusion according to an embodiment of the present invention receives tracking/road surface information and generates a tracked object list through Kalman tracking correction/prediction, The accuracy and reliability of recognition are improved (S228 and S230).

As described above, the emergency monitoring control system through data fusion according to an embodiment of the present invention processes and fuses image data and radar data to improve the resolution of the data and effectively recognize and monitor the emergency situation. Provide the environment.

In addition, the present invention provides an environment capable of effectively detecting an unexpected situation and actively responding to an unexpected situation by detecting an abnormally moving vehicle in a reverse driving direction or a left-right direction with respect to a normal moving direction of a lane using the Doppler effect.

In addition, the present invention provides an environment in which, by applying a zero padding algorithm, a moving speed resolution of a moving object can be improved, and a limitation of resolution improvement due to time constraints required for information update can be overcome.

In addition, the present invention minimizes the noise ratio of the received signal to the transmitted signal by applying a side lobe attenuation algorithm, and provides an environment in which object information and object features can be effectively extracted.

In addition, the present invention generates fusion data of image data and radar data by applying an over-sampling algorithm, thereby minimizing limitations due to hardware limitations and time limitations that occur when disparate data is fused, It provides an environment to improve the resolution of data.

In addition, the present invention provides an environment in which a blind spot of traffic information is removed by collecting traffic information and an unexpected situation in a shadow area through a repeater function, and traffic safety and quality of traffic information can be improved.

In addition, the present invention collects information of the detection device and the sudden recognition control device arranged in the shaded area through the data collection function of the information collection terminal, effectively grasping the presence or absence of an abnormality in the detection device or the sudden recognition control device, and the detection device and It provides an environment in which the accidental recognition control device can be easily maintained.

The embodiments of the present invention described above are not implemented only through an apparatus and a method, but may be implemented through a program that realizes a function corresponding to the configuration of the embodiment of the present invention or a recording medium on which the program is recorded. Such a recording medium can be executed not only in the server but also in the user terminal.

Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements by those skilled in the art using the basic concept of the present invention defined in the following claims are also provided. It belongs to the scope of rights.

Claims (10)

A detection device including at least one of an image camera, a radar sensor, or a tracking camera, and
Analyzing the data collected from the detection device, and fusing the image data of the video camera and the radar data of the radar sensor to recognize traffic information on the road and an accidental recognition control device,
The accidental recognition control device,
An unexpected situation monitoring control system through data fusion that analyzes the fusion data to recognize an unexpected situation, but controls the tracking camera to track the unexpected situation when an unexpected situation is detected on a road. In claim 1,
The accidental recognition control device,
Using the Doppler effect, an abnormal moving vehicle with respect to the normal moving direction of the lane is detected, but after detecting the moving speed of the moving object by analyzing the wavelength of the transmission signal and the received signal of the radar sensor, the moving object is compared with the normal moving direction Reverse driving determination unit to determine whether to run in reverse or move in the left and right directions
An unexpected situation monitoring control system through data fusion including a. In paragraph 2,
The accidental recognition control device,
Zero padding that signal-processes the radar data periodically collected from the radar sensor to remove noise or diffuse reflection, and processes the signal of the time axis as a zero value for a time corresponding to an integer multiple after the input signal of a certain period with respect to the periodic signal ( Noise improvement unit that improves the resolution of moving speed of moving object by applying Zero Padding) algorithm
Incident monitoring control system through data fusion further comprising a. In paragraph 3,
The accidental recognition control device,
Object information and objects are recognized by analyzing the received signal reflected from the moving object for the transmission signal of the radar sensor, but applying a side lobe attenuation algorithm to minimize the noise ratio of the received signal to the transmitted signal. Feature extraction unit to extract features
Incident monitoring control system through data fusion further comprising a. In claim 4,
The accidental recognition control device,
The analysis result of the image data and the analysis result of the radar data are fused, each data is sampled at a time period, converted from an analog signal to a digital signal, and fused, and an over-sampling algorithm is applied to generate fusion data. And a data fusion unit for fusing object features of a moving object through spatial matching between the image data and the radar data.
Incident monitoring control system through data fusion further comprising a. In claim 1,
The accidental recognition control device,
Transmitting/receiving unit that collects information by being connected to the adjacent accidental recognition control device by wire or wirelessly, and transmits the collected information to other adjacent accidental recognition control devices or to the traffic information center
An unexpected situation monitoring control system through data fusion including a. In paragraph 6,
The transceiver unit,
A repeater unit for controlling transmission/reception time information to be allocated and to transmit and receive signals in synchronization with the allocated transmission/reception time information, but to transmit information to the surrounding accidental recognition control device only at a specified time according to the transmission/reception time information
An unexpected situation monitoring control system through data fusion including a. In paragraph 6,
An information collection terminal mounted on a mobile vehicle and connected to a plurality of accidental recognition control devices through wireless communication to collect information from a plurality of accidental recognition control devices.
It further includes,
The above information collection terminal,
Meter reading signal transmission unit for transmitting the meter reading signal to a plurality of sudden recognition control devices,
A carrier wave detection unit for detecting a carrier wave transmitted from at least one of the plurality of sudden recognition control apparatuses that received the meter reading signal, and
An information collection unit that determines whether there is an abnormality in the accidental recognition control device that transmitted the carrier wave or if there is an abnormality in the detection device connected to the accidental recognition control device that transmitted the carrier wave, and transmits it to the traffic information center.
An unexpected situation monitoring control system through data fusion including a. In clause 8,
The above information collection terminal,
Meter reading signal retransmission control unit that controls to retransmit the meter reading signal after a certain time delay when multiple carriers are detected in the same frequency band
Incident monitoring control system through data fusion further comprising a. In clause 7,
The accidental recognition control device,
The road surface condition is determined through a road surface condition determination algorithm that uses the polarization of light or machine learning to determine the road surface condition, and the determined road surface condition information is transmitted to a nearby accidental recognition control device through wireless communication. Emergency monitoring and control system through data fusion that is transmitted or transmitted to the traffic information center.

Images