KR102045222B1 - System for sensing and control unexpected situation using data fusion - Google Patents

Abstract

According to the present invention, provided is a sudden situation monitoring and controlling system through data fusion, which comprises a detection device including at least one of a video camera, a radar sensor, or a tracking camera, and a sudden recognition controlling device analyzing data collected from the detection device, and fusing video data of the video camera and radar data of the radar sensor to recognize traffic information and a sudden situation on the road. The sudden recognition controlling device analyzes the fusion data to recognize a sudden situation, and controls the tracking camera to track the sudden situation when detecting the sudden situation on the road. Therefore, the system improves the resolution of the data, and provides an effect of effectively recognizing and monitoring the sudden situation.

Description

Accident situation monitoring and control system through data fusion {SYSTEM FOR SENSING AND CONTROL UNEXPECTED SITUATION USING DATA FUSION}

The present invention relates to a sudden situation monitoring control system through the fusion of image data and radar data.

With the development of the Intelligent Transport System (ITS), roads are becoming increasingly intelligent. In addition, road monitoring cameras are widely installed and distributed, such as monitoring vehicles and pedestrians through cameras such as CCTV and image detectors. In addition, recently, methods such as tracking or classifying a vehicle using a laser sensor to provide information to a driver or supporting unmanned autonomous driving have been developed.

Also, in recent years, accidents on roads or in tunnels such as traffic accidents, stopped vehicles, falling objects, pedestrians, and low-speed vehicles may cause chain collisions or secondary accidents. A technology for quickly detecting and informing a rear vehicle through road signs, roadside broadcasting, and inter-vehicle communication has been developed.

In addition, as a prior art document related to the present invention to be described later, the heterogeneous sensor fusion processing apparatus and method for the classification and position detection of road objects in the Republic of Korea Patent Publication No. 10-2013-0127822 (published on November 25, 2013) There is.

However, in the related art, the resolution is degraded due to hardware limitations and time constraints required to update information, and thus, there is a difficulty in not accurately detecting a moving object or accurately detecting a sudden situation.

In addition, there are difficulties in maintaining and maintaining millions of CCTVs and controllers in the past, and it is difficult to effectively cope with unexpected situations by failing to collect traffic information from CCTVs disposed in shaded areas.

Matters described in this Background section are intended to enhance the understanding of the background of the invention, and may include matters other than the prior art already known to those skilled in the art.

An object of the present invention is to propose an accident monitoring and control system through data fusion, which can improve resolution by signal processing image data and radar data, and can effectively recognize and monitor an unexpected situation.

In addition, the present invention can easily collect the maintenance information of the detection device and the sudden recognition control device disposed in the shadow area, the traffic information and the incident situation of the shadow area to remove the blind spot of the traffic information, the traffic safety and traffic information In this paper, we propose a supervisory monitoring and control system through data convergence to improve quality.

Analyzing a sudden monitoring and control image camera, a radar sensor, or a tracking camera through the data fusion of the present invention, and analyzes the data collected from the detection device, and the image data of the video camera and the radar And a sudden recognition control device that fuses radar data of a sensor to recognize traffic information and an unexpected situation on a road, wherein the sudden recognition control device recognizes an unexpected situation by analyzing the fusion data, but detects an unexpected situation on a road. When the tracking camera is controlled to track the accident.

The sudden recognition control device detects an abnormal moving vehicle in a normal moving direction of the lane by using a Doppler effect, analyzes the wavelengths of the transmitted signal and the received signal of the radar sensor, detects the moving speed of the moving object, and then detects the specific speed of the moving object. It may include a reverse driving determination unit for determining whether the reverse driving or the left and right moving in comparison with the normal moving direction of the.

The abrupt recognition control device processes the radar data collected periodically from the radar sensor to remove noise or diffuse reflection, and zeros the signal on the time axis for a time corresponding to an integer multiple after a predetermined period input signal with respect to the periodic signal. The apparatus may further include a noise improving unit configured to improve a moving speed resolution of the moving object by applying a zero padding algorithm for signal processing.

The abrupt recognition control device recognizes object information by analyzing a received signal reflected from a moving object with respect to the transmitted signal of the radar sensor, and has a side lobe attenuation algorithm to minimize noise ratio of the received signal to the transmitted signal. The apparatus may further include a feature extractor configured to extract object information and object features by applying.

The abrupt recognition control device may be configured to fuse the analysis result of the image data and the analysis result of the radar data, sample each data at a time period, convert the analog signal into a digital signal, and fuse the oversampling algorithm. The apparatus may further include a data fusion unit configured to generate fusion data and to fuse object features of a moving object through spatial matching of the image data and the radar data.

The sudden recognition control device may be connected to an adjacent sudden recognition control device to collect information by wire or wirelessly, and includes a transceiver for transmitting the collected information to another adjacent sudden recognition control device or a traffic information center. can do.

The transceiver unit may include a repeater unit configured to transmit and receive time information, and to transmit and receive a signal in synchronization with the allocated transmission / reception time information, and to transmit information to a peripheral sudden recognition control device only at a designated time according to the transmission / reception time information. Can be.

It is mounted on a mobile vehicle, and connected to a plurality of sudden recognition control device by wireless communication further comprises an information collecting terminal for collecting information from the plurality of sudden recognition control device, wherein the information collecting terminal, a plurality of sudden recognition signal reading signal Meter reading signal transmitting unit for transmitting to a control device, a carrier detecting unit for detecting a carrier transmitted from at least one of the sudden recognition control device receiving the reading signal among the plurality of sudden recognition control device, and a sudden recognition control device transmitting the carrier It may include an information collecting unit for determining whether there is an error or whether there is an abnormality of the detection device connected to the sudden recognition control device that transmitted the carrier to transmit to the traffic information center.

The information collecting terminal may further include a metering signal retransmission control unit which controls to retransmit the metering signal after a predetermined time delay when a plurality of carriers are detected in the same frequency band.

The abrupt recognition control device determines a road surface state through a road state determination algorithm using a polarization phenomenon of light or a machine learning to determine a road state, and the road surface information of the road is determined by wireless communication. It can be sent to an adjacent accident control device or to a traffic information center.

According to the present invention, by processing the image data and the radar data by signal processing to improve the resolution of the data, to provide an environment that can effectively recognize and monitor the unexpected situation.

In addition, the present invention provides an environment capable of effectively detecting an unexpected situation and actively responding to an unexpected situation by detecting an abnormal moving vehicle in a reverse driving direction or a left and right direction with respect to the normal moving direction of the lane using the Doppler effect.

In addition, the present invention by applying a zero padding (Zero Padding) algorithm, improves the moving speed resolution of the moving object, and provides an environment that can overcome the limitations of the resolution improvement due to the time constraints required to update the information.

In addition, the present invention by applying a side lobe attenuation algorithm to minimize the noise ratio of the transmitted signal to the received signal, and provides an environment that can effectively extract the object information and object features.

In addition, the present invention generates the fusion data of the image data and the radar data by applying the Over Sampling algorithm, thereby minimizing the limitations due to the limitations of hardware and time that occur during the fusion of heterogeneous data, It provides an environment to improve the resolution of the data.

In addition, the present invention by collecting the traffic information and the incident of the shaded area through the repeater function, to remove the blind spot of the traffic information, to provide an environment that can improve the traffic safety and quality of the traffic information.

In addition, the present invention by collecting the information of the detection device and the sudden recognition control device disposed in the shadow area through the data collection function of the information collecting terminal, to effectively detect the abnormality of the detection device or the sudden recognition control device, and the detection device and Provides an environment that can easily maintain a sudden recognition control device.

1 is a view showing a schematic configuration of a system for monitoring and monitoring a sudden situation through data fusion according to an embodiment of the present invention.
2 is a view briefly showing the configuration of the sudden recognition control device according to an embodiment of the present invention.
3 illustrates data before zero padding and after zero padding according to an embodiment of the present invention.
4 is a diagram illustrating an example of detecting an abnormal moving vehicle using the Doppler effect according to an embodiment of the present invention.
FIG. 5 is a diagram illustrating an example in which a sudden recognition control device transmits data to a traffic information center according to an embodiment of the present invention.
6 is a diagram schematically showing the configuration of an information collecting terminal according to an embodiment of the present invention.
FIG. 7 is a diagram illustrating an example in which an information collecting terminal wirelessly collects information of a sudden recognition control device and transmits the information to a traffic information center according to an embodiment of the present invention.
8 is a flowchart schematically illustrating a process of generating traffic information by recognizing image data and radar data and recognizing / tracking a sudden situation according to an embodiment of the present invention.
9 is a flowchart illustrating a process of extracting an object feature by fusing image data and radar data and detecting a sudden situation based on machine learning according to an embodiment of the present invention.

DETAILED DESCRIPTION Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art may easily implement the present invention. As those skilled in the art would realize, the described embodiments may be modified in various different ways, all without departing from the spirit or scope of the present invention. In the drawings, parts irrelevant to the description are omitted in order to clearly describe the present invention, and like reference numerals designate like parts throughout the specification.

Throughout the specification, when a part is said to "include" a certain component, it means that it can further include other components, without excluding other components unless specifically stated otherwise. In addition, the terms “… unit”, “… unit”, “module”, etc. described in the specification mean a unit that processes at least one function or operation, which may be implemented by hardware or software or a combination of hardware and software. have.

Referring now to Figures 1 to 9 will be described in detail the incident monitoring control system through data fusion according to an embodiment of the present invention.

1 is a view showing a schematic configuration of a system for monitoring and monitoring a sudden situation through data fusion according to an embodiment of the present invention. At this time, the incident monitoring and control system 10 through data fusion only shows a schematic configuration necessary for the description according to an embodiment of the present invention, but is not limited to this configuration.

Referring to FIG. 1, an incident monitoring and control system 10 through data fusion according to an embodiment of the present invention may include a detection device 100, a sudden recognition control device 200, an information collection terminal 300, and traffic. Information center 400 is included.

In the incident monitoring and control system 10 through data fusion according to an embodiment of the present invention, the sudden recognition control device 200 collects data from the detection device 100 in which an image camera and a radar device are integrated.

The detection device 100 is installed on roads such as curved sections, intersections, tunnels, and the like, and collects traffic related data and transmits it to the sudden recognition control device 200. Here, the detection device 100 is installed at least one of the image camera 110, the radar sensor 120, and the tracking camera 130 is integrated. In addition, the tracking camera 130 may detect the road under the control of the sudden recognition control device 200 and photograph the sudden situation. For example, the sudden recognition control device 200 may collect traffic-related data from CCTV and the like, and track the unexpected situation.

In addition, the incident monitoring control system 10 through the data fusion according to an embodiment of the present invention is a sudden recognition control device 200 is the traffic information center 400 and Ethernet (ehthernet) based TCP / IP communication method or It can be connected and controlled by wireless communication. At this time, the sudden recognition control device 200 may analyze the control signals of the traffic information center 400 to perform the functions of the zoom lens, pan / tilt, and radar sensor of the video camera and the tracking camera.

In addition, the abrupt situation monitoring control system 10 through data fusion according to an embodiment of the present invention applies a Gaussian mixture model according to whether the image data acquired from the image camera 110 is in a high speed mode, and a tracking object list algorithm. Apply.

In addition, the abrupt situation monitoring control system 10 through data fusion according to an embodiment of the present invention acquires radar data from the radar sensor 120 by using a K-band beam, and processes noise / reflection reflection signal and 2D / An algorithm that extracts object features through 3D image processing is applied.

In addition, the abrupt situation monitoring control system 10 through data fusion according to an embodiment of the present invention extracts highly reliable object information by matching characteristics of objects extracted from the video camera and the radar sensor. In addition, the incident monitoring and control system 10 through data fusion according to one embodiment of the present invention tracks / corrects or predicts an unexpected situation through a Kalman technique, and detects information related to the unexpected situation 100. Or by transmitting to the sudden recognition control device 200 provides an environment that can effectively monitor the unexpected situation.

In addition, the incident monitoring and control system 10 through the data fusion according to an embodiment of the present invention, the information acquisition terminal 300 installed in the maintenance vehicle wirelessly with the sudden recognition control device 200 equipped with a wireless communication technology Communication to monitor the current state of the detection device 100 or the sudden recognition control device 200, and transmits the monitored information to the traffic information center (400).

In addition, the incident monitoring control system 10 through the data fusion according to an embodiment of the present invention transfers traffic information and maintenance information between the sudden recognition control device 200, or the information collection terminal 300 traffic information Collecting and maintaining information and passing it to the traffic information center (400), eliminating blind spots of public and private traffic information, improve the quality of traffic safety and traffic information, thereby overcoming the limitations of the existing detection system Provide an environment that can

2 is a view briefly showing the configuration of the sudden recognition control device according to an embodiment of the present invention. At this time, the sudden recognition control device 200 shows only the schematic configuration necessary for the description according to an embodiment of the present invention, but is not limited to this configuration.

Referring to FIG. 2, the sudden recognition control device 200 according to an embodiment of the present invention includes a sudden recognition control module 210, a data collection unit 220, a signal processing unit 230, a determination unit 240, and It includes a transceiver 250.

The sudden recognition control module 210 analyzes the data collected from the detection device 100 and controls the tracking camera 130 to monitor the unexpected situation. In addition, the sudden recognition control module 210 analyzes the data of the image camera 110 and the radar sensor 120, and fuses the image data of the image camera 110 and the radar data of the radar sensor 120, traffic on the road Recognize information and accidents. In addition, the sudden recognition control module 210 may track the unexpected situation by controlling the tracking camera 130 when a sudden situation is detected on the road.

The data collector 220 collects data of the image camera 110 and the radar sensor 120. In addition, the data collector 220 includes an image data collector 222, a radar data collector 224, and a road surface data collector 226 according to an exemplary embodiment.

The image data collector 222 collects image data from the image camera 110. For example, the image data collector 222 may collect 2D image data captured by the image camera 110.

The radar data collector 224 collects radar data from the radar sensor 120. For example, the radar data collector 224 may collect 2D radar data detected by the radar sensor 120.

The road surface data collection unit 226 collects information related to road surface conditions. For example, the road surface data collection unit 226 may use the polarization of light to determine the road state of the road, so that the road surface captured by the image camera 110 and the tracking camera 130 may be vertically polarized data and horizontally polarized light. You can collect data.

The signal processor 230 signal-processes the data collected by the data collector 220 to improve the resolution of the data or to improve the reliability of object information extraction through data fusion.

The signal processor 230 includes a noise improver 232, a feature extractor 234, and a data fusion unit 236 according to an embodiment of the present invention.

The noise improvement unit 232 removes noise or diffuse reflection by signal processing the radar data periodically collected from the radar sensor 120. The noise improvement unit 232 may improve the moving speed resolution of the moving object by applying a zero padding algorithm.

For example, the noise improvement unit 232 may be applied to an integer multiple (2 times, 4 times, etc.) after a predetermined period input signal with respect to the periodic signal of the signal detected from the radar sensor 120 through a zero padding algorithm. The signal on the time base can be signaled to a zero value ("0") for the corresponding time.

3 illustrates data before zero padding and after zero padding according to an embodiment of the present invention.

Referring to FIG. 3, the abrupt situation monitoring control system 10 through data fusion according to an embodiment of the present invention is a) zero padding (zero padding) than data before zero padding without applying a zero padding algorithm. B) The zero-padding data using the algorithm is significantly improved in resolution.

Through this, the abrupt situation monitoring control system 10 through data fusion according to an embodiment of the present invention improves the moving speed resolution of a moving object and overcomes the limitation of resolution improvement due to time constraints required for information update. Provide an environment for doing this.

The feature extractor 234 extracts and recognizes object information of the moving object by analyzing image data of the image camera 110 and radar data of the radar sensor 120. Also, the feature extractor 234 recognizes the object information by analyzing the received signal reflected from the moving object with respect to the transmission signal of the radar sensor 120.

In addition, the feature extractor 234 controls to increase the output of the transmission signal of the radar sensor 120 to a predetermined value or more, and a side lobe attenuation algorithm for minimizing the noise ratio of the reception signal to the transmission signal. It can be applied to effectively extract object information and object features.

For example, the feature extractor 234 may include a phase and a frequency between the received signal received by the radio wave transmitted by the transmitter (not shown) of the radar sensor 120 and the received signal reflected by the object to be detected. The distance or shape of an object can be detected based on the deviation. In this case, the shape of the object is roughly displayed according to the resolution of the received signal reflected on the object, and data about the object shape is extracted based on the measured result.

However, if the parasitic generation signal around the radar is generated as well as the main parasitic signal or the side lobe is large, the power distribution of the main divergence signal is lowered so that the detection signal transmitted is small, and the peripheral object signal detected by the side lobe signal is a noise signal. The noise ratio of the original signal becomes worse.

Therefore, the incident monitoring control system 10 through data fusion according to an embodiment of the present invention increases the output of the radar transmission signal to a predetermined value or more, and the side lobe (Side) to minimize the noise ratio of the reception signal to the transmission signal Lobe) By applying the attenuation algorithm, we provide an environment that can effectively extract object information and object features.

The data fusion unit 236 fuses the analysis result of the image data obtained from the image camera 110 with the analysis result of the radar data obtained from the radar sensor 120.

The data fusion unit 236 samples each data at a time period, converts the analog signal into a digital signal, and fuses the data. In this case, in order to extract accurate data during signal processing for data fusion, sampling of signals may be performed a lot, but limitations occur due to limitations in hardware and time.

Accordingly, the data fusion unit 236 generates fusion data by applying an over sampling algorithm that samples a signal at twice the bandwidth or the highest sampling frequency that can be sampled. The data fusion unit 236 fuses the object feature of the moving object through spatial matching of the image data and the radar data. In addition, the abrupt situation monitoring control system 10 through data fusion according to an embodiment of the present invention effectively measures the position, distance, size, shape, and the like of a fixed object or a moving object through improved fusion data and object information. Can be determined.

As described above, the abrupt situation monitoring control system 10 through data fusion according to an embodiment of the present invention applies an over-sampling algorithm to generate fusion data of image data and radar data, thereby fusion of heterogeneous data. Minimize the limitations due to hardware limitations and time limitations and provide an environment that can improve the resolution of data.

The determination unit 240 recognizes the traffic information and the accident on the road based on the analysis result of the data collected from the detection device 100. For example, the determination unit 240 may detect the abnormal moving vehicle traveling in the reverse driving direction by analyzing the fusion data. In addition, the determination unit 240 may determine the road surface state of the road based on the analysis result of the data collected from the detection apparatus 100.

The determination unit 240 includes a sudden situation recognition unit 242, a reverse driving determination unit 244, a sudden situation tracking unit 246, and a road surface state determination unit 248 according to an embodiment of the present invention.

The unexpected situation recognition unit 242 analyzes the data and the fusion data collected from the detection apparatus 100 to recognize the unexpected situation. For example, the sudden situation recognition unit 242 may recognize a sudden situation caused by a traffic accident and the resulting stop vehicle, a slow moving object, a rockfall, a shoulder stop object, and the like.

The reverse driving determination unit 244 analyzes the fusion data or the radar data to recognize the unexpected information, and detects an abnormal moving vehicle in the reverse driving direction or the left and right directions with respect to the normal moving direction of the lane by using the Doppler effect.

In addition, the reverse driving determiner 244 analyzes the wavelengths of the transmission signal and the reception signal of the radar sensor, detects the moving speed of the moving object, and compares the moving direction with the normal moving direction of the specific lane to determine whether the driving direction is reversed or left and right. . For example, the reverse driving determiner 244 may recognize whether the moving direction of the object according to the absolute value of the Doppler shift frequency is abnormally moved according to the difference from the average moving direction.

In addition, the reverse driving determining unit 244 may detect the speed in the left and right directions or the reverse driving direction of the abnormal moving vehicle with respect to the average moving direction of the road by using the Doppler effect.

4 is a diagram illustrating an example of detecting an abnormal moving vehicle using the Doppler effect according to an embodiment of the present invention.

For example, the sudden recognition control device 200 according to an embodiment of the present invention calculates the Doppler shift by the wavelength of the radar signal used for the speed difference between the radar sensor and the moving object. In addition, the sudden recognition control device 200 according to an embodiment of the present invention assumes the reference of the speed difference as a radar sensor, and divides the speed shift between objects that are far away and objects that are closer to each other by a positive / negative sign. After detecting the moving speed of the object, it is possible to determine whether the vehicle is driving backward or left and right by comparing with the normal moving direction of the corresponding lane.

Referring to FIG. 4, the sudden recognition control device 200 according to an embodiment of the present invention compares the radar transmission signal x with the reception signals y1 and y2 reflected from the moving object, and whether the vehicle is reverse driving or left and right driving. Can be determined.

For example, in the case of the reception signal y1, the distance between the radar sensor and the moving object decreases to increase the frequency of the wave, and the reception signal y1 is converted into the reverse driving vehicle compared to the normal moving direction of the corresponding lane. You can judge.

When the accident situation tracking unit 246 detects the accident situation on the road to control the tracking camera 130 to track the accident situation.

The road surface state determination unit 248 determines the road surface state of the road through a road surface state determination algorithm using light polarization or machine learning to determine the road surface state. And, the incident monitoring and control system 10 through the data fusion according to an embodiment of the present invention transmits the road condition information of the determined road to the surrounding sudden recognition control device or information collection terminal by wireless communication, or collected Road condition information can be transmitted to the traffic information center.

The transceiver 250 collects information by being connected to a sudden recognition control device adjacent to the wire or wirelessly. In addition, the transceiver 250 transmits the collected information to another adjacent sudden recognition control device 200, the information collecting terminal 300, and the traffic information center 400.

The transceiver 250 includes a repeater 252 according to an embodiment of the present invention.

The repeater unit 252 controls the transceiver 250 to transmit its information to an adjacent sudden recognition control device. In addition, when the transmission / reception time information is allocated, the repeater 252 may control the transmission / reception unit 250 to automatically transmit information only at a designated time according to the allocated transmission / reception time information.

For example, the repeater unit 252 is assigned an ID and transmission and reception time information for each sudden recognition control device, and controls the transmission and reception unit 250 to transmit data at the time that the user should respond based on the allocated transmission and reception time information. can do.

In addition, when the transmission and reception time information is assigned to the data received from the adjacent sudden recognition control device, the repeater unit 252 synchronizes the time information to which it should respond based on the time information of the received data, at this time synchronization The transmitter / receiver 250 may be controlled to transmit information to another adjacent sudden recognition control device based on the received time.

FIG. 5 is a diagram illustrating an example in which a sudden recognition control device transmits data to a traffic information center according to an embodiment of the present invention.

Referring to FIG. 5, in the incident monitoring and control system 10 through data fusion according to an embodiment of the present invention, the second sudden recognition control device 200-A2 in the shadowed area may have a first flash in the shadowed area. Collect information of the recognition control device 200-A1. Here, the shaded area includes an area where the sudden recognition control device is not connected to the traffic information center 400 by wire or wirelessly, or is outside the control area of the traffic information center 400.

In addition, the second sudden recognition control device 200-A2 transmits its own information together with the information collected by the first sudden recognition control device 200-A1 to the adjacent third sudden recognition control device 200-A3. do. In addition, the third sudden recognition control device 200-A3 transmits its own information together with the information of the first sudden recognition control device 200-A1 and the second sudden recognition control device 200-A2 in the shaded area. Send to the information center (400).

Through this, in the traffic information center 400, maintenance of the sudden recognition control devices 200-A1 and 200-A2 in the shaded area as well as the third sudden recognition control device 200-A3 directly connected by wire or wirelessly. The information, information of the detection device connected thereto, various traffic information detected by the detection device, and accidental situation information can be effectively collected.

Therefore, the incident monitoring control system 10 through data fusion according to an embodiment of the present invention through the repeater function of the sudden recognition control device information of the devices in the shadow area, traffic information and the sudden situation of the shadow area, etc. It provides an environment that can be effectively collected and easily communicated to a traffic information center.

6 is a diagram schematically showing the configuration of an information collecting terminal according to an embodiment of the present invention. In this case, the information collecting terminal 300 only shows a schematic configuration necessary for explanation according to an embodiment of the present invention, but is not limited to this configuration.

Information collection terminal 300 according to an embodiment of the present invention is mounted on the mobile vehicle (A), is connected to the plurality of sudden recognition control device by wireless communication maintenance information of the plurality of sudden recognition control device and traffic on the road Collect information. In addition, the information collecting terminal 300 according to an embodiment of the present invention provides the collected information to the traffic information center 400.

Referring to FIG. 6, the information collecting terminal 300 according to an embodiment of the present invention uses a metering signal transmitter 310, a carrier detector 320, a meter signal retransmission controller 330, and an information collector 340. Include.

The meter reading signal transmitting unit 310 transmits a meter reading signal to the plurality of sudden recognition control devices 200-C1 to 200-CX.

The carrier detecting unit 320 detects a carrier wave transmitted from at least one abrupt recognition control device that receives the meter reading signal transmitted from the meter reading signal transmitting unit 310 among the plurality of sudden recognition control devices 200-C1 to 200-CX. do.

The meter signal retransmission control unit 330 controls the meter signal transmitter 310 to retransmit the meter signal after a predetermined time delay when a plurality of carriers are detected in the same frequency band. For example, when a plurality of carriers are detected in the same frequency band, data loss and breakage may occur due to RF collision, and thus, the meter signal retransmission controller 330 controls to retransmit the meter signal after a predetermined time delay. .

The information collecting unit 340 recognizes a sudden recognition control device that transmits the carrier on the basis of the carrier detected by the carrier detection unit 320, the presence or absence of abnormality detection control devices or the sudden recognition control device that transmits the carrier; The abnormality of the connected detection device is determined and transmitted to the traffic information center 400.

FIG. 7 is a diagram illustrating an example in which an information collecting terminal wirelessly collects information of a sudden recognition control device and transmits the information to a traffic information center according to an embodiment of the present invention.

Referring to FIG. 5, a sudden situation monitoring control system 10 through data fusion according to an embodiment of the present invention may include a plurality of sudden recognition control devices through an information collecting terminal 300 mounted in a mobile vehicle A. By collecting the information of 200-C1 to 200-CX), it is easy to collect the information of the shadowed area, and effectively detects the abnormality of the detection device or the sudden recognition control device, and through this, the failed detection device or the failed sudden recognition control device. It provides an environment that can be easily maintained.

8 is a flowchart schematically illustrating a process of generating traffic information by recognizing image data and radar data, and recognizing / tracking a sudden situation according to an embodiment of the present invention. At this time, the following flow chart will be described using the same reference numerals in connection with the configuration of Figs.

Referring to FIG. 8, the abrupt situation monitoring control system 10 through data fusion according to an embodiment of the present invention obtains image data from the image camera 110 (S102).

In addition, the abrupt situation monitoring control system 10 through data fusion according to an embodiment of the present invention obtains radar data from the radar sensor 120 (S104).

In addition, the abrupt situation monitoring control system 10 through the data fusion according to an embodiment of the present invention fused the analysis results of the image data and image data, and the analysis results of the radar data and radar data, based on the road The object information and the object feature are extracted (S106 and S108).

In addition, the incident monitoring control system 10 through data fusion according to an embodiment of the present invention generates traffic information based on the data analysis result, the fusion data and the object information, and recognizes the accident (S110).

In addition, the sudden situation monitoring control system 10 through the data fusion according to an embodiment of the present invention, when the sudden situation is recognized through the tracking camera 130 to capture / track the situation and monitor (S112).

9 is a flowchart illustrating a process of extracting an object feature by fusing image data and radar data and detecting a sudden situation based on machine learning according to an embodiment of the present invention. At this time, the following flow chart will be described using the same reference numerals in connection with the configuration of Figs.

Referring to FIG. 9, the abrupt situation monitoring control system 10 through data fusion according to an embodiment of the present invention obtains image data from the image camera 110 and generates a tracking object list by applying a Gaussian mixture model. (S202 to S208).

In addition, the abrupt situation monitoring control system 10 through the data fusion according to an embodiment of the present invention obtains the radar data from the radar sensor 120, the radar-based through noise / diffuse reflection signal processing and 2D / 3D image processing Extract the object feature (S210 to S216).

In addition, the abrupt situation monitoring control system 10 through data fusion according to an embodiment of the present invention is based on the image of the object based on the analysis result of the radar data and the object list based on the analysis result of the image data The data and the radar data are fused and the image object feature is extracted (S218 and S220).

In addition, the sudden situation monitoring control system 10 through data fusion according to an embodiment of the present invention detects the unexpected situation by learning and classifying the fused features based on the machine learning (S222 to S226).

In addition, the sudden situation monitoring control system 10 through data fusion according to an embodiment of the present invention receives the tracking / road information feedback, and generates a tracking object list through Kalman tracking correction / prediction, machine learning and accident situation The accuracy and reliability of the recognition are improved (S228 and S230).

As such, the incident monitoring and control system through data fusion according to an embodiment of the present invention improves the resolution of data and effectively recognizes and monitors the sudden situation by signal processing and fusion of image data and radar data. Provide an environment.

In addition, the present invention provides an environment capable of effectively detecting an unexpected situation and actively responding to an unexpected situation by detecting an abnormal moving vehicle in a reverse driving direction or a left and right direction with respect to the normal moving direction of the lane using the Doppler effect.

In addition, the present invention by applying a zero padding (Zero Padding) algorithm, improves the moving speed resolution of the moving object, and provides an environment that can overcome the limitations of the resolution improvement due to the time constraints required to update the information.

In addition, the present invention by applying a side lobe attenuation algorithm to minimize the noise ratio of the transmitted signal to the received signal, and provides an environment that can effectively extract the object information and object features.

In addition, the present invention generates the fusion data of the image data and the radar data by applying the Over Sampling algorithm, thereby minimizing the limitations due to the limitations of hardware and time that occur during the fusion of heterogeneous data, It provides an environment to improve the resolution of the data.

In addition, the present invention by collecting the traffic information and the incident of the shaded area through the repeater function, to remove the blind spot of the traffic information, to provide an environment that can improve the traffic safety and quality of the traffic information.

In addition, the present invention by collecting the information of the detection device and the sudden recognition control device disposed in the shadow area through the data collection function of the information collecting terminal, to effectively detect the abnormality of the detection device or the sudden recognition control device, and the detection device and Provides an environment that can easily maintain a sudden recognition control device.

The embodiments of the present invention described above are not only implemented through the apparatus and the method, but may be implemented through a program for realizing a function corresponding to the configuration of the embodiments of the present invention or a recording medium on which the program is recorded. Such a recording medium can be executed not only in the server but also in the user terminal.

Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements of those skilled in the art using the basic concepts of the present invention defined in the following claims are also provided. It belongs to the scope of rights.

Claims (10)

A detection device including an imaging camera, a radar sensor, and a tracking camera, and
Analyze the data collected from the detection device, generate a tracking object list from the image data of the image camera, signal the radar data of the radar sensor to extract the radar-based object features, the tracking object list and the Convergence data based on object features, and recognize and categorize fused features based on machine learning, and recognize accidental situations on the road. Includes a tracking control device
The sudden recognition control device,
The Doppler effect is used to detect an abnormal moving vehicle in the normal moving direction of the lane, analyze the wavelength of the radar sensor transmission signal and the received signal, detect the moving speed of the moving object, and compare it with the normal moving direction of the specific lane. It determines whether the driving in the reverse direction and the left and right directions, and if the frequency of the received signal is greater than the frequency of the transmission signal, the reverse driving to determine the moving object reflecting the received signal compared to the normal moving direction of the lane as a reverse driving vehicle Sudden situation monitoring control system through data fusion comprising a determination unit. delete In claim 1,
The sudden recognition control device,
Zero padding to signal-process the radar data collected periodically from the radar sensor to remove noise or diffuse reflection, and to process the signal on the time axis to zero value for a time corresponding to an integer multiple of a certain period after the input signal. Noise Padding to improve the moving speed resolution of moving objects by applying Zero Padding) algorithm
Sudden situation monitoring control system through a data fusion further comprising. In claim 3,
The sudden recognition control device,
Recognizing object information by analyzing the received signal reflected from the moving object to the transmission signal of the radar sensor, and controls to increase the output of the transmission signal of the radar sensor to a predetermined value or more, Feature extraction unit that extracts object information and object features by applying side lobe attenuation algorithm to minimize noise ratio
Sudden situation monitoring control system through a data fusion further comprising. In claim 4,
The sudden recognition control device,
The analysis result of the image data and the analysis result of the radar data are fused, and each data is sampled at a time period and converted from an analog signal to a digital signal and fused, and an fusion data is generated by applying an over-sampling algorithm. And a data fusion unit for fusing object features of the moving object to determine the position, distance, size, or shape of the moving object from the fusion data through spatial matching of the image data and the radar data.
Sudden situation monitoring control system through a data fusion further comprising. In claim 1,
The sudden recognition control device,
Transmitting / receiving unit that collects information by wired or wireless connection with the sudden recognition control device adjacent to it and transmits the collected information to another adjacent sudden recognition control device or to the traffic information center
Incident monitoring control system through data fusion comprising a. In claim 6,
The transceiver unit,
Repeater unit for transmitting and receiving time information is allocated, and synchronized with the allocated transmission and reception time information to transmit and receive a signal, and transmits the information to a peripheral sudden recognition control device only at a designated time according to the transmission and reception time information.
Incident monitoring control system through data fusion comprising a. In claim 6,
An information collection terminal mounted on a mobile vehicle and connected with a plurality of sudden recognition control devices to collect information from the plurality of sudden recognition control devices.
More,
The information collecting terminal,
A meter signal transmitting unit for transmitting a meter signal to a plurality of abrupt recognition controllers;
A carrier detection unit for detecting a carrier wave transmitted from at least one abrupt recognition control device which has received the meter reading signal among the plurality of abrupt recognition control devices, and
Information collecting unit for determining whether there is an abnormality of the sudden recognition control device that transmitted the carrier or whether there is an abnormality of the detection device connected to the sudden recognition control device that transmitted the carrier wave and transmits it to the traffic information center.
Incident monitoring control system through data fusion comprising a. In claim 8,
The information collecting terminal,
Meter signal retransmission control unit controlling to retransmit the meter signal after a predetermined time delay when a plurality of carriers are detected in the same frequency band
Sudden situation monitoring control system through a data fusion further comprising. In claim 6,
The sudden recognition control device,
The road surface discrimination algorithm uses the polarization phenomenon of light or machine learning to determine the road surface condition. The road surface information of the road is determined by a wireless communication device. Accident situation monitoring and control system through data fusion that transmits or transmits to traffic information center.

Images