KR20210030819A - Road falling object recognition and management system through sensor fusion and artificial intelligence - Google Patents

Abstract

The present disclosure relates to a system for recognizing and managing an object falling on a road through sensor fusion and artificial intelligence. The system comprises: a falling object detection device installed on a moving object moving on a road, obtaining image data through at least one of LIDAR or stero camera when the moving object moves along the road, recognizing a falling object on the road from the image data through machine learning, and transmitting information on the falling object to a management server; and a management server checking the type and location of the falling object based on information on the falling object, and transmitting a falling object collection message to a falling object collection device based on the location. According to the present disclosure, the system may automatically detect and quickly collect the falling object, which is difficult to confirm with the naked eye, on the road.

Description

Road falling object recognition and management system through sensor fusion and artificial intelligence {ROAD FALLING OBJECT RECOGNITION AND MANAGEMENT SYSTEM THROUGH SENSOR FUSION AND ARTIFICIAL INTELLIGENCE}

The present disclosure relates to a road falling object recognition and management system for recognizing and collecting fallen objects on a road.

Freight loaded on a vehicle traveling on the road falls on the road, or falling objects such as rocks from a hill near the road fall onto the road frequently. Previously, employees of the Korea Expressway Corporation went on patrols and cleaned up falling objects on the road. However, this method is a patrol using human eyes, and the probability of finding fallen objects is lowered when visibility is not secured, such as at night or in rainy weather. In addition, when observed with the naked eye, there may be cases in which falling objects cannot be found due to carelessness.

Accordingly, it is necessary to introduce a method for effectively detecting a falling object on a road.

Against this background, the object of the present disclosure is to provide a road falling object recognition and management system capable of automatically recognizing falling objects on the road using sensor fusion and artificial intelligence, and collecting the falling objects based on the recognized information. will be.

In order to achieve the above object, in one aspect, the present disclosure is a system for recognizing and managing falling objects on a road through sensor fusion and artificial intelligence, when installed on a moving object moving on the road, and the moving object moves along the road A falling object detection device that acquires image data on the road through at least one of a lidar or a stereo camera, recognizes a falling object on the road from the image data through machine learning, and transmits information on the fallen object to the management server, and the falling object. Provides a road falling object recognition and management system including a management server that checks the type and location of a falling object based on information and transmits a falling object collection message to a falling object collection device based on the location.

According to the present disclosure, by automatically recognizing falling objects on the road using sensor fusion and artificial intelligence, and collecting the falling objects based on the recognized information, the probability of finding falling objects on the road is increased, thereby minimizing damage caused by falling objects and efficiently. It has the effect of collecting falling objects.

1 is a diagram for describing a falling object on a road according to the present disclosure.
2 is a diagram for describing a system for recognizing and managing falling objects on a road according to an exemplary embodiment of the present disclosure.
3 is a diagram for describing a location of a falling object according to an embodiment of the present disclosure.
FIG. 4 is a diagram for describing transmission of a location and an image of a falling object according to an embodiment of the present disclosure.
5 is a diagram for describing recognition of a falling object according to the present disclosure.
6 is a diagram for describing detection of a falling object according to movement of a moving object according to an exemplary embodiment of the present disclosure.

Hereinafter, some embodiments of the present disclosure will be described in detail through exemplary drawings. In adding reference numerals to elements of each drawing, the same elements may have the same numerals as possible even if they are indicated on different drawings. In addition, in describing the present disclosure, if it is determined that a detailed description of a related known configuration or function may obscure the subject matter of the present disclosure, the detailed description may be omitted.

In addition, in describing the constituent elements of the present disclosure, terms such as first, second, A, B, (a), and (b) may be used. These terms are for distinguishing the constituent element from other constituent elements, and the nature, order, or order of the constituent element is not limited by the term. When a component is described as being "connected", "coupled" or "connected" to another component, the component may be directly connected or connected to that other component, but another component between each component It will be understood that elements may be “connected”, “coupled” or “connected”.

On the road on which the vehicle moves, a load dropped from the vehicle or a rock falling onto the road in the surrounding environment may exist. Referring to Table 1, the number of fallen objects collected on such roads represents a considerable number, and if they are not collected quickly, serious damage may occur.

1 is a diagram for describing a falling object on a road according to the present disclosure.

If the rapid collection of falling objects on the road is not performed, serious damage may be caused by falling objects on the road. Referring to FIG. 1, accidents caused by falling objects are constantly occurring year after year. It is imperative to prevent accidents caused by falling objects on the road and minimize damage, and for this purpose, safety management to prevent secondary accidents, as well as rapid detection of falling objects, and rapid removal of fallen objects are necessary. It needs to be prevented.

Conventionally, in order to remove the falling objects on the road, it was checked whether there were any falling objects on the road while moving along the road with the naked eye. However, in the case of road patrol by the naked eye, the probability of finding a falling object is lowered when visibility is not secured, such as at night or in rain.

Accordingly, in the present disclosure, a system for finding a falling object by fusing data from a Lidar sensor and a stereo camera is proposed.

2 is a diagram for describing a system for recognizing and managing falling objects on a road according to an exemplary embodiment of the present disclosure. 3 is a diagram for describing a location of a falling object according to an embodiment of the present disclosure. FIG. 4 is a diagram for describing transmission of a location and an image of a falling object according to an embodiment of the present disclosure. 5 is a diagram for describing recognition of a falling object according to the present disclosure. 6 is a diagram for describing detection of a falling object according to movement of a moving object according to an exemplary embodiment of the present disclosure.

According to an embodiment, the road falling object recognition and management system is installed on a moving object moving on the road, and when the moving object moves along the road, image data on the road is acquired through at least one of a lidar or a stereo camera, and the machine A falling object detection device that recognizes falling objects on the road from image data through running and transmits information about the falling objects to the management server, and a falling object collection device based on the location and the type and position of falling objects based on information about the falling objects. It may include a management server that transmits the falling object collection message.

The falling object detection device may be installed on a moving object moving on a road. For example, the moving object may be a falling object inspection vehicle. However, this is an example, and the moving object may be a means of transportation other than a vehicle or a means of flying such as a drone.

When the moving object moves along the road, the falling object detection apparatus may acquire image data of the road through at least one of a lidar or a stereo camera. However, this is an example and is not limited thereto. The falling object detection apparatus may include various sensors regardless of the type or name as long as it can acquire image data on the road. The falling object detection apparatus includes a gyro sensor, and may correct the positions of the camera and the lidar based on an output value of the gyro sensor.

According to an example, the apparatus for detecting a falling object may acquire image data on a falling object by fusing lidar data and stereo camera data. The falling object detection device may detect whether there is a falling object through the fused image data.

Referring to FIG. 4, an example in which a lidar and a camera are configured as a front channel is illustrated. However, according to an example, the lidar and the camera may be configured with multiple channels (360 degrees).

According to an example, the apparatus for detecting a falling object may collect image information through an infrared camera at night.

By detecting falling objects using lidar data and stereo camera data, it is possible to increase the detection probability of falling objects even in weather such as night, rain, fog, etc., where it is difficult to identify the falling objects with the naked eye.

The falling object detection device may recognize a falling object on a road from image data acquired through machine learning. Recognition of falling objects may be recognition of the size and type of falling objects. If falling objects can be recognized from image data, machine learning can be applied to deep learning and other methods using various artificial intelligences. In this case, various types of falling objects may be learned in order to recognize the falling objects.

The falling object detection device may store information on the falling object in a storage device such as a black box and transmit it to a management server on a traffic management system. According to an example, GPS information, a size of a falling object and an object type, camera data, and lidar data may be transmitted to the management server.

Referring to FIG. 3, as an example of location information of a falling object, GPS information is shown. The falling object detection device may receive GPS information of a moving object in which the falling object detection device is installed. According to an example, the apparatus for detecting a falling object may transmit GPS information of a moving object as GPS information of the falling object. Alternatively, according to another example, the apparatus for detecting a falling object may obtain GPS information of a falling object by mapping GPS coordinates according to GPS information of a moving object and coordinates of a falling object on image data, and transmit the corresponding information.

The management server may check the type and location of the falling object based on the received information on the falling object. The management server can automatically store the data of falling objects in the storage device. The management server may transmit a falling object collection message to the falling object collecting device based on the position of the falling object. For example, the message can be transmitted to a falling object collection vehicle that can move to the location of the falling object most quickly.

According to an example, the management server may investigate the cause of the falling object and find a solution through the CCTV installed around the position of the falling object. Referring to Table 2, the causes of falling objects are classified.

If, as a case where the load of the vehicle has fallen, and the vehicle causing the falling object is identified, the management server may issue penalties and fines to the user of the vehicle. For example, in the case of falling objects on the road, the regulation and punishment laws impose imprisonment of 10 years or less and a fine of 15 million won or less pursuant to Article 11 of the Highway National Highway Act, and imprisonment of 1 year to 10 years pursuant to Article 12 of the Highway National Highway Act. do.

5 is a diagram illustrating an experiment in a building to recognize falling objects from image data. You can see that the objects placed on the floor are recognized. In the present disclosure, the falling object is recognized by actually targeting the road surface.

Referring to FIG. 6, an example of recognizing an object while a moving object moves a predetermined path is illustrated. A falling object investigation vehicle equipped with a falling object detection device according to the present disclosure only needs to slowly travel on a road to be investigated. Then, it automatically recognizes the falling object and transmits the related information to the management server of the Korea Transportation Authority. The management server of the Transportation Management Authority automatically contacts the processing team adjacent to the falling objects, and the processing team collects the falling objects accordingly.

Furthermore, the management server investigates the cause of the falling object. The investigation method is done through recorded CCTV, and if the cause of the falling object is found through reverse investigation, measures such as penalties and fines are taken.

According to this, the falling objects on the road are automatically recognized using sensor fusion and artificial intelligence, and the falling objects are collected based on the recognized information, thereby increasing the probability of finding falling objects on the road, minimizing damage caused by falling objects, and efficiently preventing falling objects. There is an effect that can be collected.

The above-described present disclosure can be implemented as a computer-readable code on a medium in which a program is recorded. The computer-readable medium includes all types of recording devices that store data that can be read by a computer system. Examples of computer-readable media include hard disk drives (HDDs), solid state disks (SSDs), silicon disk drives (SDDs), ROMs, RAM, CD-ROMs, magnetic tapes, floppy disks, optical data storage devices, etc. There is also a carrier wave (for example, transmission over the Internet) also includes the implementation of the form.

The description above and the accompanying drawings are merely illustrative of the technical idea of the present disclosure, and those of ordinary skill in the technical field to which the present disclosure pertains, combinations of configurations without departing from the essential characteristics of the present disclosure Various modifications and variations, such as separation, substitution, and alteration, will be possible. Accordingly, the embodiments disclosed in the present disclosure are not intended to limit the technical idea of the present disclosure, but to describe, and the scope of the technical idea of the present disclosure is not limited by these embodiments. That is, as long as it is within the scope of the object of the present disclosure, one or more of the components may be selectively combined and operated. The scope of protection of the present disclosure should be interpreted by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present disclosure.

Claims (1)

In a system that recognizes and manages falling objects through sensor fusion and artificial intelligence,
Installed on a moving object moving on a road, and when the moving object moves along a road, image data about the road is acquired through at least one of a lidar or a stereo camera, and falling objects on the road are recognized from the image data through machine learning And a falling object detection device that transmits information on the falling object to a management server; And
A management server that checks the type and position of the falling object based on the information on the falling object, and transmits a falling object collection message to the falling object collection device based on the position;
Road falling object recognition and management system comprising a.

Images