KR102290548B1 - Device and method for preventing traffic accident through judging road condition based on deep learning model - Google Patents

Abstract

The present invention relates to a device and a method for deep learning-based traffic accident prevention. The device includes: a communication unit collecting public data and image data; an accident risk determination unit generating accident risk information with respect to each weather type preset based on the public data; a risk area detection unit extracting a road region in the image data, analyzing road conditions in the extracted road region using a first deep learning model, and generating a risk area list based on the analyzed road conditions; and a processor determining the real-time weather in a user area based on location information of a user device. In a case where the user is within a preset range from the risk area in the risk area list and the accident risk information regarding the weather type corresponding to the determined real-time weather, the processor transmits a risk area detection signal to the user device through the communication unit.

Description

DEVICE AND METHOD FOR PREVENTING TRAFFIC ACCIDENT THROUGH JUDGING ROAD CONDITION BASED ON DEEP LEARNING MODEL

The present invention relates to an apparatus and method for preventing traffic accidents through road condition determination based on a deep learning model.

Every year, a lot of traffic accident deaths or injuries occur due to traffic accidents, which incurs a lot of social cost. Traffic accidents are largely caused by complex problems such as road environment, weather, drivers, and vehicles. In particular, it is necessary to install additional safety facilities or improve the road in the section of the road with a high risk of traffic accidents.

In the prior art, there have been many methods for preventing traffic accidents. For example, based on the traffic accident data on the site where the traffic accident occurred, provided from the user terminal, traffic accident data that can effectively prevent traffic accidents is generated, or various information about the driver's condition and the vehicle's operating status while driving the vehicle. Methods for generating data to prevent traffic accidents by collecting and analyzing information have been proposed.

However, the above patents do not disclose a method for detecting a dangerous area and predicting the possibility of an accident by using traffic accident data, weather observation data, and user information for drivers, pedestrians, and safety-related ministries in combination.

In particular, there is no way to prevent traffic accidents in advance by judging the road conditions in advance rather than data after the accident and determining that the area is a high risk area for traffic accidents. Also, there is no way to prevent traffic accidents in connection with weather information that can cause traffic accidents.

Republic of Korea Patent Publication No. 10-2018-0073852 (2018.07.03)

The present invention for solving the above-described problems is to provide an apparatus and method for preventing traffic accidents through road condition determination based on a deep learning model.

However, the problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

The apparatus for preventing traffic accidents through road condition determination based on a deep learning model of the present invention for solving the above-described problems includes a communication unit that collects public data and image data, and an accident for each weather type preset based on public data An accident risk determination unit that generates risk information, extracts a road area included in the image data, analyzes the road condition in the extracted road area using a first deep learning model, and based on the analyzed road condition Determines real-time weather of the area where the user is located based on the location information of the hazardous area detection unit and the user device that generates the hazardous area list, and accident risk information on the type of weather corresponding to the determined real-time weather and the risk area When the user is located within a preset range from the danger area included in the list, including a processor for transmitting the danger area detection information to the user device through the communication unit, the public data, traffic accident information for a preset period and It includes climate statistical information, and the road condition includes at least one of normal, crack, rutting and porthole, and the dangerous area detection unit, wherein the road condition corresponds to at least one of the crack, the rutting and the porthole. In this case, it is determined that the road is a dangerous area, and the dangerous area list including the road determined as the dangerous area is generated.

In addition, the weather type includes a first weather type indicating sunny, a second weather type indicating cloudy or foggy, and a third weather type indicating rain or snow, and the accident risk determination unit includes the traffic accident information and climate statistics Based on the information, the number of daily accidents, the number of daily deaths, and the number of daily injuries are calculated for each of the first to third weather types.

In addition, the processor calculates accident probability information for each road condition for the first to third weather types, and transmits the accident probability information based on the location information of the user device and real-time weather.

In addition, the dangerous area detecting unit further determines a risk factor of the road, the risk factor including black ice in the road, wherein the dangerous area detecting unit determines the black ice in the road by using an autoencoder, The autoencoder uses a second deep learning model to give weights to each of the R, G, and B values of the image data, and then converts the weighted R, G, and B values through a one-dimensional vector representation (1D Vector Representation). The black ice of the road is determined by converting it into an IR value, and the road including the black ice is further included in the dangerous area list.

In addition, the processor is further configured to be configured to be further configured to be further configured to: when the real-time weather of the road including the black ice in the dangerous area list corresponds to a third weather type among the weather types and the temperature is less than the first reference value, management information or control information about the road to the manager device.

In addition, when the user device is included in the vehicle, the processor includes guide information that allows a driver of the vehicle to drive the brake device of the vehicle, and the guide information is calculated using Equation (1).

In addition, the communication unit receives the movement path information of the user from the user device, and the processor determines, based on the movement path information and the dangerous area list, a dangerous area included in the movement path, and the determined New movement route information is generated based on the dangerous area and transmitted to the user device.

In addition, the processor, when a distance between a plurality of dangerous regions included in the dangerous region list is less than or equal to a preset distance, and the number of daily accidents according to a weather type corresponding to the real-time weather is greater than or equal to a preset number, the plurality of dangerous regions Further, accident possibility information about an area is generated and transmitted to the user device.

A computer program for performing a method for preventing traffic accidents based on deep learning according to another aspect of the present invention for solving the above-described problems may be stored in a storage medium.

In addition to this, another method for implementing the present invention, another system, and a computer-readable recording medium for recording a computer program for executing the method may be further provided.

According to the present invention as described above, accident risk information according to weather types is generated based on public data. Through this, by providing the user with accident risk information according to the type of weather in the area where the user is located, determined based on the user terminal, it is particularly effective in preventing traffic accidents.

In addition, by identifying the state of the road based on the deep learning model and generating a risk area list based on the location information of the road, if the user is located within the danger area included in the danger area list, the danger area detection information is provided to provide traffic accident have a preventive effect.

Effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1 is a schematic configuration diagram of a deep learning-based traffic accident prevention system according to the present invention.
2 is a schematic configuration diagram of a deep learning-based traffic accident prevention system according to the present invention.
3 is a diagram for identifying black ice according to an embodiment of the present invention.
4 is a diagram illustrating a source code for determining black ice according to an embodiment of the present invention.
5 is a schematic flowchart of a method for preventing traffic accidents based on deep learning according to the present invention.

Advantages and features of the present invention and methods of achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only these embodiments allow the disclosure of the present invention to be complete, and those of ordinary skill in the art to which the present invention pertains. It is provided to fully understand the scope of the present invention to those skilled in the art, and the present invention is only defined by the scope of the claims.

The terminology used herein is for the purpose of describing the embodiments and is not intended to limit the present invention. As used herein, the singular also includes the plural unless specifically stated otherwise in the phrase. As used herein, “comprises” and/or “comprising” does not exclude the presence or addition of one or more other components in addition to the stated components. Like reference numerals refer to like elements throughout, and "and/or" includes each and every combination of one or more of the recited elements. Although "first", "second", etc. are used to describe various elements, these elements are not limited by these terms, of course. These terms are only used to distinguish one component from another. Accordingly, it goes without saying that the first component mentioned below may be the second component within the spirit of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used herein will have the meaning commonly understood by those of ordinary skill in the art to which this invention belongs. In addition, terms defined in a commonly used dictionary are not to be interpreted ideally or excessively unless specifically defined explicitly.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Before the description, the meaning of the terms used herein will be briefly described. However, it should be noted that the description of terms is for the purpose of helping the understanding of the present specification, and is not used in the meaning of limiting the technical idea of the present invention unless explicitly described as limiting the present invention.

Also, as used herein, the term “unit” refers to a hardware element such as software, FPGA, or ASIC, and “unit” performs certain roles. However, "part" is not meant to be limited to software or hardware. A “unit” may be configured to reside on an addressable storage medium and may be configured to refresh one or more processors. Thus, by way of example, “part” refers to elements such as software elements, object-oriented software elements, class elements, and task elements, and processes, functions, properties, procedures, subroutines, and programs. It includes segments of code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, and variables. The functionality provided within elements and “parts” may be combined into a smaller number of elements and “parts” or further separated into additional elements and “parts”.

In addition, in this specification, all “units” may be controlled by at least one processor, and at least one processor may perform operations performed by the “units” of the present disclosure.

Embodiments of the present specification may be described in terms of a function or a block performing a function. Blocks, which may be referred to as 'parts' or 'modules', etc. in the present disclosure include logic gates, integrated circuits, microprocessors, microcontrollers, memories, passive electronic components, active electronic components, optical components, hardwired circuits, and the like. It may be physically implemented by analog or digital circuitry, such as, and optionally driven by firmware and software.

Embodiments of the present specification may be implemented using at least one software program running on at least one hardware device and may perform a network management function to control an element.

Unless otherwise defined, all terms (including technical and scientific terms) used herein will have the meaning commonly understood by those of ordinary skill in the art to which this invention belongs. In addition, terms defined in a commonly used dictionary are not to be interpreted ideally or excessively unless specifically defined explicitly.

In the present invention, 'public data' includes climate statistical information and traffic accident information that can be obtained (or received) through services provided by government related ministries. For example, public data includes traffic accident information obtainable through the public data portal (data.go.kr) and traffic accident information open system (taas.koroad.or.kr), and climate statistical information obtainable through the Meteorological Administration website, etc. , including road images and current traffic situation information.

On the other hand, 'image data' is data related to a moving picture or image that can be acquired through a user terminal or a camera module attached to a public facility, and more preferably, refers to image data that can identify a specific road condition.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a schematic configuration diagram of a traffic accident prevention system based on a deep learning model according to an embodiment of the present invention.

First, referring to FIG. 1 , the deep learning model-based traffic accident prevention system 10 is a deep learning model-based traffic accident prevention device 100 , a user device 200 , a public facility device 300 , and a manager device 400 . includes

Traffic accident prevention apparatus 100 of the present invention is a server, computer, UMPC (Ultra Mobile PC), workstation, net-book (net-book), PDA (Personal Digital Assistants), portable (portable) computer, web tablet (web tablet) ), a wireless phone, a mobile phone, a smart phone, and an electronic device such as a portable multimedia player (PMP). Also, the traffic accident prevention apparatus 100 may be an electronic device included in a server (not shown) operating a specific site or platform.

Meanwhile, the user device 200 is a personal digital assistant (PDA), a portable computer, a web tablet, a wireless phone, a mobile phone, and a smart phone carried by a pedestrian or a driver. It may be one of electronic devices such as a smart phone or a portable multimedia player (PMP). The user device 200 shoots a road image in real time, and the accident occurrence prediction device 100 is a user

A road condition may be determined by receiving image data from the device 200 and analyzing a road image included in the personal data.

In addition, the user device 200 may not only provide image data to the apparatus 100 for preventing traffic accidents, but also receive danger area list information generated based on the image data. More specifically, when the user of the user device 200 is located within a preset range from the danger area included in the danger area list, the danger area detection information is provided from the traffic accident prevention apparatus 100 . That is, the traffic accident prevention apparatus 100 provides the danger area detection information to the user device 200 while receiving image data from the user device 200 registered in the device.

In addition, the user device 200 is a terminal having a photographing function and a data communication function, and may be an electronic device that can be built in or attached to a vehicle. In this case, the user device 200 may be composed of a plurality of electronic devices that can be fixed or attached to a vehicle, for example, a black box that can acquire a road image in real time and a traffic that can receive traffic information in real time. A navigation device, which is a guide device, may correspond to this. In this case, the road image may be captured through a vehicle photographing device (eg, a black box) and transmitted to the traffic accident prevention device 100 through a vehicle communication device.

Meanwhile, the public facility device 300 may be a smart power pole capable of generating image data by photographing a road and simultaneously performing data communication. Alternatively, as a terminal having a photographing function and a data communication function, it may be an electronic device that can be built-in or attached to a public facility, such as a power pole.

Meanwhile, the public facility device 300 may also receive accident risk information and danger area detection information for preventing traffic accidents from the traffic accident prevention device 100 like the user device 200 described above. For example, after receiving accident risk information according to a weather type corresponding to real-time weather, the accident risk information is provided through an audio output module (not shown) or an image output module (not shown) included in the public facility device 300 . can be printed out. Through this, it may be possible to provide accident risk information to citizens close to the public facility device 300 . In addition, based on the image data provided from the public facility device 300, the surrounding road condition of the public facility device 300 is analyzed, and when it is detected as a dangerous area based on the road condition, the danger area detection information is used as an accident prevention device After being provided from 100 , the risk of an accident may be prevented by outputting the dangerous area detection information whenever a person, a vehicle, or the like approaches the public facility device 300 .

On the other hand, when the user device 200 and the public facility device 300 are electronic devices or terminals having a photographing function and a data communication function, respectively, a photographing device having a sensor (eg, a thermal image sensor and an RGB sensor) ( For example, a road image may be photographed through a camera) and image data may be transmitted to the accident prevention apparatus 100 through a communication module having a wireless communication function.

On the other hand, the manager device 400 means a device used by an administrator who controls and manages each danger area, and includes personal digital assistants (PDA), portable computers, web tablets, and wireless phones. phone), a mobile phone, a smart phone, or an electronic device such as a portable multimedia player (PMP). Meanwhile, the traffic accident prevention apparatus 100 may receive image data from the manager apparatus 400 .

2 is a schematic configuration diagram of an apparatus for preventing traffic accidents based on deep learning according to an embodiment of the present invention.

Referring to FIG. 2 , the apparatus 100 for preventing a traffic accident includes a communication unit 110 , an accident risk determination unit 120 , a danger area detection unit 130 , a memory 140 , and a control unit 150 .

According to an embodiment of the present invention, the communication unit 120 collects public data and image data. Specifically, the communication unit 120 receives public data from a related department site. And the image data is received from the user device 200 , the public facility device 300 , or the manager device 400 . Meanwhile, the communication unit 120 transmits accident risk information and danger area detection information corresponding to each user to each user terminal by the control unit 150 .

In this case, the communication unit 110 may communicate with various types of external devices according to various types of communication methods. The communication unit 110 may include at least one of a Wi-Fi chip, a Bluetooth chip, a wireless communication chip, and an NFC chip.

The Wi-Fi chip and the Bluetooth chip may perform communication using a WiFi method and a Bluetooth method, respectively. In the case of using a Wi-Fi chip or a Bluetooth chip, various types of connection information, such as an SSID and a session key, are first transmitted and received, and various types of information can be transmitted/received after communication connection using the same. The wireless communication chip is IEEE, Zigbee, 3G (3rd

Generation) refers to a chip that performs communication according to various communication standards such as 3rd Generation PartnershIP Project (GPP), Long Term Evolution (LTE), and the like. The NFC chip refers to a chip operating in an NFC (Near Field Communication) method using a 13.56 MHz band among various RF-ID frequency bands such as 135 kHz, 13.56 MHz, 433 MHz, 860 to 960 MHz, and 2.45 GHz.

The accident risk determination unit 120 generates accident risk information for each preset weather type based on the public data acquired through the communication unit 110 .

Specifically, when the public data includes traffic accident information and climate statistical information as described above, the accident risk determination unit 120 generates traffic accident information according to each preset weather type. That is, assuming that traffic accident information is, for example, the number of daily accidents, the number of fatalities, and the number of injured Generate accident risk information.

On the other hand, the number of injured can be further subdivided and classified according to the degree of injury. For example, after classifying serious injuries, minor injuries, and reported injuries, it is possible to identify the weather type on the day each injury occurred, and then calculate the number of serious injuries, the number of minor injuries, and the number of reported injuries according to the weather type.

Through this, the accident prevention apparatus 100 calculates the correlation of the occurrence of traffic accidents according to the weather, determines which weather type the real-time weather corresponds to among preset weather types, and provides accident risk information according to the corresponding weather type to the user. It is transmitted to the terminal 200 .

In this case, as an embodiment of the present invention, the weather type may include a first weather type indicating sunny weather, a second weather type indicating cloudy or foggy, and a third weather type indicating rain or snow. The accident risk determination unit 120 determines the type of weather at the time of the occurrence of a traffic accident based on the temperature, humidity, and wind speed, and calculates the number of deaths and injuries according to the occurrence of each traffic accident in association with the corresponding weather type.

Meanwhile, according to an embodiment of the present invention, the accident risk information means the number of daily accidents, the number of daily deaths, and the number of daily injuries for each of the first to third weather types calculated based on traffic accident information and climate statistical information. For example, if it is assumed that the first weather type is sunny, the accident risk determination unit 120 calculates information on traffic accidents occurring in a preset period compared to the number of days in the preset period. In other words, it calculates the number of daily traffic accidents, the number of daily fatalities, and the number of daily injuries that occur in clear weather.

On the other hand, the danger area detection unit 130 extracts a road area included in the image data, analyzes the road condition in the extracted road area using the first deep learning model, and a risk based on the analyzed road condition Create a local list.

Specifically, the dangerous area detection unit 130 extracts a road area from each image data. To this end, a process of removing a background area excluding a road area of each image data may be performed. Then, the extracted road area is input to the first deep learning model, and the road condition of the corresponding road area is analyzed.

At this time, the deep neural network (DNN) constituting the first deep learning model may include a system or network that builds one or more layers in one or more computers and performs a judgment based on a plurality of data. can A deep neural network may be implemented as a set of layers including a convolutional pooling layer, a locally-connected layer, and a fully connected layer.

The convolutional pooling layer or local connection layer may be configured to extract features in an image. In addition, the fully connected layer may determine a correlation between features of an image.

As another example, the overall structure of the deep neural network of the present invention may be formed in a form in which a local access layer is connected to a convolutional pooling layer, and a fully connected layer is formed in the local access layer. The deep neural network may include various judgment criteria (ie, parameters), and may add new judgment criteria (ie, parameters) through input image analysis.

A deep neural network according to embodiments of the present invention is a structure called a convolutional neural network suitable for image analysis, and a feature extraction layer that learns by itself a feature with the greatest discriminative power from given image data. A prediction layer that learns a predictive model to obtain the highest predictive performance based on and extracted features may be configured in an integrated structure.

The feature extraction layer spatially integrates a convolution layer that creates a feature map by applying a plurality of filters to each region of the image and a feature map that is invariant to changes in position or rotation. It can be formed in a structure that alternately repeats the pooling layer for extracting the . Through this, various level features can be extracted from low-level features such as points, lines, and planes to complex and meaningful high-level features.

The convolutional layer obtains a feature map by taking a nonlinear activation function on the dot product of the filter and the local receptive field for each patch of the input image. In comparison, CNNs are characterized by using filters with sparse connectivity and shared weights. Such a connection structure reduces the number of parameters to be learned, makes learning through the backpropagation algorithm efficient, and consequently improves prediction performance.

The integration layer (Pooling Layer or Sub-sampling Layer) generates a new feature map by using local information of the feature map obtained from the previous convolutional layer. In general, the newly created feature map by the integration layer is reduced to a smaller size than the original feature map. Representative integration methods include Max Pooling, which selects the maximum value of the corresponding region in the feature map, and the corresponding feature map in the feature map. There is an average pooling method that calculates the average value of a region. In general, the feature map of the integrated layer may be less affected by the position of an arbitrary structure or pattern existing in the input image than the feature map of the previous layer. That is, the integration layer can extract features more robust to regional changes such as noise or distortion in the input image or the previous feature map, and these features can play an important role in classification performance. Another role of the integration layer is to reflect the features of a wider area as you go up to the upper learning layer in the deep structure. It is possible to generate features that reflect the more abstract features of the entire image.

As such, the features finally extracted through iteration of the convolutional layer and the integration layer are fully connected to the classification model such as multi-layer perception (MLP) or support vector machine (SVM). -connected layer) and can be used for classification model training and prediction.

Also, according to an embodiment of the present invention, training data for machine learning may be generated based on the U-Net-dhSgement model. Here, the U-Net-dhSgement model is based on an end-to-end fully connected convolutional network (FCN), and contracts an expansive path and a symmetric path. It could be a model that created a U-shaped architecture with skip connections for each level by setting it to .

Meanwhile, in an embodiment of the present invention, the road condition analyzed through the first deep learning model includes at least one of normal, crack, rutting, and porthole. At this time, when the state of the corresponding road includes at least one of cracks, rutting, and portholes, the dangerous area detecting unit 130 includes the area in which the corresponding road is located in the list of dangerous areas. Through this, the accident prevention apparatus 100 may analyze the road condition without being limited to a regional range based on the collected image data, and may create and manage a hazardous area list according to the corresponding road condition.

On the other hand, although not clearly shown in the drawing, the danger area detection unit 130 may periodically update the danger area list. Specifically, with respect to a dangerous region included in the dangerous region list, the corresponding dangerous region may be managed through the manager device 400 and image data collection may be periodically requested. Through this, if the road corresponding to at least one of cracks, rutting, and potholes is restored to its normal condition through subsequent repairs, paving, etc., it is excluded from the list of dangerous areas.

Meanwhile, the memory 130 is a local storage medium that may include a database. The database may include public data, image data, and accumulated accident risk list information received by the communication unit 110 . Also, the memory 220 of the present invention may store instructions for operating the processor 150 .

The memory 130 should retain data even when the power supplied to the accident prevention device 100 is cut off, and may be provided as a writable non-volatile memory (Writable Rom) to reflect changes. That is, the memory 130 may be provided with either a flash memory, an EPROM, or an EEPROM. In the present invention, it is described that all instruction information is stored in one memory 130 for convenience of description, but the present invention is not limited thereto, and the accident prevention apparatus 100 may include a plurality of memories.

On the other hand, the processor 150 of the traffic accident prevention apparatus 100 determines the real-time weather of the area where the user is located based on the location information of the user device, and accident risk information on a weather type corresponding to the determined real-time weather and when the user is located within a preset range from the dangerous region included in the dangerous region list, the dangerous region detection information is transmitted to the user device through the communication unit.

Specifically, the processor 150 receives the user's location information from each user terminal registered in the traffic accident prevention apparatus 100 . Through this, real-time weather information of a region where each user is located can be searched. This may be searched by matching the climate statistics information of the above-described public data with the location information of the user. Then, accident risk information on a weather type corresponding to the real-time weather in the area where the user is located is extracted and transmitted to the user terminal through the communication unit 110 . For example, when the first weather type indicates that the weather in the area where the first user is located is clear through the first user terminal, accident risk information for the first weather type is provided. That is, the number of daily traffic accidents, the number of daily fatalities, and the number of daily injuries that occur in clear weather is calculated and transmitted to the first user terminal.

In addition, when the user is located within a preset range from the danger area included in the danger area list, the processor 150 transmits danger area detection information indicating that the user is close to or adjacent to the danger area. In this case, as described above, when the user terminal 200 includes a navigation device attached to the vehicle, it may be transmitted to display dangerous area detection information on a display unit (not shown) of the corresponding navigation device. Danger area detection information includes state information of the corresponding road (eg, at least one of cracks, rutting, and potholes), lane information of the corresponding road (eg, when the road includes a plurality of lanes, lane information) distance information to and from the user terminal, and image data including the corresponding road (ie, image data provided to analyze the road condition) may be transmitted together.

Through this, the user may be provided with accident risk information according to the real-time weather type and danger area detection information based on the user's real-time location, thereby preventing traffic accidents in advance.

Meanwhile, according to an embodiment of the present invention, the processor 150 calculates accident probability information for each road condition for the first to third weather types, and based on the location information of the user device and real-time weather , and transmits the accident possibility information.

The accident probability information refers to information obtained by calculating a correlation between each weather type and road condition and a traffic accident. For example, if the user is located within a preset range from the list included in the hazardous area list, the processor 150 correlates the number of traffic accidents, the number of fatalities, and the number of injuries based on the road conditions and real-time weather information of the area. It calculates and generates accident probability information.

Referring again to the above-described embodiment of the present invention, when the road condition is at least one of normal, crack, rutting, and pothole, the number of traffic accidents for each weather type of each road condition is calculated. For example, the number of traffic accidents occurring in the first weather type, the number of traffic accidents occurring in the second weather type, and the number of traffic accidents occurring in the third weather type in the cracked road condition are calculated. This is because, even in the case of the same road, the frequency of occurrence of traffic accidents according to weather may be different. Therefore, in order to provide users with more accurate traffic accident probability information, based on real-time weather information and real-time location information of the area where the user is located, information on the possibility of an accident that may occur in the hazardous area is calculated and provided for the type of weather. do.

The above-described example has been described based on only the first weather type and the number of traffic accidents. However, it will be apparent to those skilled in the art that various accident probability information may be generated based on the number of dead and injured in addition to the second and third weather types.

Meanwhile, the accident prevention apparatus 100 may further generate accident location information along with accident risk information, danger area detection information, and accident possibility information and transmit it to the user device.

Here, the accident location information may include location-related information on which a plurality of accidents occur, provided by the site of the relevant department, and information on the number of accidents and casualties that occurred during a preset period at the location of multiple accidents. Accordingly, if the real-time location of the current user is included in a preset range from the location of multiple accidents, the processor 150 calculates the predicted damage by dividing the number of casualties during the preset period by the number of accidents. In this case, even when the user's location is not included in the accident-prone location, the processor 150 may determine the predicted damage amount based on information on the number of accidents and the number of casualties occurring in the same road condition.

Meanwhile, according to an embodiment of the present invention, the danger area detection unit 130 may further determine the risk factor of the road. That is, in addition to cracks, rutting, and potholes determined according to road conditions, it is possible to further determine risk factors of the road that may occur depending on weather.

For example, black ice formed on the road corresponds to this. Black ice refers to a phenomenon in which snow or rain that has melted on the road freezes back into thin ice due to a sudden drop in temperature. It mainly occurs in places with low surface temperature such as shady roads or tunnels.

Therefore, according to an embodiment of the present invention, the danger area detection unit 130 determines the black ice in the road using an autoencoder, and the autoencoder uses a second deep learning model to determine the R, G, By assigning a weight to each B value, the weighted R, G, and B values are converted into IR values through 1D Vector Representation to determine the black ice of the road, and the black ice is included Roads may be further included in the list of dangerous areas.

On the other hand, according to an embodiment of the present invention, the traffic accident prevention apparatus 100 obtains information about the area where the traffic accident occurred due to black ice through public data, and through the public facility apparatus 300 installed in the area Real-time video data can be collected. Alternatively, when the preset weather type and temperature fall below preset values, the collection of image data for the corresponding region may be started.

Hereinafter, a method of identifying black ice according to an embodiment of the present invention will be described with reference to FIGS. 3 and 4 .

3 is a diagram for identifying black ice according to an embodiment of the present invention.

Autoencoder refers to an unsupervised learning neural network that copies input to output through training data. The autoencoder used to identify black ice of the present invention may include an uncompleted autoencoder, a stacked autoencoder, a denoising autoencoder, a sparse autoencoder, and a variational autoencoder.

Road image data included in public data or personal data may have R, G, and B values for each pixel. The processor 150 may convert the R, G, and B values of each pixel of the image data into IR values by using the autoencoder. Specifically, the processor may output the converted IR value by converting the R, G, and B values into an IR value through 1D Vector Representation.

In addition, the autoencoder of the present invention can convert each of the R, G, and B values by weighting the weighted R, G, and B values into an IR value through 1D Vector Representation. . In this case, the weights assigned to each of the R, G, and B values may be weights generated by learning the autoencoder. The autoencoder also learns a lot of image data through deep learning (or machine learning, machine learning), and whenever the autoencoder converts an image into an IR value through a one-dimensional vector representation, α, β and γ are is created In this case, α, β, and γ generated in one image are the same for all pixels.

Therefore, when analyzing a road image, the autoencoder actually converts α* R, β* G, and γ* B values into IR values by giving α, β, and γ weights to each of the input R, G, and B values. can In this case, the given α, β, and γ may be values generated by IR-converting an image most similar to a road image to be recognized currently among road images input as learning data.

According to an embodiment of the present invention, the source code for learning the autoencoder 320 may be a code written in the python language of the Google colab program, and the library for learning may include openCV and numpy.

4 is a diagram illustrating a source code for detecting black ice according to an embodiment of the present invention.

Referring to FIG. 4 , it is possible to convert an input road image into a luminance value (gray scale) through the openCV library using the gray = cv2.cvtColor(img, cv2.COLOR_RGB2GRAY) function.

Also, by using the red = gray.copy(), green = gray.copy(), blue = gray.copy() functions, the array of R, G, and B values of each pixel is assigned to a new address to copy the array. can

In addition, using the R=.642, G=.532, B=.44, sum=R+G+B, R=R/sum, G=G/sum, B=B/sum functions, R, G, Weights α, β, and γ to be assigned to the B value may be determined. Here, the values of 0.642, 0.532, and 0.44 may be values generated while IR-converting an image most similar to the input road image.

In addition, R of the road image input using the red=(R*red), green=(G*green), blue=(B*blue), result=cv2.merge([red, green, blue]) function, It is possible to convert G and B values into IR values.

Also, max=np.amax(result), result = ((255/max)*result).clip(0,255). By using the astype(np.uint8), return result function, the image is scaled to a ratio, and the value increased to the fully dynamic range is returned as the result value to determine whether there is black ice on the road.

Meanwhile, in an embodiment of the present invention, the processor 150 determines that the real-time weather of the road including the black ice included in the dangerous area list corresponds to a third weather type among the weather types, and the temperature is less than the first reference value. In this case, the management information or control information on the road may be transmitted to the manager device.

As described above, based on the real-time weather of the road where black ice occurs included in the list of dangerous areas, the weather type corresponds to the third weather type, that is, rain or snow, and the real-time temperature of the area where the road is located is If it is less than 1 reference value, the occurrence of black ice is predicted and the management information and control information on the road are transmitted to the manager device to prevent accidents in advance.

Meanwhile, when the user device is included in the vehicle, the processor includes guide information that allows the driver of the vehicle to drive the brake device of the vehicle, and the guide information may be calculated using the following equation .

[Equation 1]

In this case, cv _n is the current speed (km) of the vehicle.

Specifically, if it is predicted that an accident is likely to occur at a specific road point based on real-time weather information and location information of the area where the user is located, the processor 150 determines the location and speed of the vehicle running within a certain range from the dangerous area. can be identified through satellite signals and GPS signals, and guide information can be transmitted to drive the braking device immediately from the accident point to the vehicle located within the stopping distance calculated according to Equation 1 above. In addition, the guide information may include alarm information (sound or image) to allow the driver to immediately drive the braking device.

According to an embodiment of the present invention, the processor 150 may provide accident probability related information to a plurality of user devices located within a predetermined area in the accident occurrence area. For example, when the processor 150 generates accident probability information based on image data collected from a specific user, the user device 130 for pedestrians located within a 300 m circle diameter around the road point photographed by the user. The accident possibility information may be transmitted.

On the other hand, in an embodiment of the present invention, the communication unit 110 receives the movement path information of the user from the user device, and the processor 150, based on the movement path information and the dangerous area list, the movement It is possible to determine a dangerous area included in the path, generate new moving path information based on the determined dangerous area, and transmit it to the user device.

That is, upon receiving the movement route information of the user from the user device 200 , the processor 150 determines whether the movement route of the user belongs to a dangerous area included in the dangerous area list based on the user's location information and weather information. It determines, generates and transmits new moving route information to the corresponding user. Most of the moving route information provided by telecommunication companies or means of transportation so far is provided based on real-time traffic conditions. However, the present invention provides new movement route information through which the user can safely drive based on the accident possibility information, the accident risk information, and the dangerous area detection information. This will prevent traffic accidents in advance.

On the other hand, the new movement route information includes information that provides the user with lane information including at least one of cracks, rutting, and portholes among the roads in the movement route to allow the user to drive through a lane corresponding to a normal road. .

Meanwhile, in an embodiment of the present invention, the processor 150 determines that a distance between a plurality of dangerous regions included in the dangerous region list is less than or equal to a preset distance, and the number of daily accidents according to a weather type corresponding to the real-time weather is preset. In the case of more than the number of cases, accident possibility information on a plurality of dangerous areas may be further generated and transmitted to the user device.

For example, when each road in a cracked state is less than 50 m, and the number of daily accidents of a weather type according to real-time weather is 10 or more, information on a user within a preset range from the road or movement route including the road Further information on the possibility of an accident on the road is provided to the user who transmitted the message.

5 is a flowchart schematically illustrating a method for preventing a traffic accident through road condition determination based on a deep learning model, according to an embodiment of the present invention.

The deep learning-based traffic accident prevention method may be performed by the aforementioned traffic accident prevention apparatus 100 . In this case, the traffic accident prevention apparatus 100 may be a server computer operated by an institution or a company that manages traffic accidents as described above. However, the present invention is not limited thereto.

First, referring to FIG. 5 , the server collects public data and image data ( S610 ).

And based on the public data, each of the accident risk information for each preset weather type is generated (S620), the road area included in the image data is extracted, and the extracted road is used using a first deep learning model. A road condition in the area is analyzed, and a dangerous area list is generated based on the analyzed road condition (S630).

Meanwhile, S620 and S630 may be periodically generated and updated based on public data and image data collected periodically at a preset time period.

After S630, the real-time weather of the area where the user is located is determined based on the location information of the user device (S640).

In this case, the user device 200 is a plurality of user devices 200 registered with the server, and the server determines a region where each user is located and real-time weather of the location based on real-time location information of each user device.

On the other hand, when the user is located within a preset range from the risk area included in the risk area list and accident risk information on the weather type corresponding to the determined real-time weather, the danger area detection information is transmitted ( S650 ).

At this time, as described above, the road condition includes at least one of normal, crack, rutting, and porthole, and in the step of generating the hazardous area list, the road condition corresponds to at least one of the crack, the rutting and the porthole. , it is determined that the road is a dangerous area, and the dangerous area list including the road determined as the dangerous area is generated.

Meanwhile, although not clearly shown in the drawings, in an embodiment of the present invention, the weather type includes a first weather type indicating sunny, a second weather type indicating cloudy or fog, and a third weather type indicating rain or snow and, the step S620 of generating the accident risk information is to calculate the number of daily accidents, the number of daily deaths and the number of daily injuries for each of the first to third weather types, based on the traffic accident information and climate statistical information. can

On the other hand, although not clearly shown in the drawings, as an embodiment of the present invention, the method for preventing traffic accidents through the above-described deep learning model-based road condition determination is provided for each road condition for the first to third weather types. The method may further include calculating the accident probability information and transmitting the accident probability information based on the location information of the user device and real-time weather (not shown).

In addition, although not clearly shown in the drawings, in an embodiment of the present invention, step S630 may further include a step (not shown) of generating the risk area list by further determining the risk factors of the road, and the risk In the step of further determining the factor, the black ice in the road is determined using an autoencoder, wherein the autoencoder uses a second deep learning model to give weights to each of the R, G, and B values of the image data. The black ice of the road may be determined by converting the weighted R, G, and B values into IR values through 1D Vector Representation.

In addition, in the method of preventing traffic accidents by determining the road condition based on the deep learning model described above, the real-time weather of the road including the black ice included in the dangerous area list corresponds to a third weather type among the weather types, and the temperature is The method may further include (not shown) transmitting management information or control information on the road to a manager device when it is less than the first reference value.

In this case, when the user device is included in the vehicle, the method may further include generating and transmitting guide information for allowing a driver of the vehicle to drive a braking device of the vehicle. Meanwhile, the guide information is calculated using the following equation.

[Equation 2]

(Wherein, cv _n is the current speed (km) of the vehicle.)

On the other hand, step S610 further includes the step of receiving the movement route information of the user from the user device, the step of transmitting the guide information, based on the movement route information and the dangerous area list, the risk included in the movement route The method may further include determining an area, generating new movement route information based on the determined dangerous area, and transmitting the information to the user device (not shown).

In this case, in the step of transmitting the accident probability information according to an embodiment of the present invention, the distance between the plurality of dangerous areas included in the list is less than or equal to a preset distance, and the number of daily accidents according to the weather type corresponding to the real-time weather is set. When the number of cases is greater than or equal to the set number, accident possibility information regarding the plurality of dangerous areas may be further generated and transmitted to the user device.

The deep learning-based traffic accident prevention method according to an embodiment of the present invention described above may be implemented as a program (or application) and stored in a medium to be executed in combination with a server that is hardware.

The above-described program is C, C++, JAVA, machine language, etc. that a processor (CPU) of the computer can read through a device interface of the computer in order for the computer to read the program and execute the methods implemented as a program It may include code (Code) coded in the computer language of Such code may include functional code related to functions defining functions necessary for executing the methods, etc. can do. In addition, the code may further include additional information necessary for the processor of the computer to execute the functions or code related to memory reference for which location (address address) in the internal or external memory of the computer to be referenced. there is. In addition, when the processor of the computer needs to communicate with any other computer or server located remotely in order to execute the above functions, the code uses the communication module of the computer to determine how to communicate with any other computer or server remotely. It may further include a communication-related code for whether to communicate and what information or media to transmit and receive during communication.

The storage medium is not a medium that stores data for a short moment, such as a register, a cache, a memory, etc., but a medium that stores data semi-permanently and can be read by a device. Specifically, examples of the storage medium include, but are not limited to, ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage device. That is, the program may be stored in various recording media on various servers that the computer can access or in various recording media on the computer of the user. In addition, the medium may be distributed in a computer system connected by a network, and a computer readable code may be stored in a distributed manner.

The steps of a method or algorithm described in relation to an embodiment of the present invention may be implemented directly in hardware, as a software module executed by hardware, or by a combination thereof. A software module may contain random access memory (RAM), read only memory (ROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, hard disk, removable disk, CD-ROM, or It may reside in any type of computer-readable recording medium well known in the art to which the present invention pertains.

As mentioned above, although embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art to which the present invention pertains can realize that the present invention can be embodied in other specific forms without changing its technical spirit or essential features. you will be able to understand Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive.

10 : Traffic accident prevention system based on deep learning model
100: Traffic accident prevention device based on deep learning model
200: user device
300: public facility equipment
400 : manager device

Claims (17)

a communication unit that collects public data and image data;
an accident risk determination unit generating accident risk information for each preset weather type based on public data, respectively;
a danger area detection unit that extracts a road area included in the image data, analyzes a road condition within the extracted road area using a first deep learning model, and generates a dangerous area list based on the analyzed road condition; and
Determine the real-time weather of the area where the user is located based on the location information of the user device, and within a preset range from the accident risk information on the weather type corresponding to the determined real-time weather and the dangerous area included in the risk area list A processor that transmits danger area detection information to the user device through the communication unit when the user is located; including,
The public data includes traffic accident information and climate statistical information for a preset period,
The road condition includes at least one of normal, crack, rutting and pothole,
The dangerous area detection unit, when the road condition corresponds to at least one of the crack, the rutting, and the porthole, determines the road as a dangerous area, and the dangerous area list including the road determined as the dangerous area create,
the weather type includes a first weather type indicating sunny, a second weather type indicating cloudy or fog, and a third weather type indicating rain or snow,
The accident risk determination unit,
Calculate the number of daily accidents, the number of daily deaths and the number of daily injuries for each of the first to third weather types based on the traffic accident information and climate statistical information,
The processor is
calculating the accident probability information for each road condition for the first to third weather types, and transmitting the accident probability information based on the location information of the user device and real-time weather,
If the distance between the plurality of dangerous areas included in the list is less than or equal to a preset distance, and the number of daily accidents according to the weather type corresponding to the real-time weather is greater than or equal to the preset number,
Further generating and transmitting accident probability information about the plurality of dangerous areas to the user device,
traffic accident prevention device delete delete According to claim 1,
The danger area detection unit,
Further determining the risk factors of the road,
The risk factor includes black ice in the road,
The dangerous area detection unit determines the black ice in the road using an autoencoder,
The autoencoder uses a second deep learning model to give weights to each of the R, G, and B values of the image data, so that the weighted R, G, and B values are expressed as a one-dimensional vector representation (1D Vector Representation). to determine the black ice of the road by converting it into an IR value through
Further comprising a road containing the black ice in the dangerous area list,
Traffic accident prevention device. 5. The method of claim 4,
The processor is
When the real-time weather of the road including the black ice included in the dangerous area list corresponds to a third weather type among the weather types and the temperature is less than the first reference value,
transmitting management information or control information about the road to a manager device,
Traffic accident prevention device. 5. The method of claim 4,
The processor is
When the user device is included in the vehicle, further generating guide information for allowing the driver of the vehicle to drive the brake device of the vehicle and transmitting it to the user device,
The guide information is calculated using the following equation,
[Equation 1]

(Wherein, cv _n is the current speed (km) of the vehicle.)
Traffic accident prevention device. According to claim 1,
The communication unit,
Receive the user's moving path information from the user device,
The processor is
Based on the moving path information and the dangerous area list, determining a dangerous area included in the moving path, generating new moving path information based on the determined dangerous area and transmitting it to the user device,
Traffic accident prevention device. delete A method performed by a server, comprising:
collecting public data and image data;
generating accident risk information for each preset weather type, respectively, based on the public data;
extracting a road area included in the image data, analyzing a road condition in the extracted road area using a first deep learning model, and generating a dangerous area list based on the analyzed road condition;
determining real-time weather in a region where the user is located based on the location information of the user device; and
transmitting accident risk information on a weather type corresponding to the determined real-time weather and risk area detection information when the user is located within a preset range from a dangerous area included in the hazardous area list;
The public data includes traffic accident information and climate statistical information for a preset period,
The road condition includes at least one of normal, crack, rutting and pothole,
The risk area list creation step is,
When the road condition corresponds to at least one of the crack, the rutting, and the porthole, the road is determined as a dangerous area, and the dangerous area list including the road determined as the dangerous area is generated,
the weather type includes a first weather type indicating sunny, a second weather type indicating cloudy or fog, and a third weather type indicating rain or snow,
The accident risk information generation step is,
Based on the traffic accident information and climate statistical information, to calculate the number of daily accidents, the number of daily fatalities, and the number of daily injuries of each of the first to third weather types,
The method is
The method further comprises calculating the accident probability information for each road condition for the first to third weather types, and transmitting the accident probability information based on the location information of the user device and real-time weather,
The accident possibility information transmission step is,
When the distance between the plurality of dangerous areas included in the list is less than or equal to a preset distance, and the number of daily accidents according to the weather type corresponding to the real-time weather is greater than or equal to the preset number, information on the possibility of occurrence of accidents related to the plurality of hazardous areas Further generating and sending to the user device,
A method for preventing traffic accidents based on deep learning. delete delete 10. The method of claim 9,
The risk area list generation step further comprises the step of generating the risk area list by further determining the risk factors of the road,
In the step of further determining the risk factor, the black ice in the road is determined using an autoencoder, and the autoencoder gives weights to each of the R, G, and B values of the image data using a second deep learning model. to determine the black ice of the road by converting the weighted R, G, and B values into IR values through 1D Vector Representation,
A method for preventing traffic accidents based on deep learning. 13. The method of claim 12,
When the real-time weather of the road including the black ice included in the dangerous area list corresponds to a third weather type among the weather types and the temperature is less than the first reference value, the management information or control information about the road is transmitted to the manager device further comprising the step of transmitting,
A method for preventing traffic accidents based on deep learning. 14. The method of claim 13,
When the user device is included in the vehicle, the method further comprising the step of generating and transmitting guide information for allowing a driver of the vehicle to drive a braking device of the vehicle,
The guide information is calculated using the following equation,
[Equation 1]

(Wherein, cv _n is the current speed (km) of the vehicle.)
A method for preventing traffic accidents based on deep learning. 15. The method of claim 14,
The collection step is
Further comprising the step of receiving the movement path information of the user from the user device,
The step of transmitting the guide information,
Determining a dangerous area included in the moving path based on the moving path information and the dangerous area list, generating new moving path information based on the determined dangerous area, and transmitting the information to the user device doing,
A method for preventing traffic accidents based on deep learning. delete In a computer program stored in a computer-readable recording medium to perform a deep learning-based traffic accident prevention method in combination with a computer,
The program is
collecting public data and image data;
generating accident risk information for each preset weather type, respectively, based on the public data;
extracting a road area included in the image data, analyzing a road condition in the extracted road area using a first deep learning model, and generating a dangerous area list based on the analyzed road condition;
determining real-time weather in a region where the user is located based on the location information of the user device; and
When the user is located within a preset range from the risk area included in the risk area list and accident risk information on the weather type corresponding to the determined real-time weather, transmitting the risk area detection information;
The public data includes traffic accident information and climate statistical information for a preset period,
The road condition includes at least one of normal, crack, rutting and pothole,
The risk area list creation step is,
When the road condition corresponds to at least one of the crack, the rutting, and the porthole, the road is determined as a dangerous area, and the dangerous area list including the road determined as the dangerous area is generated,
the weather type includes a first weather type indicating sunny, a second weather type indicating cloudy or fog, and a third weather type indicating rain or snow,
The accident risk information generation step is,
Based on the traffic accident information and climate statistical information, to calculate the number of daily accidents, the number of daily fatalities, and the number of daily injuries of each of the first to third weather types,
The program is
calculating the accident probability information for each road condition for the first to third weather types, and transmitting the accident probability information based on the location information of the user device and real-time weather,
The accident possibility information transmission step is,
When the distance between the plurality of dangerous areas included in the list is less than or equal to a preset distance, and the number of daily accidents according to the weather type corresponding to the real-time weather is greater than or equal to the preset number, information on the possibility of occurrence of accidents related to the plurality of hazardous areas Further generating and sending to the user device,
A traffic accident prevention program based on deep learning.

Images