KR102565227B1 - Traffic safety device for providing risk prediction guidance and traffic safety system including the same - Google Patents

Abstract

The present invention relates to a traffic safety device for providing risk prediction guidance and a traffic safety system including the same and, more specifically, to a traffic safety device for providing risk prediction guidance and a traffic safety system including the same, which may identify objects corresponding to pedestrians and vehicles by mapping an image for a road including a crosswalk received from each of a plurality of cameras with space information corresponding to an actual space, based on this, allow a learning model to learn a passage state of the vehicles and pedestrians that have had an accident, and then guide a safety risk and protect pedestrians in the passage state expected to have an accident through an electronic display device through the learning model. The present invention may have an effect of effectively preventing a traffic accident by aggressively inducing a vehicle driver to check an accident occurrence possibility through the electronic display device and then decelerate the velocity of the vehicle or stop the vehicle while paying attention to pedestrians, and even by notifying pedestrians about the accident occurrence possibility to induce the pedestrians to protect themselves from a vehicle that violates laws.

Description

Traffic safety device for providing risk prediction guidance and traffic safety system including the same}

The present invention relates to a traffic safety device that provides risk prediction guidance and a traffic safety system including the same. and mapping to identify objects corresponding to pedestrians and vehicles, and based on this, the traffic conditions of vehicles and pedestrians in which accidents occur at crosswalks are trained in the learning model, and then through the learning model, in traffic conditions in which accidents are predicted The present invention relates to a traffic safety device that provides risk prediction guidance for protecting pedestrians and preventing traffic accidents by guiding a safety risk level through an electronic signboard device, and a traffic safety system including the same.

Currently, there are many cases where there is no traffic signal for right-turning vehicles at intersections with right-turn lanes, so pedestrian accidents caused by right-turning vehicles are increasing. There is an increasing number of traffic accidents in which a vehicle collides with a vehicle at a crosswalk after predicting departure from a yellow light, and countermeasures to prevent this are required.

In particular, in a straight lane, when another vehicle is stopped at a crosswalk in another lane located on the right side of a driving vehicle among a plurality of different lanes with the same traveling direction, the pedestrian is covered by another vehicle and the driving vehicle turns off the yellow light as a pedestrian. In order to protect pedestrians from right-turning vehicles, right-turning vehicles are forced by law to stop and start if there are pedestrians around the crosswalk. Right-turning vehicles cannot be prevented by law alone.

In the past, pedestrians and vehicles are separately detected through surveillance cameras, and vehicles passing by violating signals or speeding at crosswalks are detected and enforced, or pedestrians located close to the roadway or jaywalking are detected and guided. A surveillance system providing broadcasting is being provided.

However, this surveillance system can only prevent traffic accidents at an extremely passive level by alerting drivers to the presence of speed cameras, and even passing a crosswalk at a yellow light is not a signal violation, so pedestrians predictably depart There is a problem in which accidents cannot be actively prevented.

In addition, the existing surveillance system provides only a passive function of alerting the driver through a surveillance camera or cracking down on a vehicle that has already been violated even in the case of a law-violating vehicle making a right turn without stopping in the presence of a pedestrian, but actively It provides functions to protect pedestrians and has a tough problem.

In addition, traffic accidents between pedestrians and vehicles also occur due to law violations such as jaywalking by pedestrians, but the existing monitoring system detects jaywalking pedestrians and only provides voice warning broadcasts to pedestrians only, and these warning broadcasts are delivered to vehicles. It is difficult to become, and there is a limit to preventing accidents caused by pedestrian violations.

Korean Patent Registration No. 10-2409485

The present invention maps images received from a plurality of cameras disposed at different locations around a space to be monitored, including a road with a crosswalk, to spatial information corresponding to the space to be monitored, and a pedestrian identified in the image. And identifying an object corresponding to a vehicle, tracking the identified object according to the mapping, and providing movement information of a vehicle approaching a crosswalk and a pedestrian moving to a crosswalk in the space to be monitored, along with whether or not an accident has occurred. based learning model, predicts the possibility of an accident according to the moving state of vehicles and pedestrians through the learning model that has completed learning, and guides the risk of accidents to vehicles or pedestrians through the electronic signboard device to prevent traffic accidents. Its purpose is to prevent it from occurring.

In addition, the present invention identifies an object corresponding to a right-turning vehicle and an object corresponding to a pedestrian in images received through the plurality of cameras, and determines the right-turning vehicle corresponding to the object according to mapping information between the image and spatial information. It acquires movement information and pedestrian movement information, learns through a deep learning-based learning model, and predicts and provides the risk of an accident with a pedestrian by a right-turning vehicle through the learning model. Its purpose is to help prevent it in advance.

A traffic safety device for providing risk prediction guidance according to an embodiment of the present invention includes: an image receiver for receiving images from each of a plurality of different camera units for capturing a road area where one or more crosswalks exist as a space to be monitored; A spatial mapping unit for generating mapping information obtained by mutually mapping spatial information including actual position values of the space to be monitored and pixel coordinates of the image, for each of the plurality of camera units; and An object tracking unit that identifies and tracks a vehicle object corresponding to a vehicle and a pedestrian object corresponding to a pedestrian in an image and provides movement information in the space to be monitored based on the mapping information, and movement to pass through the crosswalk A learning model in which the correlation between the movement information of one or more vehicle objects moving to walk on the crosswalk and the movement information of one or more pedestrian objects moving to walk the crosswalk and whether or not an accident has occurred is pre-learned is set in advance, and the vehicle from the object tracking unit After receiving movement information for each object of interest including an object or a pedestrian object, generating a prediction unit that calculates an accident probability by applying it to the learning model and guide information that sets the accident probability as a safety risk level It may include a guidance unit for displaying the guidance information for inducing the vehicle to stop before passing the crosswalk by transmitting it to a preset first electronic display board device and displaying the safety risk level through the first electronic display board device.

As an example related to the present invention, the actual location value may be a GPS-based location value.

As an example related to the present invention, the object tracking unit determines the object type of the identified object according to the location and size of the object identified in the image based on the mapping information in which the size for each position in the image is set for each object type. It may be characterized in that it is identified as either a vehicle object or a pedestrian object, or the vehicle object or pedestrian object is identified in the image through a preset object identification model by learning the vehicle object and the pedestrian object in the image.

As an example related to the present invention, when the object tracking unit receives a plurality of images corresponding to the plurality of camera units, the mapping of the plurality of images respectively corresponding to the plurality of images among objects identified in each of the plurality of images. Based on the information, an object having the same location value is detected as the same object in the spatial information, and the object type of the detected object is identified as a vehicle object or a pedestrian object, and then tracking through the plurality of images, while the mapping information It may be characterized in that the movement information for the detected object in the monitored space is calculated based on.

As an example related to the present invention, the guide unit may transmit the guide information to the first electronic display board device when the accident probability is greater than or equal to a predetermined reference value.

As an example related to the present invention, the space to be monitored includes a plurality of lanes including a straight lane with a first crosswalk and a right turn lane branching from the straight lane with a second crosswalk, and the learning The model includes movement information for each of one or more vehicle objects moving to pass through the first crosswalk or second crosswalk and movement information for each of one or more pedestrian objects moving to walk in the first crosswalk or second crosswalk, and A correlation between the occurrence or non-occurrence of each of the plurality of lanes is learned in advance, and the prediction unit applies the movement information for each of the one or more objects of interest received from the object tracking unit to the learning model, thereby classifying the plurality of lanes. It may be characterized in that the accident occurrence probability is calculated.

As an example related to the present invention, the guide unit, when the probability of an accident in the right turn lane is equal to or greater than a preset reference value, provides a safety risk level according to the probability of an accident in the right turn lane to a second electronic display board arranged to correspond to the right turn lane; and Vehicle guidance information including preset first text information for a vehicle is transmitted and displayed through the second electronic signboard device, and an accident probability of the right turn lane is determined by a third electronic signboard device arranged to correspond to the right turn lane. It may be characterized in that pedestrian guidance information including a safety risk level according to the present invention and preset second text information for pedestrians is transmitted and displayed through the third electronic display board device.

As an example related to the present invention, the traffic safety device further includes a signal collecting unit that communicates with the traffic light controller to provide traffic light status information for a current traffic light condition, and the learning model is configured to use the traffic light status information and the one or more traffic light conditions. Learns a correlation between movement information for each vehicle object, movement information for each one or more pedestrian objects, and whether an accident has occurred, and the prediction unit converts the current traffic light state information received from the signal collection unit into one received from the object tracking unit. It may be characterized in that the accident occurrence probability is calculated through the learning model by applying the above movement information for each object of interest to the learning model.

A traffic safety system for providing risk prediction guidance according to an embodiment of the present invention receives images from each of a plurality of different camera units that capture a road area where one or more crosswalks exist as a space to be monitored, and the monitoring target Spatial information including actual position values in space and mapping information obtained by mutually mapping pixel coordinates of the image are generated for each of the plurality of camera units, and the vehicle corresponding to the vehicle in the image received from each of the plurality of camera units A pedestrian object corresponding to an object and a pedestrian is identified and tracked to generate movement information in the space to be monitored based on the mapping information, and movement information for each of one or more vehicle objects moving to pass through the crosswalk and the crossing In a learning model in which a correlation between movement information of one or more pedestrian objects moving to walk on the sidewalk and whether or not an accident has occurred has been pre-learned, movement information of one or more objects of interest including a vehicle object or a pedestrian object is converted to the learning model. It may include a traffic safety device that generates and transmits guidance information in which the accident probability is set as a safety risk level after calculating the probability of an accident by applying the method, and one or more electronic display board devices that output guidance information received from the traffic safety device. there is.

As an example related to the present invention, the traffic safety device generates and transmits first guidance information including the safety risk level and first text information for a vehicle when the accident probability is equal to or greater than a predetermined reference value, while generating and transmitting the safety risk level and Second guide information including second text information for pedestrians is generated and transmitted, the first signboard device is arranged to face a vehicle approaching the crosswalk and outputs the first guide information, and the second signboard device may be arranged to face a pedestrian walking on the crosswalk and output the second guide information.

The present invention maps images received from each of a plurality of surveillance cameras that take a road with one or more crosswalks as a space to be monitored, with spatial information corresponding to the space to be monitored, and objects corresponding to vehicles and pedestrians in the images. Through identification and tracking, the movement information of the vehicle and pedestrian in which the accident occurred is taught to the learning model, and the pedestrian crossing the crosswalk in a straight lane is obscured by a vehicle in a specific lane and is concerned about an accident with a vehicle traveling in another lane. The possibility of an accident can be predicted through a learning model, such as when an accident is worrisome due to a vehicle passing through a crosswalk at speed or when an accident is worrisome due to a vehicle passing through a right-turn lane without temporarily stopping when turning right. By guiding pedestrians through different electronic signage devices, the driver of the vehicle can check the possibility of an accident through the electronic signage device and actively induce the vehicle to slow down or stop the vehicle while paying attention to the pedestrian, as well as causing an accident to the pedestrian. By notifying the possibility of a traffic accident, it is possible to actively induce pedestrians to protect themselves from law-violating vehicles, thereby effectively preventing the occurrence of traffic accidents in advance.

1 is a configuration environment diagram of a traffic safety system that provides risk prediction guidance according to an embodiment of the present invention.
2 is a detailed configuration diagram of a traffic safety device that provides risk prediction guidance according to an embodiment of the present invention.
3 to 6 are diagrams illustrating operation of a traffic safety device that provides risk prediction guidance according to an embodiment of the present invention.

Hereinafter, detailed embodiments of the present invention will be described with reference to the drawings.

1 is a configuration environment diagram of a traffic safety system (traffic safety system) that provides risk prediction guidance according to an embodiment of the present invention.

As shown, the traffic safety system includes a plurality of different camera units 10 disposed at different locations for capturing a road area where one or more crosswalks exist as a space to be monitored (surveillance target area), and the monitoring A vehicle that communicates with the camera unit 10 and the electronic signboard device 20 through one or more electric signboard devices 20 disposed in the target space and a communication network and approaches the crosswalk in the space to be monitored through the electric signboard device 20 or a traffic safety device 100 that outputs safety risk-related guide information about the probability of an accident to a pedestrian trying to walk on a crosswalk.

To this end, the traffic safety device 100 receives images from each of the plurality of camera units 10, and provides space information, which is a virtual space including actual location values of the space to be monitored, and pixels of the image. Mapping information obtained by mapping coordinates to each other may be generated for each of the plurality of camera units 10 .

In addition, the traffic safety device 100 identifies and tracks a vehicle object corresponding to a vehicle and a pedestrian object corresponding to a pedestrian in images received from each of the plurality of camera units 10, and detects a vehicle object corresponding to a pedestrian based on the mapping information. Movement information in a virtual space corresponding to the space to be monitored is generated, and movement information for each of one or more vehicle objects moving to pass the crosswalk and movement information for each of one or more pedestrian objects moving to walk the crosswalk are generated. An accident probability is calculated through the learning model by applying the movement information of each of the one or more vehicle objects and one or more pedestrian objects to a learning model in which the correlation between movement information and whether or not an accident has occurred has been pre-learned, and the accident probability is calculated through the learning model. After the guide information included in the safety risk level is generated, it may be transmitted to the one or more electronic display board devices 20 to be output.

Through this, the traffic safety device 100 identifies objects corresponding to vehicles and pedestrians based on images received from each of the plurality of camera units 10 that capture the space to be monitored for the road including the crosswalk. Based on the information that maps the virtual space and the image corresponding to the real space, the space to be monitored, while tracking, the moving state of the vehicle and pedestrian identified in the image in the space to be monitored is checked, and the moving state of these vehicles and pedestrians Predict the probability of an accident based on the currently identified moving state of a vehicle or pedestrian through a deep learning-based learning model in which the correlation between accident occurrence and occurrence is learned, and inform the vehicle and pedestrian of the safety risk according to the probability of an accident occurrence By doing so, it is possible to support preventing an accident in advance by notifying at least one of the vehicle and pedestrian of the possibility of an accident when a moving state of a vehicle and pedestrian with a high possibility of accident occurs and drawing attention thereto.

In this case, the learning model described in the present invention may be composed of a deep learning algorithm, and the deep learning algorithm may be composed of one or more neural network models.

In addition, the neural network model (or neural network) described in the present invention may be composed of an input layer, one or more hidden layers, and an output layer, and the neural network model includes a deep neural network (DNN) , various types of neural networks such as a recurrent neural network (RNN), a convolutional neural network (CNN), and a support vector machine (SVM) may be applied.

Based on the above configuration, the detailed operation configuration of the traffic safety system including the traffic safety device 100 according to an embodiment of the present invention will be described with reference to the drawings below.

2 is a detailed configuration diagram of a traffic safety device 100 constituting a traffic safety system according to an embodiment of the present invention.

As shown, the traffic safety device 100 may include a communication unit 110, a storage unit 120, and a control unit 130.

First of all, the communication unit 110 may communicate with the plurality of camera units 10 and one or more electronic display board devices 20 through a communication network.

At this time, the communication network described in the present invention may include a wired / wireless communication network, and as an example of such a wireless communication network, a wireless LAN (WLAN), a DLNA (Digital Living Network Alliance), a WiBro (Wireless Broadband: Wibro), Wimax (World Interoperability for Microwave Access: Wimax), GSM (Global System for Mobile communication), CDMA (Code Division Multi Access), CDMA2000 (Code Division Multi Access 2000), EV-DO (Enhanced Voice-Data Optimized or Enhanced Voice- Data Only), WCDMA (Wideband CDMA), HSDPA (High Speed Downlink Packet Access), HSUPA (High Speed Uplink Packet Access), IEEE 802.16, Long Term Evolution (LTE), LTE-A (Long Term Evolution- Advanced), Broadband Wireless Mobile Broadband Service (WMBS), 5G mobile communication service, Bluetooth, LoRa (Long Range), RFID (Radio Frequency Identification), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, Near Field Communication (NFC), Ultra Sound Communication (USC), Visible Light Communication (VLC), Wi-Fi, Wi-Fi Direct and the like may be included. In addition, wired communication networks include wired local area network (LAN), wired wide area network (WAN), power line communication (PLC), USB communication, Ethernet, serial communication, optical/coaxial cables, etc. may be included.

In addition, the storage unit 120 may store various data necessary for the operation of the traffic safety device 100 .

In addition, the control unit 130 performs overall control functions of the traffic safety device 100, and the control unit 130 may include RAM, ROM, CPU, GPU, and a bus, and RAM, ROM, and CPU , GPUs, etc. may be connected to each other through a bus.

In addition, at least one of the communication unit 110 and the storage unit 120 may be included in the controller 130.

In addition, the control unit 130 includes an image receiving unit 131, a spatial mapping unit (or mapping unit) 132, an object tracking unit 133, a prediction unit 134, a guidance unit 135, and the like. It can be.

In addition, at least one of the plurality of components configured in the control unit 130 may mutually communicate with at least one of the plurality of camera units 10 and one or more electronic signboard devices 20 through the communication unit 110, In the following description, communication configuration through the communication unit 110 will be omitted.

Hereinafter, a detailed operation configuration of the traffic safety device 100 will be described with reference to FIG. 3 .

First of all, each of the plurality of camera units 10 may capture an area of the road where one or more crosswalks exist as a space to be monitored (target area to be monitored), and generate an image of the space to be monitored at different locations to detect the traffic It can be transmitted to the safety device 100.

Accordingly, the image receiving unit 131 may receive the image from each of the plurality of camera units 10 .

In addition, spatial information corresponding to a space to be monitored, which is a real space corresponding to the real road area, may be set in advance in the spatial mapping unit 132, and the spatial information may be set in advance for each different location of the target space to be monitored. It can be composed of a virtual space including a location value.

In this case, the actual position value (or position value) may mean a global positioning system (GPS)-based coordinate value.

In addition, the space mapping unit 132 maps pixel coordinates and actual position values of the virtual space for each position of the space to be monitored, which appears in the image received from the camera unit 10 for each of the plurality of camera units 10. One mapping information can be created.

To this end, the space mapping unit 132 sets a surveillance region corresponding to the surveillance target space in the image of the camera unit 10 that captures the surveillance target space, sets a plurality of reference points in the surveillance region, , It is possible to communicate with the mobile GPS device through the communication unit 110 to mutually map the pixel coordinates of the reference point in the image and the actual position value transmitted by the mobile GPS device for each of the plurality of reference points.

In addition, the spatial mapping unit 132 calculates a mapping function for obtaining a position value corresponding to a pixel coordinate based on the pixel coordinates and position values obtained for each of the plurality of reference points, and provides mapping information including the mapping function. may be created, and the mapping information may be stored in the storage unit 120 .

In this case, the traffic safety device 100 may further include a user input unit for receiving a user input, and the spatial mapping unit 132 may match pixel coordinates for each of the plurality of reference points based on the user input. and position values may be set, and the mapping function may be calculated based on mutually matched pixel coordinates and position values for each of a plurality of reference points.

Also, the spatial mapping unit 132 may calculate the mapping function by further considering flatness according to the spatial information.

In addition, the space mapping unit 132 performs a size by location (pixel coordinates) of a vehicle object corresponding to a vehicle among a plurality of preset object types and a location (pixel coordinates) of a pedestrian object corresponding to a pedestrian in an image of a vehicle. After generating setting information in which the size is set, it may be included in the mapping information.

At this time, the spatial mapping unit 132 may generate the setting information according to the user input and then include it in the mapping information.

According to the configuration described above, the spatial mapping unit 132 may generate a plurality of mapping information corresponding to the plurality of camera units 10 and store them in the storage unit 120 .

At this time, the camera identification information corresponding to the camera unit 10 is set in advance for each of the plurality of camera units 10, and the camera identification information can be included in an image generated by the camera unit 10 and transmitted. The mapping unit 132 may match and store camera identification information corresponding to mapping information among a plurality of camera identification information corresponding to the plurality of camera units 10 and store them in the storage unit 120 . In addition, different camera identification information may be set between the plurality of camera units 10 .

Meanwhile, the object tracking unit 133 identifies an object type according to the location and size of an object identified in the image based on the mapping information, and sets the object type of the identified object to a vehicle, which is an object corresponding to a vehicle. It may be identified as either an object or a pedestrian object corresponding to a pedestrian.

Alternatively, the object tracking unit 133 has an object detection model for distinguishing and identifying a plurality of different preset object types corresponding to vehicles or pedestrians in the image received from the camera unit 10. It may be set in advance, and an object corresponding to a vehicle or a pedestrian may be identified through the object identification model, and object information including an object type and size of the identified object may be generated.

At this time, the object tracking unit 133 calculates a position value of spatial information corresponding to a pixel coordinate at which the identified object is located in an image based on the mapping information, and then converts the calculated position value to the object information. can include

In addition, the object identification model may be composed of a deep learning-based model, and the vehicle object and the pedestrian object in the image may be learned to distinguish and identify the vehicle object and the pedestrian object in the image.

In addition, as shown in FIG. 4, the object tracking unit 133 generates images for each object identified in each of a plurality of images received from each of the plurality of camera units 10 through the image receiving unit 131. Based on object information and a plurality of mapping information corresponding to the plurality of images, among objects identified in each of the plurality of images, an object having the same position value between each other in spatial information is detected as the same object, and each of the plurality of images The same object can be identified in , through which the same specific object is identified in each of the plurality of images, and surveillance according to spatial information of the specific object identified for each of the plurality of images based on the mapping information for each of the plurality of images. The specific object may be tracked while determining the exact location (GPS coordinates) of the target space.

At this time, the object tracking unit 133 compares object information generated for the identified object in each of the plurality of images, and compares the plurality of images and object information for each identified plurality of objects respectively corresponding to each other. When the actual position value is less than or equal to a preset threshold value, the plurality of objects identified in different images may be determined as the same object.

In addition, the object tracking unit 133 may assign an object ID to a specific object identified as the same object in the plurality of images and track it, such as a preset feature point extraction algorithm or a Kalman filter. The same object may be tracked in the plurality of images using an object tracking algorithm.

In addition, the object tracking unit 133 tracks the object identified in each of the plurality of images, and based on the mapping information for each of the plurality of images stored in the storage unit 120, the identified object in the surveillance target space. Movement information on a movement path of the identified object according to a change in the position value of the object may be calculated.

In this case, the object tracking unit 133 may generate the movement information at a predetermined period or in real time for the identified object.

In addition, the object tracking unit 133 tracks a change in the actual position value of the identified object in the space to be monitored according to the space information, and determines the starting position, final position, moving direction, moving speed, and movement speed of the identified object. Movement information including a movement path and the like may be generated.

At this time, the storage unit 120 may include an object DB for storing object information on the object identified in the image.

Accordingly, the object tracking unit 133 stores object information about the object identified in the image in the object DB, and when generating movement information corresponding to the object information, the movement information is matched with the object information It can be stored in the object DB.

In this case, the object information and movement information may each include an object ID assigned to the object.

Also, the movement information may include the object information for distinguishing (distinguishing) from other objects.

At this time, in the space information, one or more regions of interest respectively corresponding to one or more crosswalks existing in the space to be monitored may be set in advance.

In addition, the object tracking unit 133 identifies the object type of the object through the above-described image analysis of the image, identifies a vehicle object in which the object type corresponds to a vehicle, and then tracks it. , It is possible to generate movement information for the vehicle object.

In this case, in the case of the vehicle object, the object tracking unit 133 tracks the vehicle object while being located within a preset radius based on the region of interest, and as a result of tracking the vehicle object, the distance to the region of interest decreases and moves to the region of interest. The movement information may be generated only for the vehicle object.

In addition, the object tracking unit 133 identifies the object type of the object through image analysis of the image, identifies a pedestrian object whose object type corresponds to a pedestrian, and then tracks the pedestrian object. It is possible to generate movement information for .

In this case, in the case of the pedestrian object, the object tracking unit 133 may generate the movement information for only the pedestrian object located within a preset radius based on the region of interest.

Through this, the object tracking unit 133 may generate movement information by tracking objects corresponding to vehicles and pedestrians approaching the region of interest corresponding to the crosswalk.

Meanwhile, the prediction unit 134 includes movement information for each vehicle object moving to pass a crosswalk existing in the monitoring target space, movement information for each one or more pedestrian objects moving to walk the crosswalk, and A learning model in which a correlation between occurrences of an accident is pre-learned may be set in advance.

To this end, the storage unit 120 may include a learning DB in which learning data for learning the learning model is stored, and the prediction unit 134 receives the learning data from an external device through the communication unit 110. may be received and stored in the learning DB.

In addition, when an accident occurs in a specific region of interest in the space to be monitored, the prediction unit 134 receives region identification information corresponding to the specific region of interest in which the accident occurred and event information about the time of occurrence of the accident from an external device, or the object In conjunction with the tracking unit 133, the event information may be automatically generated when a vehicle object and a pedestrian object in an image are located within a preset distance within a specific region of interest.

In addition, the prediction unit 134, when receiving or generating the event information, based on one or more movement information for each vehicle object generated within a preset past time based on the accident occurrence time of the event information and the accident occurrence time After extracting movement information by one or more pedestrian objects generated within a preset past time from the object DB, setting the occurrence of an accident according to the extracted movement information by one or more vehicle objects, the extracted movement information by one or more pedestrian objects, and the event information. The generated accident occurrence information can be matched with each other to generate learning data including this information, and then the learning model can be trained.

In addition, the prediction unit 134 interlocks with the object tracking unit 133 and at least one of movement information for each vehicle object and movement information for each one or more pedestrian objects when an accident does not occur in a preset period and an accident Accident occurrence information set to non-occurrence may be mutually matched to generate learning data including the accident occurrence information, and then the learning model may be trained.

That is, the prediction unit 134 may allow the learning model to learn a correlation between traffic conditions of vehicles, traffic conditions of pedestrians, and occurrence of accidents.

Thereafter, when learning of the learning model is completed, the prediction unit 134 generates one or more vehicles through the object tracking unit 133 based on the plurality of images received from the plurality of camera units 10. Movement information for each object or one or more object movement information for each pedestrian may be received from the object tracker 133 .

At this time, in the object tracking unit 133, a vehicle object and a pedestrian object are preset as objects of interest, and the object tracking unit 133 transmits movement information for each one or more objects of interest to the prediction unit 134. can be provided to

In addition, the prediction unit 134 applies the movement information for each object of interest provided by the object tracking unit 133 to the learning model, and calculates the probability of occurrence of an accident corresponding to the movement information for each object of interest. can be derived through

In this case, the movement information for each object of interest may include at least one of movement information for each one or more vehicle objects and movement information for each one or more pedestrian objects based on a specific time point, and It can mean traffic situation.

In addition, the movement information for each object of interest may include only movement information for each one or more vehicle objects, or only movement information for each one or more pedestrian objects, or may include movement information for each of one or more vehicle objects and one or more pedestrian objects. there is.

In addition, the object tracking unit 133 tracks the movement state of each object of interest, including a vehicle object or a pedestrian object that changes in real time in the image, and updates movement information for each object of interest in real time in real time. It may be provided to the prediction unit 134.

Through this, the object tracking unit 133 can provide the traffic situation according to the movement of vehicles and pedestrians in the space to be monitored to the prediction unit 134 in real time, and the prediction unit 134 determines the real-time traffic situation. Whenever the reflected movement information for each object of interest is received, the movement information for each object of interest received at the same specific point in time is applied to the learning model to predict the probability of accident occurrence (probability of accident occurrence) corresponding to the specific point in time. can provide.

Alternatively, the object tracking unit 133 may generate movement information for each object of interest by tracking the object of interest for a preset period of time, and then provide the movement information to the prediction unit 134 . ) may periodically calculate the accident occurrence probability by applying movement information for each object of interest periodically provided from the object tracking unit 133 to the learning model.

In addition, the prediction unit 134 applies movement information of each of one or more objects of interest received from the object tracking unit 133 as an input to the learning model, and thinks in correspondence with the input through the learning model. The probability of occurrence and the probability of no accident (probability of non-occurrence of an accident) can be calculated as output.

In addition, upon receiving the prediction information from the prediction unit 134, the guidance unit 135 sets the accident probability according to the prediction information as a safety risk level (or accident risk level), and guide information including the safety risk level. After generating, the guide information may be displayed on the electronic signboard device 20 by transmitting it to a preset electronic signboard device 20 (eg, the first electronic signboard device 20a of FIG. 5 ).

At this time, the guide unit 135 may generate the guide information and transmit the guide information to the electric signboard device 20 only when the accident probability is greater than or equal to a predetermined reference value.

In addition, the guide information may include the safety risk level and guide contents including at least one of video, audio, and text for arousing user's attention, and the electronic signboard device 20 determines the safety risk level according to the guide information. And guidance content including at least one of video, audio, and text may be output.

In addition, the electric signboard device 20 is disposed in the space to be monitored so as to face the vehicle and can display the guide information so that a driver of a vehicle approaching a crosswalk can check the information, and the information displayed through the electric signboard device 20 is displayed. The driver of the vehicle who has checked the guidance information can check the safety risk level and induce the vehicle to stop or pause before passing the crosswalk if the probability of an accident is high.

For example, as shown in FIG. 5 , the guidance unit 135 may generate guidance information including a preset guidance text and a safety risk level. For example, “The safety risk level is 90%. , pass while protecting pedestrians” may be transmitted to the first electronic display board device 20a so as to be output.

In addition, the first electronic display board device 20a may be disposed in a space to be monitored so as to face pedestrians passing on a crosswalk, and through this, pedestrians passing (crossing) the crosswalk may be placed on the first electronic display board device 20a By providing the displayed guidance information so that it can be checked, it can be induced to cross the crosswalk while paying more attention when the safety risk is high.

Through the above-described configuration, the present invention captures a road area where one or more crosswalks exist as a space to be monitored through a plurality of camera units 10, and spatial information on the space to be monitored and the plurality of camera units ( 10) Reciprocally mapping the images received from each to identify and track vehicles and pedestrians passing in the space to be monitored as objects of interest through the images, and pass patterns in which accidents occur to a deep learning-based preset learning model. Accidents calculated by applying the movement information of vehicles and pedestrians obtained by tracking a vehicle driving toward a crosswalk in the space to be monitored and a pedestrian trying to pass through the crosswalk through video after the learning model is completed are applied to the learning model It is possible to predict a traffic situation with a high probability of occurrence of an accident according to a probability and output guidance information to call attention to at least one of a vehicle and a pedestrian about a traffic accident through the electronic signboard device 20, so that the vehicle confirms the guidance information. By encouraging drivers to stop temporarily at crosswalks, it is possible to actively intervene and prevent traffic accidents at crosswalks in advance.

On the other hand, if the space to be monitored includes multiple crosswalks and an intersection composed of multiple lanes, safety for vehicles or pedestrians passing through the lane is predicted by predicting the probability of an accident according to traffic conditions for each lane constituting the intersection. Guidance information on the degree of risk may be provided, which will be described in detail with reference to FIG. 5 .

First of all, the space to be monitored may include a plurality of lanes including a straight lane with a first crosswalk and a right turn lane with a second crosswalk diverging from the straight lane.

In addition, the learning model includes movement information for each of one or more vehicle objects moving to pass any one of the first and second crosswalks and one or more pedestrians moving to walk to any one of the first and second crosswalks. A correlation between movement information for each object and whether an accident has occurred in each of the plurality of lanes may be learned in advance.

At this time, the prediction unit 134 provides movement information for each of one or more vehicle objects moving to pass through any one of the first and second crosswalks and movement to walk to one of the first and second crosswalks. Learning data may be generated by matching movement information for each pedestrian object with a plurality of accident occurrence information corresponding to the straight lane and the right turn lane, respectively, and the learning data may be taught to the learning model.

Also, the accident occurrence information may include lane identification information for a lane corresponding to the accident occurrence information.

Accordingly, when learning of the learning model is completed, the prediction unit 134 applies the movement information for each object of interest received from the object tracking unit 133 to the learning model to determine the probability of accident occurrence for each of the plurality of lanes. can be calculated, and prediction information on the accident probability for each of the plurality of lanes can be provided to the guidance unit 135 .

In this case, the prediction information may include a plurality of lanes, lane identification information corresponding to each lane, and an accident probability. For example, while including first lane identification information corresponding to a straight lane and accident probability, corresponding to a right turn lane Secondary car identification information and accident probability may be included.

In addition, the first electric signboard device 20a may be configured to face a vehicle approaching the crosswalk of the straight lane, and the guidance unit 135 may provide a first display corresponding to the straight lane based on the prediction information. When the accident probability matched to the primary lane identification information is equal to or greater than a predetermined reference value, guide information including a safety risk level and guide text according to the accident probability of the straight lane is generated, and the first electronic display board arranged to correspond to the straight lane can be transmitted to the device 20a.

At this time, in the guidance unit 135, a plurality of different lane identification information and corresponding device identification information of a plurality of electric signboard devices 20 may be matched and set in advance, and accordingly, the prediction unit 134 The electronic display board device 20 corresponding to the lane identification information of the lane for which the accident probability is calculated to be greater than or equal to a predetermined reference value may be identified, and guidance information may be transmitted to the identified electronic display device 20 .

In addition, the guidance unit 135 is arranged to correspond to the right turn lane when the accident probability of the right turn lane matched to the second lane identification information corresponding to the right turn lane based on the prediction information is equal to or greater than a preset reference value. By transmitting vehicle guidance information (vehicle guidance information) including a safety risk level according to the probability of an accident in the right turn lane and preset first text information for a vehicle to the second electronic display board device 20b, the second electronic display board device 20b ), the vehicle guidance information may be displayed.

For example, the second electronic display board 20b is disposed in the space to be monitored so as to face the right-turning vehicle passing through the right-turning lane, and outputs the vehicle guidance information received from the traffic safety device 100 to the driver of the right-turning vehicle. may display the vehicle guidance information so as to be verifiable.

At this time, the vehicle guidance information includes information such as "The safety risk level is over 60%. Please look around and pass while protecting pedestrians." By doing so, it is possible to protect pedestrians crossing the crosswalk in the right turn lane from the right turn vehicle.

In addition, the guidance unit 135 not only transmits vehicle guidance information to the second electric signboard device 20b based on the prediction information, but also provides safety according to the accident probability of the right turn lane based on the prediction information. After generating pedestrian guidance information (guide information for pedestrians) including a degree of danger and preset second text information for pedestrians, the third electronic display board device 20c is transmitted to the third electronic display board device 20c arranged to correspond to the right turn lane. Through (20c), pedestrian guidance information can be displayed.

For example, the third electric signboard device 20c is disposed in the space to be monitored so as to face a pedestrian crossing the second crosswalk existing in the right turn lane and transmits the pedestrian guidance information received from the traffic safety device 100. It can be output so that a pedestrian crossing the second crosswalk existing in the right turn lane can check the pedestrian guidance information.

At this time, the pedestrian guidance information may include content such as "The safety risk level is over 60%. Wait for 5 seconds, look around, and cross." By guiding approaching vehicles to cross the crosswalk with caution, it is possible to assist pedestrians in protecting themselves from vehicles attempting to pass the right turn lane without stopping.

As described above, the present invention maps images received from each of a plurality of surveillance cameras that take a road with one or more crosswalks as the space to be monitored with spatial information corresponding to the space to be monitored, and in the image, the vehicle and Through the identification and tracking of objects corresponding to pedestrians, the movement information of vehicles and pedestrians in which accidents have occurred is taught to the learning model. Accidents are predicted through a learning model when there is a concern about an accident, when an accident is concerned due to a vehicle passing through a crosswalk at speed, or when an accident is concerned due to a vehicle passing through a right-turn lane without temporarily stopping when turning right By guiding the possibility of occurrence to vehicles and pedestrians through different display board devices 20, the vehicle driver checks the possibility of an accident through the display board device 20 and actively slows down the vehicle or stops the vehicle while paying attention to pedestrians. In addition, it can effectively prevent traffic accidents in advance by notifying pedestrians of the possibility of an accident and actively inducing pedestrians to protect themselves from law-violating vehicles.

On the other hand, the present invention learns the traffic conditions of the space to be monitored in connection with the signal conditions of the traffic lights installed in the space to be monitored, which corresponds to the road area, and predicts the traffic conditions with high possibility of accidents for each signal condition. The occurrence can be prevented in advance, which will be described in detail with reference to FIG. 6 .

As shown, the control unit 130 may further include a signal collection unit 136, and the signal collection unit 136 is one disposed in the space to be monitored through the communication unit 110. By communicating with the traffic light controller 200 that controls the above traffic lights, it is possible to collect traffic light state information (or signal state information) for the current traffic light state from the traffic light controller 200, and the traffic light state information is converted into the prediction unit ( 134) can be provided.

In this case, the traffic light state information (or signal state information) may include information about signal states for each of the one or more traffic lights.

Accordingly, the prediction unit 134 may cause the learning model to learn a correlation between the traffic light state information, the movement information for each one or more vehicle objects, the movement information for each one or more pedestrian objects, and whether an accident has occurred. .

In addition, the prediction unit 134 converts the current traffic light state information received from the signal collection unit 136 to movement information for each one or more objects of interest received from the object tracking unit 133 after learning of the learning model is completed. It is applied to the learning model together with to calculate the accident occurrence probability through the learning model.

In addition, the prediction unit 134 provides prediction information including the calculated accident probability to the guidance unit 135, and the guidance unit 135, based on the prediction information, as described above. Guide information to be displayed on the electronic signboard device 20 may be generated and transmitted to the electronic signboard device 20 .

Components described in the embodiments of the present invention, for example, a storage unit such as a memory, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable One or more general-purpose computers or hardware, such as gate arrays, programmable logic units (PLUs), microprocessors, software containing instruction sets, or combinations thereof, or any other device capable of executing and responding to instructions. It can be implemented using a special purpose computer.

The foregoing may be modified and modified by those skilled in the art without departing from the essential characteristics of the present invention. Therefore, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention, but to explain, and the scope of the technical idea of the present invention is not limited by these embodiments. The protection scope of the present invention should be construed according to the claims below, and all technical ideas within the equivalent range should be construed as being included in the scope of the present invention.

10: camera unit 20: electronic display board device
100: traffic safety device 110: communication department
120: storage unit 130: control unit
131: image receiving unit 132: space mapping unit
133: object tracking unit 134: prediction unit
135: guidance unit 136: signal collection unit
200: traffic light controller

Claims (10)

an image receiving unit that receives images from each of a plurality of different camera units that capture a road area of multiple lanes including one or more crosswalks as a space to be monitored;
a signal collecting unit that communicates with the traffic light controller and provides traffic light status information about the current traffic light status;
a spatial mapping unit generating mapping information obtained by mutually mapping spatial information including an actual position value of the space to be monitored and pixel coordinates of the image, for each of the plurality of camera units;
an object tracking unit that identifies and tracks a vehicle object corresponding to a vehicle and a pedestrian object corresponding to a pedestrian in images received from each of the plurality of camera units, and provides movement information in the space to be monitored based on the mapping information;
The traffic light state information, movement information for each of a plurality of vehicle objects moving through multiple lanes to pass the crosswalk, movement information according to the location of one or more pedestrian objects that want to use the crosswalk, and the vehicle and pedestrian A learning model in which the correlation between whether an accident occurs or not is set in advance, and the position of each object of interest including the traffic light state information received from the signal collection unit and the vehicle object and pedestrian object received from the object tracking unit a prediction unit for calculating an accident probability by applying movement information according to the learning model; and
After generating vehicle guide information including the probability of an accident, it is transmitted to the first electric signboard device disposed for vehicles approaching the crosswalk, thereby reducing the risk of accidents with pedestrians due to blind spots or speeding conditions caused by vehicles in other lanes. The vehicle guidance information notifying the approaching vehicle is displayed through the first electronic display board device, and after generating the pedestrian guidance information including the probability of occurrence of the accident, a second electronic display board disposed for pedestrians using the crosswalk A traffic safety device that provides risk prediction guidance including a guidance unit for displaying the guidance information for pedestrians, which is transmitted to the device and informs the pedestrian of the risk of an accident with the current driving vehicle, inducing them to stop using the crosswalk, through the second electronic display board device .
The method of claim 1,
The traffic safety device providing risk prediction guidance, characterized in that the actual location value is a GPS-based location value.
The method of claim 1,
The object tracking unit converts the object type of the identified object into one of a vehicle object and a pedestrian object according to the location and size of the object identified in the image based on the mapping information in which the size for each position in the image is set for each object type. A traffic safety device that provides risk prediction guidance, characterized in that the vehicle object or pedestrian object in the image is identified or the vehicle object or pedestrian object in the image is learned and the vehicle object or pedestrian object is identified in the image through a preset object identification model.
The method of claim 1,
When receiving a plurality of images corresponding to the plurality of camera units, the object tracking unit obtains the spatial information based on the plurality of mapping information corresponding to the plurality of images among objects identified in each of the plurality of images. Objects having the same position value are mutually detected as the same object, and the object type of the detected object is identified as a vehicle object or a pedestrian object, and then tracked through the plurality of images, based on the mapping information, in the space to be monitored. A traffic safety device that provides risk prediction guidance, characterized in that for calculating movement information for the detected object.
The method of claim 1,
The guidance unit transmits the guidance information to the first electronic display board device when the accident probability is equal to or greater than a predetermined reference value.
The method of claim 1,
The space to be monitored includes a plurality of lanes including a straight lane with a first crosswalk and a right turn lane branching from the straight lane with a second crosswalk,
The learning model includes movement information for each of one or more vehicle objects moving to pass the first crosswalk or the second crosswalk and movement for each one or more pedestrian objects moving to walk to the first crosswalk or the second crosswalk. A correlation between the information and whether each of the plurality of cars has an accident is learned in advance,
The prediction unit calculates an accident probability for each of the plurality of lanes by applying the movement information for each of the one or more objects of interest received from the object tracking unit to the learning model. Device.
The method of claim 6,
When the probability of an accident in the right turn lane is greater than or equal to a preset reference value, the information unit transmits a safety risk level according to the probability of an accident in the right turn lane to a second electronic display board arranged to correspond to the right turn lane and a preset first text for the vehicle. Vehicle guide information including information is transmitted and displayed through the second electronic signboard device, and the safety risk according to the probability of an accident in the right turn lane and the safety risk level according to the probability of occurrence of an accident in the right turn lane and the third electronic signboard device disposed to correspond to the right turn lane and a warning message for pedestrians are provided. A traffic safety device that provides risk prediction guidance, characterized in that the pedestrian guidance information including the set second text information is transmitted and displayed through the third electronic display board device.
delete Receiving images from each of a plurality of different camera units that capture a road area of multiple lanes including one or more crosswalks as a space to be monitored, and communicating with a traffic light controller to receive traffic light state information on a current traffic light state, Spatial information including actual position values of the space to be monitored and pixel coordinates of the image are mutually mapped to generate mapping information for each of the plurality of camera units, and the image received from each of the plurality of camera units corresponds to the vehicle. It identifies and tracks a vehicle object and a pedestrian object corresponding to a pedestrian, generates movement information in the space to be monitored based on the mapping information, and moves multiple lanes to pass the traffic light state information and the crosswalk. The correlation between the movement information for each of a plurality of vehicle objects to be used, the location and movement information for each of one or more pedestrian objects that want to use the crosswalk, and whether or not an accident between the vehicle and the pedestrian has occurred is received in the learning model pre-learned. A vehicle approaching the crosswalk by calculating the probability of an accident by applying the traffic light state information and the location and movement information for each object of interest including the vehicle object and the pedestrian object, and then providing vehicle guidance information including the accident probability. The vehicle guide information, which informs the approaching vehicle of the risk of an accident with a pedestrian due to a blind spot or speeding condition by a vehicle in another lane, is displayed through the first electronic signboard device, After generating pedestrian guidance information including the probability of occurrence of the accident, it is transmitted to the second electric signboard device disposed for the pedestrian using the crosswalk to inform the risk of an accident with the currently driving vehicle and induce the pedestrian to stop using the crosswalk A traffic safety device for displaying the guide information for pedestrians through the second electronic display board device;
A first electronic display board device that receives and outputs vehicle guidance information including an accident probability from the traffic safety device and is disposed to face a vehicle approaching the crosswalk;
Traffic providing risk prediction guidance, characterized in that it includes a second electric sign board device that receives and outputs pedestrian guidance information including an accident probability from the traffic safety device and is disposed to face pedestrians walking on the crosswalk safety system. delete

Images