WO2021254175A1

WO2021254175A1 - Road safety monitoring method and system, and computer device

Info

Publication number: WO2021254175A1
Application number: PCT/CN2021/098258
Authority: WO
Inventors: 杨阳; 冷继南; 吕跃强; 程林松
Original assignee: 华为技术有限公司
Priority date: 2020-06-17
Filing date: 2021-06-04
Publication date: 2021-12-23
Also published as: CN113808409A; CN113808409B

Abstract

The present application discloses a road safety monitoring method and system, and a computer device, achieving attribute recognition or source locating of an object thrown on a road, providing an accurate road safety alarm, and reducing traffic accidents on the road caused by the thrown object. The road safety monitoring method comprises: acquiring road monitoring information, and determining whether there is a thrown object on a road; acquiring trajectory information of at least one object on the road according to the road monitoring information; determining the attribute of the thrown object according to the trajectory information of the at least one object; and issuing an alarm according to the attribute of the thrown object, the at least one object comprising the thrown object and/or a non-thrown object. The method of the present application can improve the accuracy of attribute detection of the thrown object, and a worker can remove the thrown object in time according to alarm information, so as to reduce road traffic accidents.

Description

Method, system and computer equipment for road safety monitoring

Technical field

The invention relates to the field of computers, and relates to a method, system and computer equipment for road safety monitoring.

Background technique

Keeping the road clean and clean is very important for the rapid passage of vehicles. However, as the number of roads increases, the number of vehicles increases, and the spillage from the vehicles has become one of the common causes of traffic accidents on the road. When a fast-moving motor vehicle evades or collides with road sprays, it is easy to cause major property losses, even major traffic accidents, and casualties. Therefore, timely identification and treatment of spilled objects on the road has become an important issue.

The identification and handling of sprays has always been a difficult problem. First of all, because of the various types, materials, and shapes of sprays, the risk of sprays can be divided into major traffic hazards and low traffic hazards. The treatment methods are also completely different. Secondly, it is necessary to timely judge and find the source of the scattered objects, and block the source of the scattered objects. In response to this problem, if the thrown object is judged only by visual methods, it is easy to make wrong judgments.

Summary of the invention

The present application discloses a road safety monitoring method, computer equipment and system, which can realize the attribute recognition or source location of the scattered objects on the road, and reduce the traffic accidents caused by the scattered objects on the road.

In the first aspect, this application provides a road safety monitoring method, including:

Obtain road monitoring information and determine the presence of sprinkles on the road;

Obtain the trajectory information of at least one object on the road according to the road monitoring information, the at least one object includes a spray and/or at least one non-spray; judge the attribute of the spray according to the trajectory information of the at least one object;

According to the properties of the sprinkles, an alarm is issued.

Through this method, the accuracy of determining the attributes of the thrown objects can be improved, thereby determining the degree of danger of the throwing objects, and issuing accurate warning information, which can effectively reduce the possibility of traffic accidents caused by the thrown objects.

In a feasible implementation manner, acquiring the trajectory information of at least one object on the road, and judging the attribute of the spray object according to the trajectory information of the at least one object includes:

Obtain the trajectory information of the projectile on the road, and determine the attribute of the projectile according to the trajectory information of the projectile; or, obtain the trajectory information of at least one non-projectile on the road, and determine the trajectory information of the at least one non-projectile Attribute; or,

Obtain the trajectory information of at least one non-spraying object and the trajectory information of the spraying object on the road, and determine the attribute of the spraying object according to the trajectory information of the at least one non-spraying object and the trajectory information of the spraying object.

Through the trajectory information of the thrown object, such as speed, acceleration, stopover time on the trajectory, etc., the weight of the thrown object can be accurately judged, and the attribute of the thrown object can be accurately judged, such as whether the material is plastic, paper or metal; through the trajectory information of the non-sprayed object , Such as speed, acceleration, trajectory line, can judge whether the non-sprayed object has evasive behavior against the sprayed object, so as to accurately determine the degree of danger of the sprayed object.

In a feasible implementation manner, obtaining road monitoring information includes: obtaining road monitoring image information through a video monitoring device on the road, and obtaining road monitoring radar information through a radar device. By integrating image information and radar information, the accuracy of judging the trajectory of the projectile can be improved.

In a feasible implementation manner, obtaining the trajectory information of at least one object on the road includes:

The first trajectory information of at least one object is acquired according to radar information, the second trajectory information of at least one object is acquired according to image information, and the trajectory information of at least one object is obtained by combining the first trajectory information and the second trajectory information. At the near end, the accuracy of the trajectory information in the image information is higher, while at the far end, the accuracy of the trajectory information in the radar information is higher. By combining the respective advantages of radar information and image information, Get more accurate trajectory information in the full range of the end.

In a feasible implementation manner, the method further includes: obtaining the image characteristics of the tossing object according to the image information.

In a feasible implementation manner, the method further includes: judging the attributes of the thrown object according to the image characteristics of the thrown object. Image features, such as size and shape, can assist in improving the accuracy of judging the attributes of the sprinkles.

In a feasible implementation manner, when at least one object includes a sprinkle, the method for judging the attribute of the at least one object sprinkle specifically includes:

Through the feature database to compare the trajectory information of the spray and the image characteristics of the spray, the attributes of the spray are judged.

The characteristic data inventory contains the information of the sprinkles whose attributes have been determined. Through feature comparison, the attributes of the sprinkles can be judged more quickly and accurately.

In a feasible implementation manner, in the case of unsuccessful comparison with the feature database, the trajectory information or image feature of the projectile is refreshed into the feature database. Refreshing the characteristic database can continuously enrich the characteristic data of the sprinkles in the characteristic database, and improve the characteristic database through continuous accumulation, so as to achieve the purpose of improving the speed and accuracy of judging the properties of the sprinkles.

In a feasible implementation manner, the attributes of the thrown object include the degree of danger of the thrown object.

In a feasible implementation manner, different levels of alarms are issued according to the different levels of danger of the thrown objects.

In a feasible implementation, determining the presence of sprinkles on the road includes:

The object in the image information is detected by the optical flow method, the type of the object is recognized, and the object whose type is unknown is determined as a sprinkler.

In a feasible implementation, the method further includes:

According to the situation where the trajectory information of the spray and the trajectory information of at least one non-projectile intersect, the source of the spray is determined.

In a feasible implementation manner, the trajectory information includes current trajectory information and backward-extended trajectory information of the current trajectory information.

The above method can quickly locate the source of the sprays, and the staff can prevent new sprays from appearing on the road and reduce the number of sprays by handling the source of the sprays, thereby reducing the probability of safety accidents.

In the second aspect, this application provides a road safety monitoring method, including:

Obtain road monitoring information and determine that there is a spray on the road; obtain the trajectory information of the spray and at least one non-spray on the road according to the road monitoring information; According to the intersection of the trajectory information of the spray and the trajectory information of the at least one non-spray , Determine the source of the thrown objects, and issue an alarm based on the source of the thrown objects.

In a feasible implementation manner, acquiring road monitoring information includes: acquiring road monitoring image information through a video monitoring device on the road, and acquiring road monitoring radar information through a radar device.

In a feasible implementation manner, the acquired trajectory information of the projectile on the road includes: obtaining the first trajectory information of the projectile from the radar information, obtaining the second trajectory information of the projectile from the image information, and synthesizing the first trajectory The information and the second trajectory information obtain the trajectory information of the projectile. Through the above method, the accuracy of the acquired trajectory information can be improved.

In a feasible implementation method, determining the source of the throwing objects according to the crossover of the trajectory information includes:

If the trajectory information of the spray and the trajectory information of the at least one non-spray cross at the first moment, the spray is from at least one non-spray; or

If the trajectory information of the spray and the trajectory information of the at least two non-sprays intersect at the first moment, the non-spray closest to the spray is selected from the at least two sprays as the source of the spray.

At the same time, there is an intersection between the trajectory information of the sprinkler and the non-spray. Through the intersection of the trajectory information, the non-spray closest to the intersection of the trajectory information is the most likely source. Therefore, the source of the sprinkle can be judged more accurately . .

In a feasible implementation manner, the trajectory information includes current trajectory information and backward-extended trajectory information of the current trajectory information. The backward extension of the trajectory information supplements the information not obtained by the monitoring equipment, and the more complete trajectory information can improve the accuracy of finding the source of the projectile through the trajectory information.

In a feasible implementation, the road safety monitoring method also includes:

Judging the attributes of the projectile based on the trajectory information of at least one non-projectile; or,

According to the trajectory information of the thrown object, judge the attribute of the thrown object; or,

According to the trajectory information of the thrown object and the trajectory information of at least one non- thrown object, the attribute of the thrown object is judged.

In a feasible implementation, the road safety monitoring method also includes:

Obtain the image characteristics of the tossing object according to the image information. The image features can assist in judging the attributes of the thrown objects and improve the accuracy of judging the attributes of the thrown objects.

In a feasible implementation manner, the method further includes: judging the attributes of the thrown object according to the image characteristics of the thrown object.

In the third aspect, this application provides a road safety monitoring system, the system includes: a monitoring device, an attribute detection device, and an alarm device;

The detection device is used to obtain road monitoring information and determine that there is a spray on the road; obtain trajectory information of at least one object on the road according to the road monitoring information, and the at least one object includes the spray and/or at least one non-spray;

The attribute detection device is used for judging the attribute of the thrown object according to the trajectory information of at least one object;

The alarm device is used to issue an alarm according to the properties of the thrown object.

The above-mentioned system can quickly locate the properties of the scattered objects, and according to the properties of the scattered objects and the degree of danger, timely safety alarms are issued. The staff can remove the scattered objects from the road in time according to the level of the safety alarms, and reduce the traffic caused by the scattered objects on the road. accident.

In a feasible implementation manner, the system further includes: an attribution judging device;

The attribution judging device is used to determine the source of the thrown object according to the situation where the trajectory information of the thrown object intersects the trajectory information of at least one non- thrown object.

The attribution judging device allows the staff to quickly locate the source of the sprinkles and prevent new sprinkles on the road, reducing the probability of traffic safety accidents.

In a feasible implementation manner, the attribute detection device is used to determine the attribute of the thrown object according to the trajectory information of at least one object, including:

By judging the attributes of the sprays and determining the degree of danger of the sprays, it can help the staff to deal with the road sprays with a high degree of danger in time.

In a feasible implementation manner, the monitoring device includes: video monitoring equipment and radar equipment;

The video monitoring equipment obtains road monitoring image information, and the radar equipment obtains road monitoring radar information.

In a feasible implementation manner, the monitoring device obtains first trajectory information of at least one object according to radar information, obtains second trajectory information of at least one object according to image information, and synthesizes the first trajectory information and the second trajectory information to obtain at least one object Track information.

In a fourth aspect, this application provides a road safety monitoring system, which includes: a monitoring device, a belonging judgment device, and an alarm device;

The attribution judging device is used to determine the source of the thrown object according to the situation where the trajectory information of the thrown object intersects the trajectory information of at least one non-drop object;

The alarm device is used to issue an alarm according to the source of the thrown object.

Through the above-mentioned system, the staff can quickly locate the source of the thrown objects on the road, prevent new thrown objects, and effectively reduce the probability of traffic accidents caused by the thrown objects.

In a feasible implementation manner, the system further includes: an attribute detection device;

The attribute detection device judges the attribute of the thrown object according to the trajectory information of at least one object.

Through the above method, it is possible to quickly and accurately determine the attributes of the sprays, and determine the degree of danger of the sprays according to the attributes of the sprays, which is helpful for the staff to deal with the sprays on the road according to the degree of danger.

In a feasible implementation manner, the monitoring device is used to obtain first trajectory information of at least one object according to radar information, obtain second trajectory information of at least one object according to image information, and synthesize the first trajectory information and the second trajectory information to obtain Track information of at least one object.

In the fifth aspect, this application provides a computer device for road safety monitoring. The computer device includes a processor and a memory, wherein:

Computer instructions are stored in the memory;

The processor executes computer instructions to make the computer device execute the method of the first aspect and its possible implementations.

In a sixth aspect, this application provides a computer device for road safety monitoring. The computer device includes a processor and a memory, wherein:

Computer instructions are stored in the memory;

The processor executes computer instructions to make the computer device execute the method of the second aspect and its possible implementations.

In a seventh aspect, the present application provides a computer-readable storage medium that stores computer instructions. When the computer instructions in the computer-readable storage medium are executed by a computer device, the computer device executes claim 1 13. The method of any one of claims, or the computer equipment to realize the function of the device in the system of the third aspect and its possible implementation manners.

In an eighth aspect, this application provides a computer-readable storage medium that stores computer instructions. When the computer instructions in the computer-readable storage medium are executed by a computer device, the computer device executes claim 14- 21. The method according to any one of the claims, or the computer equipment can realize the function of the device in the system of the fourth aspect and its possible implementation manners.

Description of the drawings

The following will briefly introduce the drawings used in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, no creative work is required. Under the premise of, other drawings can be obtained based on these drawings.

FIG. 1 is a schematic diagram of a road spraying scene related to an embodiment of the application;

Figure 2 is a schematic diagram of a system according to an embodiment of the application;

3A is a schematic diagram of another system according to an embodiment of the application;

FIG. 3B is a schematic diagram of another system according to an embodiment of the application;

4 is a schematic diagram of the functions of the monitoring device according to an embodiment of the application;

FIG. 5 is a flowchart of a processing method of a monitoring device according to an embodiment of the application;

FIG. 6 is a schematic diagram of the function of the attribute detection device according to an embodiment of the application;

FIG. 7 is a flowchart of a processing method of an attribute detection device according to an embodiment of the application;

FIG. 8 is a flowchart of a processing method of an attribution judging apparatus according to an embodiment of the application;

FIG. 9A is a schematic diagram of a scenario for determining the ownership of a thrown object according to an embodiment of the application; FIG.

FIG. 9B is a schematic diagram of another scenario for determining the attribution of sprays according to an embodiment of the application;

FIG. 10 is a schematic diagram of the function of an alarm device according to an embodiment of the application;

FIG. 11 is a schematic diagram of hardware of a computer device according to an embodiment of the application.

detailed description

In the field of video surveillance, the detection of scattered objects on the road has always been an important issue, and the scattered objects are the main source of the scattered objects on the road. On roads with many fast-moving motor vehicles, the sudden appearance of throwing objects often catches drivers off guard. If they can't make correct evasion, it is easy to cause traffic accidents. Figure 1 shows a scene diagram, which includes: two

trucks

106 and 110, a car 102, a sprinkler 104, and a video surveillance device 112. Wherein, the truck 106 carries goods, and the goods are thrown onto the road as a spray 104. At this time, when the car 102 that is driving fast on the road encounters a sudden spray 104, if it is not avoided in time, it is easy to cause a traffic accident, and the spray 104 becomes a major hidden danger to road safety.

To solve this problem: on the one hand, it is necessary to quickly determine the source of the spray 104, cut off the source of the spray, and prevent more sprays from being thrown on the road; on the other hand, quickly determine the type and degree of danger of the spray 104, if it is a spray 104 Low hazard, such as paper, plastic, etc., you can lower the priority and let the staff clean up regularly, but if it is a high-risk spill, such as tires, big rocks, etc., the risk is very high Next, the staff will be asked to deal with it urgently.

In the prior art, the characteristic image is mainly obtained from the video monitoring device 112, and then the sprinkle 104 is identified from the characteristic image. One of the identification technical solutions is to use the average pixel value method by comparing the sprinkle 104 in the characteristic image. The average pixel value and the average pixel value of the road identify the spray 104, but this method can only identify whether there is a spray 104 on the road, and cannot identify the type and degree of danger of the spray 104, nor can it determine the source of the spray 104. Another technical solution is to use the video surveillance equipment 112 to obtain the image information on the road, and use the image recognition method to capture the sprinkles 104, but only the marked sprinkles in the database can be identified, and it is impossible to identify all the sprinkles. The 104 types of objects and their corresponding degree of danger cannot improve the identification and intensive reading of the degree of danger of thrown objects. In addition, it is not very accurate to determine the attribution of the spray 104 by calculating the straight-line distance between the spray 104 at a standstill and its nearest motor vehicle. For example, as shown in Fig. 1, the spray 104 At a standstill, the straight-line distance between the truck 110 and the spray 104 is the shortest, but the spray 104 comes from the truck 106 instead of the truck 104.

Based on the above problems, the present application provides a feasible technical solution. In one embodiment, the system diagram of the solution will be described in detail with reference to FIG. 2. The system 200 includes a monitoring device 202, an attribute detection device 204, an attribution judgment device 206, and an alarm device 208. The monitoring device 202 obtains road information, which may include video equipment and radar equipment. The video information and image information on the road can be obtained through the video equipment, and the radar information on the road can be obtained through the radar equipment. Further, the first trajectory information of at least one object on the road can be obtained from the radar information. The at least one object includes sprays on the road and other objects. The other objects can be collectively referred to as non-sprays and non-sprays, for example, including driving on the road. Of motor vehicles, non-motor vehicles, and pedestrians. In addition, the image information can also obtain the second trajectory information of at least one object, and the trajectory information of the at least one object can be obtained by combining the first trajectory information and the second trajectory information.

The attribute detection device 204 is used to detect the attribute of at least one object, including any one or a combination of size, risk level, type, and the like. Specifically, image characteristics and trajectory information of at least one object are obtained through analysis and processing through image information and trajectory information of at least one object. The image features include: color histogram, color set, color moment, color aggregation vector, color correlation graph, texture feature, SIFT (Scale-invariant feature transform) feature, etc. The trajectory information includes: speed (including size and direction), acceleration, stopover time on the trajectory, etc.

The attribution judging device 206 obtains the trajectory information of the toss and the non-toss, and the trajectory information includes the currently obtained trajectory information and its reverse extension line. Judge the attribution source of the attribution of the attribution of the attribution of the attribution of the attribution of the attribution of the attribution of the attribution to the attribution of the attribution of the attribution of the attribution of the attribution of the attribution of the attribution and the discussion based on the intersection of the trajectory of the attribution of the attribution at the same time (the first moment).

The warning device 208 obtains the attributes of the thrown objects and the source of the thrown objects, and sends out different levels of safety alarms according to the degree of danger of the thrown objects. In a feasible implementation method, the following correspondence can be made: if the degree of danger is high, the safety alarm is It is red; if the degree of danger is medium, the safety alarm is yellow; if the degree of danger is low, the safety alarm is green. Depending on the degree of danger, the staff will react differently. For example, when the degree of danger is high, they will immediately intervene manually to deal with the road spills urgently. If the degree of danger is low, the staff can take care of it regularly. On the other hand, the warning device 208 will report the source of the spillage to the staff. For example, if the spillage comes from a truck on the road, the staff will control the truck, determine the responsibility, and modify the box conditions.

In another embodiment, as shown in FIG. 3A, the system 300 includes a monitoring device 202, an attribute detection device 204, and an alarm device 208. The working principles of the system 300 and the system 200 are similar and will not be repeated here. The monitoring device 202 obtains the monitoring information on the road, and the attribute detection device 204 judges the attributes of the sprinkles based on the monitoring information. The alarm device 208 issues different levels of safety alarms according to the attributes of the sprinkles and the degree of danger of the sprinkles. The source of the material is reported to the staff.

In another embodiment, as shown in FIG. 3B, the system 310 includes a monitoring device 202, an attribution judgment device 206, and an alarm device 208. The working principles of the system 310 and the system 200 are similar, and will not be repeated here. The monitoring device 202 obtains the monitoring information on the road, the attribution judging device 206 judges the source of the scattered objects based on the monitoring information, the alarm device 208 issues a safety alarm according to the source of the scattered objects, and the staff promptly deal with the scattered objects on the road according to the safety alarms. And cut off the source of the sprinkles to prevent additional safety hazards caused by new sprinkles.

Further, the system 200 is introduced in detail. As shown in FIG. 4, the detailed description of the monitoring device 202 is shown. In an embodiment, the monitoring device 202 includes a monitoring device 402 and an information processing module 408. The monitoring device 402 includes a video monitoring device 404 and a radar device 406, the video information on the road is obtained from the video monitoring device 404, and the radar information on the road is obtained from the radar device 406. The information processing module 408 obtains trajectory information 410 and image information 412 of at least one formation according to the video information and radar information, and completes the tossing object identification 414 from the at least one object according to the above information.

FIG. 5 introduces the processing flow of the monitoring device 202 in detail.

According to the process S502, the video information is parsed into video frames to obtain continuous frames of image information.

According to the process S504, through the optical flow method, the difference between the image information of the consecutive frames before and after is compared, the object whose position has changed is identified, which is called set A, and the target detection algorithm based on deep learning is used to identify the object in set A Known types of objects such as motor vehicles, non-motor vehicles, and pedestrians are called set B.

According to the process S506, the objects in the set A are removed from the objects in the set B, and then the redundant objects are the sprinkles.

According to the process S508, the pixel point of at least one object in the image information is converted from the pixel coordinates to the world coordinates to generate the trajectory information of the at least one object, which is called second trajectory information.

According to the process S510, the radar device 406 directly detects and tracks at least one object, and obtains radar information of the at least one object.

According to the process S512, the pixel points of the detected at least one object in the radar information are converted from polar coordinates to world coordinates to generate the trajectory information of the at least one object, the first trajectory information.

The order of execution between steps 502-506 and steps 510-512 is not limited. Steps 502-506 can be executed before steps 510-512, or steps 510-512 can be executed before steps 502-506. It can also be executed at the same time.

According to the process S514, the first trajectory information and the second trajectory information are integrated, and finally the trajectory information of at least one object is generated. The specific method is to first establish a one-to-one correspondence between at least one target in the radar information and at least one target in the image information; then, if the accuracy of the image information is higher at the near end, then the second The trajectory information is used as the trajectory information of at least one object of the target. If at least one object of the target in the image information is tracked and lost, it is compensated by the first trajectory information of the target at least one object in the corresponding radar information, as the trajectory of the target at least one object Information; if the radar information has a high accuracy rate at the remote end, the first trajectory information of the target at least one object in the radar information is used as the trajectory information of the target at least one object.

The obtained image information and trajectory information of at least one object are transferred to the attribute detection device. FIG. 6 introduces the attribute detection device 204 in detail. The attribute detection device 204 includes an attribute judgment module 602 and a feature database 610 of the sprinkler. The attribute judgment 602 of the toss includes the image feature judgment 604 of the toss, the trajectory information judgment 608 of the toss, and the attribute judgment 606 based on the behavior of the surrounding vehicles. The feature database 610 includes an image feature table 614 and a trajectory information table 612. The toss attribute judgment module 602 can determine the attributes of the toss based on the image characteristics, trajectory information, and the behavior of vehicles around the toss, or it can also determine the attributes of the toss based on the image features, trajectory information, and behavior of vehicles around the toss. Any combination can be used to judge the attributes, degree of danger, etc. of the projectile.

Figure 7 explains in detail the flow of the attribute detection device.

According to the process S702, the attribute detection device 204 first obtains the image characteristics from the image information of at least one object, and obtains the trajectory information from the trajectory information. At least one of the objects includes: sprinkles, vehicles and pedestrians around the sprinkles.

According to the process S704, the attribute detection device 204 compares the obtained image feature and trajectory information of the thrown object with the data in the feature database 610. The image feature table 614 and the trajectory information table 612 in the feature database 610 save the marked The attributes of the object, and the corresponding trajectory information and image characteristics.

According to the process S706, when the image feature and/or trajectory information of the projectile is in the feature database 610, the image feature and/or trajectory information of the object can be corresponded, for example, the similarity can be higher than a threshold, then the attributes of the object are returned Judge 602 the attributes of the drop.

According to the process S708, obtain the trajectory information of the vehicles around the spray during a certain period of time before and after the stationary time of the spray, and compare the trajectory information of the vehicle when there is the spray and the trajectory information of the vehicle when there is no such spray. difference between. The definition of the surrounding area mentioned here, in a possible implementation manner, can include the lane where the spatter is located, and the lanes on the left and right sides.

According to the process S710, a threshold is set based on the distance between the surrounding vehicles and the scattered objects to filter out the trajectory information of some invalid surrounding vehicles. For example, in a possible implementation, the lane where the scattered objects are located can be taken as well as the scattered objects Lanes on the left and right sides of the lane. The trajectory information of vehicles exceeding this distance range can be ignored.

According to the process S712, first calculate the similarity of the trajectory information of the surrounding vehicles in the vicinity of the spray and then make a judgment. If the similarity is less than a threshold, it is proved that the surrounding vehicles have appeared due to the presence of the spray. Changes in the trajectory and avoidance behavior indicate that the thrown object is dangerous. If the similarity is greater than this threshold, it proves that the surrounding vehicles have not evaded behavior, and the throwing objects are not dangerous.

According to the process S716, when there is no feature data of the thrown object in the feature database, the image feature and trajectory information of the thrown object are bound with the attribute and the degree of danger of the thrown object, and then updated to the feature database for future When a sprinkler with the same image feature and/or trajectory information appears on the road, the system can directly determine the attribute of the sprinkler, and the degree of danger is high.

According to the process S718, according to the image characteristics, the trajectory information and the behavior characteristics of the surrounding vehicles, the attribute of the judged toss is obtained, and the attributes may include: the size, the degree of danger, and the type of the toss. In one feasible way, you can only rely on the behavior characteristics of surrounding vehicles to determine the attributes of the toss; in another feasible method, you can determine the attributes of the toss based on any two of the image characteristics, trajectory information, and the behavior characteristics of the surrounding vehicles. The properties of the toss.

The graphic information and trajectory information of at least one object acquired by the monitoring device 202 are transmitted to the attribution judging device 206 at the same time. FIG. 8 introduces the processing flow of the attribution judging device 206 in detail. At least one of the objects includes sprays and non-sprays, and non-sprays include motor vehicles, non-motor vehicles, and pedestrians.

According to the process S802, first perform a reverse prediction based on the trajectory information of the projectile to obtain the reverse extended trajectory of the projectile trajectory. As shown in FIG. 9A, the current trajectory 908 of the projectile 912 is the part shown by the solid line, and 904 is the current trajectory The reverse extension line of 908 is represented by a dashed line, and the complete trajectory of the projectile is represented by the solid line part of 908 and the dashed line part of 904. The trajectory information of the sprinkler is composed of the current trajectory information and the reverse extended trajectory information.

According to the process S804, calculate the intersection of the sprinkler and the motor vehicle/non-motor vehicle/pedestrian, as a possible realization form, as shown in Figure 9A, in the figure there is a motor vehicle 910, through its trajectory information, obtain its The movement trajectory is 902, and there is an intersection 906 between the trajectory of the sprinkler 912 at the same time.

According to the process S806, it is judged how many intersections there are at the same time. If there are none, the process jumps directly to the process S814 to output the attribution relationship of the sprinkles. At this time, the attribution relationship is displayed as unknown. If there is an intersection, the process jumps to process S808.

According to the process S808, if there is only one intersection between the trajectory of the spray and the trajectory of the motor vehicle/non-motor vehicle/pedestrian at the same time, as shown in FIG. The process jumps to the process S814, and the attribution relationship of the output spray 912 is the truck 910.

According to the process S808, if there is more than one intersection between the trajectory of the spray and the trajectory of the motor vehicle/non-motor vehicle/pedestrian at the same time, as shown in FIG. 9B, there are three intersection points: 930, 932, and 938. Then the process jumps to process S810.

According to the process S810, the attribute of the thrown object is used to filter out the impossible attribution relationship. As shown in FIG. 9B, there are three non-sprayers: truck 922, truck 920 and bicycle 940, one sprayer 936, one bicycle trajectory 940, two

truck trajectories

940 and 924, and

sprayer trajectories

934 and 928. Three

intersections

932, 930, and 938 are generated at the same time. Therefore, the drop 936 may belong to the bicycle 940, the truck 920, or the truck 922. However, based on the properties of the sprinkler 936, such as its size, it can be judged that the sprinkler 936 cannot come from the bicycle 940. After eliminating the impossible attribution relationship of the sprinkles, jump to the next process S812.

According to the process S812, the motor vehicle/non-motor vehicle/person whose starting point and the intersection point are closest is selected as the attribution target. As shown in Figure 9B, the detected starting point of the spray 936 is the starting point of the current trajectory 934 of the spray 936. The straight line distance from the starting point to the intersection 930 is shorter than the straight line distance to the intersection 938. Therefore, the spray 936 Attribution is truck 922 instead of truck 920.

According to the process S814, according to the judgment result of the previous process, the process S814 outputs the attribution relationship of the sprinkles to the alarm device 208.

Fig. 10 further introduces the warning device 208 in detail. The warning device 208 includes three sub-modules: a hazard display 1002, a belonging vehicle display 1004, and a manual intervention warning 1006. Firstly, judge the danger of the spray based on the properties of the spray, for example: the spray is large, the spray is heavy, and the vehicles around the spray have evasive behavior. If the risk of the spray is high, then the danger display 1002 is dangerous. High sex. When the hazard is displayed, it is necessary to find the source of the spray immediately to prevent more sprays from appearing on the road. Therefore, after the attribution judgment device 206 determines the ownership of the spray, the vehicle display module 1004 is assigned To present the relevant information of the vehicle that the spatter belongs to, such as the license plate of the vehicle, the type of the vehicle, and so on. In this scenario, the manual intervention warning 1006 needs to complete two things. The first is the handling of the spilled objects. When the danger is high, an alarm is required to prompt manual intervention to remove the spilled objects from the road. Then there is the processing of the source of the spray. When the source of the spray is located, an alarm is required to prompt manual intervention, such as allowing the staff to find the motor vehicle/non-motor vehicle/pedestrian that generated the spray in time for processing; or through traffic The system queries the phone number of the driver of the vehicle license plate, and informs the driver to stop and deal with it through the phone to prevent the generation of new spills. On the other hand, if the sprinkles produced cause a traffic accident, this system will help find the responsible party and assist in the determination of liability.

In another embodiment, such as the scenario described in the system 300, if the attribution judging device 206 is not included, the warning device 208 does not have the sub-module of the attribution vehicle display 1004, and the warning device 208 is completed by the hazard display 1002 With the above function, manual intervention alarm 1006 reminds the staff to deal with the spills on the road in time according to the danger of the spilled objects. When the risk is high, the staff needs to deal with it immediately. If the risk is low, you can let The staff handles it regularly.

In another embodiment, such as the scenario described by the system 310, if the attribute detection device 204 is not included, the warning device 208 does not have the risk display 1002 sub-module, and the warning device 208 completes the above by displaying the belonging vehicle Function, manual intervention alarm 1006 cuts off the source of the sprinkles according to the vehicle ownership of the sprinkles to prevent additional road safety hazards caused by the new sprinkles. The methods of manual intervention include: for example, let the staff find the machine that generates the sprinkles in time. Motor vehicles/non-motor vehicles/pedestrians, to deal with; or through the traffic system, check the phone number of the driver of the vehicle license plate, and inform the driver to stop and deal with it through the phone to prevent the generation of new sprays. On the other hand, if the sprinkles produced cause a traffic accident, this system will help find the responsible party and assist in the determination of liability.

The present application also provides a computer device 1100 as shown in FIG. 11, including: a processor 1102, a memory 1104, a communication interface 1106, and a communication bus 1108. The processor 1102 in the computer device 1100 reads a set of computer instructions stored in the memory 1104 to execute the aforementioned road safety monitoring method, and realize the function of any device of any one of the system 200, the system 300, and the system 310.

It is worth noting that, in the embodiments of the present application, the video monitoring equipment included in the monitoring device may be a smart camera with a certain computing capability.

It should be understood that this application does not restrict the division of the functions of the various parts of the above-mentioned device, nor does it restrictively limit the environment in which the above-mentioned device is specifically deployed. In actual applications, appropriate function division adjustments and adaptive deployments can be made according to the computing capabilities of each computer device or specific application requirements. For example, the functions implemented by the information processing module 408 can be completed by the monitoring device 202 or by the attribute detection device 204. The functions implemented by the alarm device 208, the functions implemented by the attribute detection device 204, and the functions implemented by the attribution judgment device 206 may be implemented by a computer device 1100.

In the several embodiments provided in this application, it should be understood that the disclosed system, device, and method can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the modules is only a logical function division, and there may be other divisions during implementation. For example, multiple modules or components may be combined or may be Integrate into another system, or some features can be ignored or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication link may be indirect coupling or communication link through some interfaces, devices or modules, and may be in electrical, mechanical or other forms.

The modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, may be located in one place, or may also be distributed to multiple network modules. Some or all of the modules can be selected according to actual needs to achieve the objectives of the solutions of the embodiments.

In addition, each functional module in each embodiment of the present invention may be integrated into one processing module, or each module may exist alone physically, or two or more modules may be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware, or in the form of hardware plus software functional modules.

It should be understood that the term "and/or" in this article is only an association relationship describing associated objects, which means that there can be three relationships. For example, A and/or B can mean that A alone exists, and both A and B exist. , There are three cases of B alone. In addition, the character "/" in this text generally indicates that the associated objects before and after are in an "or" relationship.

It should be understood that the term "at least one" herein means one or more, and multiple means two or more. At least one similar term has the same meaning.

The above-mentioned integrated module implemented in the form of a software function module may be stored in a computer readable storage medium. The above-mentioned software function module is stored in a storage medium and includes a number of instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium may be a readable non-volatile storage medium, including: mobile hard disk, read-only memory (English: Read-Only Memory, ROM for short), random access memory (English: Random Access Memory, for short) Various media that can store program codes, such as RAM), magnetic disks or optical disks.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: The technical solutions recorded in the foregoing embodiments are modified, or some of the technical features thereof are equivalently replaced; these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.

Claims

A method for road safety monitoring, which is characterized in that it includes:

Obtain road monitoring information, and determine that there is a sprinkle on the road;

Obtain trajectory information of at least one object on the road according to the road monitoring information, where the at least one object includes the thrown object and/or at least one non- thrown object; determine the trajectory information of the at least one object State the properties of the throwaway;

According to the attribute of the thrown object, an alarm is issued.
The method according to claim 1, wherein acquiring the trajectory information of at least one object on the road, and judging the attribute of the thrown object according to the trajectory information of the at least one object comprises:

Obtain the trajectory information of the spray on the road, and determine the attribute of the spray according to the trajectory information of the spray; or, obtain the trajectory information of at least one non-spray on the road, according to the at least one The trajectory information of the non-sprayed object judges the attribute of the sprinkled object; or,

Obtain the trajectory information of the at least one non-spraying object and the trajectory information of the spraying object on the road, and determine the attribute of the spraying object according to the trajectory information of the at least one non-spraying object and the trajectory information of the spraying object.
The method according to claim 1, wherein the obtaining the road monitoring information comprises: obtaining road monitoring image information through a video monitoring device on the road, and obtaining road monitoring radar information through a radar device.
The method according to claim 3, wherein said acquiring trajectory information of at least one object on the road comprises:

Acquire first trajectory information of the at least one object according to the radar information, acquire second trajectory information of the at least one object according to the image information, and synthesize the first trajectory information and the second trajectory information to obtain Describes the trajectory information of at least one object.
The method according to claim 3, further comprising: obtaining image characteristics of the toss according to the image information.
The method according to claim 5, characterized in that the method further comprises: judging the attribute of the thrown object according to the image feature of the thrown object.
The method according to claim 6, wherein when the at least one object includes the thrown object, the method for judging the attribute of the at least one object thrown object specifically comprises:

Comparing the trajectory information of the throwing object with the image characteristics of the throwing object through a feature database, and determining the attribute of the throwing object.
The method according to claim 7, characterized in that, in the case of unsuccessful comparison with the characteristic database, refreshing the trajectory information or image characteristics of the projectile into the characteristic database.
The method according to any one of claims 1-7, wherein the attribute of the thrown object includes the degree of danger of the thrown object.
The method according to claim 9, characterized in that different levels of alarms are issued according to different levels of danger of the thrown objects.
The method according to any one of claims 1-10, wherein determining that there is a spray on the road comprises:

The object in the image information is detected by the optical flow method, the type of the object is recognized, and the object whose type is unknown is determined as the sprinkler.
The method according to any one of claims 2-11, wherein the method further comprises:

Determine the source of the thrown object according to the situation where the trajectory information of the thrown object intersects the trajectory information of the at least one non- thrown object.
The method according to claim 12, wherein the trajectory information includes current trajectory information and backward-extended trajectory information of the current trajectory information.
A method for road safety monitoring, which is characterized in that it includes:

Obtain road monitoring information to determine that there is a spray on the road; obtain the trajectory information of the spray and at least one non-spray on the road according to the road monitoring information; according to the trajectory information of the spray and the at least one An intersection of the trajectory information of a non-sprayed object, and determine the source of the dropped object;

According to the source of the thrown objects, an alarm is issued.
The method according to claim 14, wherein the acquiring the road monitoring information comprises: acquiring road monitoring image information through a video monitoring device on the road, and acquiring road monitoring radar information through a radar device.
The method according to claim 15, wherein the acquiring the trajectory information of the thrown object on the road comprises: acquiring the first trajectory information of the thrown object from the radar information, and obtaining the first trajectory information of the thrown object from the image information Acquire second trajectory information of the spray, and synthesize the first trajectory information and the second trajectory information to obtain the trajectory information of the spray.
The method according to any one of claims 14, wherein determining the source of the thrown object according to a situation where the trajectory information intersects comprises:

If the trajectory information of the toss and the trajectory information of the at least one non-toss cross at the first moment, the toss comes from the at least one non-toss; or

If the trajectory information of the tossing object and the trajectory information of the at least two non-tossing objects cross at the first moment, the non-tossing object closest to the tossing object is selected from the at least two tossing objects as the projected object. State the source of the sprinkles.
The method according to any one of claims 14-17, wherein the trajectory information includes current trajectory information and backward-extended trajectory information of the current trajectory information.
The method according to any one of claims 14-18, wherein the method of road safety monitoring further comprises:

Judging the attributes of the at least one non-spraying object according to the trajectory information of the at least one non-spraying object; or,

Determine the attribute of the thrown object according to the trajectory information of the thrown object; or,

According to the trajectory information of the tossing object and the trajectory information of the at least one non-tossing object, the attribute of the tossing object is determined.
The method according to any one of claims 14-19, wherein the method of road safety monitoring further comprises:

Acquiring the image characteristics of the tossing object according to the image information.
The method according to claim 20, wherein the method further comprises: judging the attribute of the thrown object according to the image characteristics of the thrown object.
A road safety monitoring system, characterized in that the system includes: a monitoring device, an attribute detection device, and an alarm device;

The detection device is used to obtain road monitoring information, and determine that there is a spray on the road; obtain trajectory information of at least one object on the road according to the road monitoring information, and the at least one object includes the spray and/ Or at least one non-spray;

The attribute detection device is used for judging the attribute of the thrown object according to the trajectory information of the at least one object;

The alarm device is used to issue an alarm according to the attribute of the thrown object.
The system according to claim 22, wherein the system further comprises: an attribution judging device;

The attribution judging device is used to determine the source of the thrown object according to the situation where the trajectory information of the thrown object intersects the trajectory information of the at least one non- thrown object.
The system according to claim 22, wherein the attribute detection device is configured to determine the attribute of the thrown object according to the trajectory information of the at least one object, comprising:

Obtain the trajectory information of the spray on the road, and determine the attribute of the spray according to the trajectory information of the spray; or, obtain the trajectory information of at least one non-spray on the road, according to the at least one The trajectory information of the non-sprayed object judges the attribute of the sprinkled object; or,

Obtain the trajectory information of the at least one non-spraying object and the trajectory information of the spraying object on the road, and determine the attribute of the spraying object according to the trajectory information of the at least one non-spraying object and the trajectory information of the spraying object.
The system according to claim 22, wherein the monitoring device comprises: video monitoring equipment and radar equipment;

The video monitoring device obtains road monitoring image information, and the radar device obtains road monitoring radar information.
The monitoring device according to claim 25, wherein the monitoring device obtains the first trajectory information of the at least one object according to the radar information, and obtains the second trajectory information of the at least one object according to the image information Information, combining the first trajectory information and the second trajectory information to obtain the trajectory information of the at least one object.
A road safety monitoring system, characterized in that the system includes: a monitoring device, a belonging judgment device, and an alarm device;

The detection device is used to obtain road monitoring information, and determine that there is a spray on the road; obtain trajectory information of at least one object on the road according to the road monitoring information, and the at least one object includes the spray and/ Or at least one non-spray;

The attribution judging device is configured to determine the source of the thrown object according to the situation where the trajectory information of the thrown object intersects the trajectory information of the at least one non- thrown object;

The alarm device is used to issue an alarm according to the source of the thrown object.
The system according to claim 27, wherein the system further comprises: an attribute detection device;

The attribute detection device judges the attribute of the thrown object according to the trajectory information of the at least one object.
The system according to claim 28, wherein the attribute detection device is configured to determine the attribute of the thrown object according to the trajectory information of the at least one object, comprising:

Obtain the trajectory information of the spray on the road, and determine the attribute of the spray according to the trajectory information of the spray; or, obtain the trajectory information of at least one non-spray on the road, according to the at least one The trajectory information of the non-sprayed object judges the attribute of the sprinkled object; or,

Obtain the trajectory information of the at least one non-spraying object and the trajectory information of the spraying object on the road, and determine the attribute of the spraying object according to the trajectory information of the at least one non-spraying object and the trajectory information of the spraying object.
The system according to claim 27, wherein the monitoring device comprises: video monitoring equipment and radar equipment;

The video monitoring device obtains road monitoring image information, and the radar device obtains road monitoring radar information.
The monitoring device according to claim 30, wherein the monitoring device is configured to obtain the first trajectory information of the at least one object according to the radar information, and obtain the information of the at least one object according to the image information. Second trajectory information, combining the first trajectory information and the second trajectory information to obtain the trajectory information of the at least one object.
A computer equipment for road safety monitoring, characterized in that the computer equipment includes a processor and a memory, wherein:

Computer instructions are stored in the memory;

The processor executes the computer instructions to cause the computer device to execute the method according to any one of the claims 1-13.
A computer equipment for road safety monitoring, characterized in that the computer equipment includes a processor and a memory, wherein:

Computer instructions are stored in the memory;

The processor executes the computer instructions to cause the computer device to execute the method according to any one of the claims 14-21.
A computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and when the computer instructions in the computer-readable storage medium are executed by a computer device, the computer device executes the The method according to any one of claims 1-13, or the computer equipment is made to realize the function of the device in the system according to any one of claims 22-26.
A computer-readable storage medium, characterized in that the computer-readable storage medium stores computer instructions, and when the computer instructions in the computer-readable storage medium are executed by a computer device, the computer device is caused to execute the The method according to any one of claims 14-21, or the computer equipment is made to realize the function of the device in the system according to any one of claims 27-31.