CN113657211A - Early warning method, device and system for abnormal behavior and storage medium - Google Patents
Early warning method, device and system for abnormal behavior and storage medium Download PDFInfo
- Publication number
- CN113657211A CN113657211A CN202110874427.4A CN202110874427A CN113657211A CN 113657211 A CN113657211 A CN 113657211A CN 202110874427 A CN202110874427 A CN 202110874427A CN 113657211 A CN113657211 A CN 113657211A
- Authority
- CN
- China
- Prior art keywords
- abnormal
- early warning
- image
- abnormal behavior
- clustering
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
- G06F18/24133—Distances to prototypes
- G06F18/24137—Distances to cluster centroïds
- G06F18/2414—Smoothing the distance, e.g. radial basis function networks [RBFN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
Abstract
The application discloses an early warning method, a device, a system and a storage medium for abnormal behaviors, wherein the method comprises the following steps: processing the monitoring video data to obtain a first monitoring image; detecting abnormal behaviors of the first monitoring image to obtain a first dangerous hotspot graph, wherein the first dangerous hotspot graph comprises the abnormal behavior type of each pixel position in the first monitoring image and a corresponding dangerous probability value; processing the first dangerous hotspot graph to obtain an abnormal area with abnormal behaviors; and generating early warning information based on the abnormal area. Through the mode, the effectiveness of early warning can be improved.
Description
Technical Field
The application relates to the technical field of artificial intelligence, in particular to an early warning method, device and system for abnormal behaviors and a storage medium.
Background
With the continuous increase of vehicles, the occurrence rate of traffic accidents is also continuously increased, and many traffic accidents occur due to factors such as roads or weather, so that the position and the motion condition of pedestrians/non-motor vehicles are not known and not reacted timely by drivers of motor vehicles, and collision occurs, and therefore how to timely and effectively remind the drivers of abnormal behaviors existing in the current scene becomes an urgent problem to be solved.
Disclosure of Invention
The application provides an early warning method, device and system for abnormal behaviors and a storage medium, and the effectiveness of early warning can be improved.
In order to solve the technical problem, the technical scheme adopted by the application is as follows: provided is an early warning method for abnormal behaviors, which comprises the following steps: processing the monitoring video data to obtain a first monitoring image; detecting abnormal behaviors of the first monitoring image to obtain a first dangerous hotspot graph, wherein the first dangerous hotspot graph comprises the abnormal behavior type of each pixel position in the first monitoring image and a corresponding dangerous probability value; processing the first dangerous hotspot graph to obtain an abnormal area with abnormal behaviors; and generating early warning information based on the abnormal area.
In order to solve the above technical problem, another technical solution adopted by the present application is: the early warning device comprises a memory and a processor which are connected with each other, wherein the memory is used for storing a computer program, and the computer program is used for realizing the early warning method of the abnormal behavior in the technical scheme when being executed by the processor.
In order to solve the above technical problem, another technical solution adopted by the present application is: the utility model provides a monitoring and early warning system, this monitoring and early warning system includes early warning device, speaker and display device, and early warning device is connected with speaker and display device for to speaker and display device send notice information, wherein, early warning device among the above-mentioned technical scheme.
In order to solve the above technical problem, another technical solution adopted by the present application is: a computer-readable storage medium is provided, which is used for storing a computer program, and when the computer program is executed by a processor, the computer program is used for implementing the method for early warning of abnormal behavior in the above technical solution.
Through the scheme, the beneficial effects of the application are that: firstly, processing the acquired monitoring video data to obtain at least one first monitoring image; then, carrying out abnormal behavior detection on the first monitoring image to obtain a first dangerous hot spot diagram, wherein the first dangerous hot spot diagram comprises an abnormal behavior type of each pixel position in the first monitoring image and a dangerous probability value corresponding to the abnormal behavior type; then, the first dangerous heat point diagram is processed to obtain an abnormal area with abnormal behaviors, early warning information related to the abnormal area is generated, whether the abnormal behaviors occur at any position on the specified road section can be monitored, and potential abnormal behaviors are comprehensively evaluated; and the abnormal behaviors are detected based on the first dangerous heat point diagram, so that the potential abnormal behaviors can be effectively detected, the accuracy of early warning of the abnormal behaviors is improved, the probability of traffic accidents is reduced, and the safety of roads is improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts. Wherein:
fig. 1 is a schematic flowchart of an embodiment of an abnormal behavior early warning method provided in the present application;
FIG. 2 is a schematic view of the installation of a camera, a display device and a speaker provided in the present application;
fig. 3 is a schematic flowchart of another embodiment of an early warning method for abnormal behavior provided in the present application;
fig. 4 is a schematic structural diagram of an embodiment of the warning device provided in the present application;
FIG. 5 is a schematic structural diagram of an embodiment of a monitoring and early warning system provided in the present application;
FIG. 6 is a schematic structural diagram of an embodiment of a computer-readable storage medium provided in the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The ghost probe is a traffic accident caused by the fact that pedestrians or non-motor vehicles suddenly appear from a sight blind area of a driver and the driver cannot respond well, and occupies a large proportion in all accident types, so that an early warning scheme is urgently needed to inform the possibility of collision in advance and remind the driver of early deceleration or parking, and accidents of the type are reduced or even avoided.
In order to reduce the occurrence of the ghost probe, some solutions in the related art determine whether a collision occurs according to the distance relationship between a vehicle and a pedestrian, but the following disadvantages exist: (1) the early warning devices are required to be respectively installed on the mobile phone of the pedestrian and the vehicle, so that the early warning device is complicated, is not beneficial to popularization, and relates to the maintenance of a remote server, and the traffic management burden is increased; (2) for pedestrians or people driving non-motor vehicles, when the ghost probe is taken, the prompt of the short message of the mobile phone can completely lose the early warning function due to the coverage of surrounding noise or the mute of the mobile phone; (3) for a driver of a motor vehicle, when the early warning system judges the distance, the distance between a pedestrian and a vehicle is continuously reduced, and the driver sends early warning information after finishing the judgment, so that the driver has difficulty in quickly making reactions such as deceleration and the like in a short time, the pedestrian or the non-motor vehicle can be collided, and the effectiveness of early warning is actually greatly reduced by the time delay; (4) only the parameter of distance is used as the basis for early warning, so that the potential abnormal behavior cannot be comprehensively evaluated, the method is not suitable for multi-target environments, and the false alarm rate is high; (5) the application scene is limited to the zebra crossing, which is randomly inconsistent with the occurrence place of the ghost probe, and the ghost probe cannot be really and effectively prevented.
In view of the problems in the related art, the application provides a scheme for early warning the abnormal behaviors of pedestrians and non-motor vehicles, can timely and effectively detect the potential ghost probe behaviors in the current monitoring scene, can simultaneously monitor all types of ghost probes in the specified road section, and can timely remind pedestrians and drivers in an effective mode, so that traffic accidents caused by the ghost probes are greatly reduced.
Referring to fig. 1, fig. 1 is a schematic flow chart of an embodiment of an abnormal behavior early warning method provided in the present application, the method including:
step 11: and processing the monitoring video data to obtain a first monitoring image.
Acquiring a real-time video stream of a current monitoring road section by a monitoring camera arranged above a certain specific road section or all road sections; in consideration of the sporadic nature of the ghost probe phenomenon, the real-time video stream acquisition can be carried out on the current monitoring road section, namely, the monitoring video data are sampled from the real-time video stream; and then preprocessing (such as scaling, denoising, clipping or geometric transformation) the sampled monitoring video data to obtain a first monitoring image.
Step 12: and carrying out abnormal behavior detection on the first monitoring image to obtain a first dangerous heat point diagram.
After the first monitoring image is obtained, the first monitoring image in the first monitoring image can be input into a pre-trained abnormal behavior detection network for identification, so that the abnormal behavior detection network processes the first monitoring image to generate a first dangerous hot spot diagram, the first dangerous hot spot diagram includes an abnormal behavior type of each pixel position in the first monitoring image and a dangerous probability value corresponding to the abnormal behavior type, and the dangerous probability value is the probability of occurrence of the abnormal behavior.
Further, the abnormal behavior detection network may be a network having a target detection function, such as a yolo (young Only Look one) network, a Long Short-Term Memory (LSTM) network, or a Convolutional Neural Network (CNN), the size of the first dangerous hotspot graph is the same as that of the monitored image, and the abnormal behavior may be a "ghost probe" including 5 abnormal behavior types: (1) a pedestrian or a non-motor vehicle suddenly runs out of the head of the motor vehicle; (2) pedestrians or non-motor vehicles run the red light; (3) a pedestrian or a non-motor vehicle suddenly rushes out of the intersection; (4) crossing the guardrail; (5) the young child suddenly rushes to the road. For example, taking the size of the first monitored image as M1 × N1 as an example, the size of the first dangerous heat map is M1 × N1, the value of each position in the first dangerous heat map is the type of abnormal behavior occurring at the corresponding position in the first monitored image and the dangerous probability value corresponding to the type, for example, the abnormal behavior type corresponding to the position [ a, b ] is the first abnormal behavior type, and the dangerous probability value is P1, 1 ≦ a ≦ M1, and 1 ≦ b ≦ N1.
Step 13: and processing the first dangerous hot spot diagram to obtain an abnormal area with abnormal behaviors.
The output of the abnormal behavior detection network is a hot spot diagram of the risk factors in the current monitoring scene, and the first dangerous hot spot diagram can be further post-processed so as to extract an area (namely an abnormal area) which is most likely to have abnormal behaviors in the current monitoring scene; specifically, the first dangerous heat point map may be clustered, an obtained clustering center is a high-risk abnormal region, coordinates of the abnormal region are calculated according to the calculated clustering center, and an image corresponding to the abnormal region may be captured; or directly acquiring the position with the maximum danger probability value in the first dangerous hotspot graph, and taking the area with the position as the center as an abnormal area, wherein the size of the abnormal area can be preset, and the shape of the abnormal area can be regular shapes such as rectangle and circle, or the shape of the abnormal area can be set to be irregular according to a specific application scene.
Step 14: and generating early warning information based on the abnormal area.
After the abnormal area is acquired, early warning information can be generated, the early warning information comprises warning information, abnormal behavior types or video data of the abnormal area, and first notification information can be sent to a loudspeaker, so that the loudspeaker plays preset warning information, and the warning information can be a whistling sound, a buzzer sound or a preset voice warning sound, but is not limited to the warning information, and the content or the sound size of the warning information can be adjusted according to specific application requirements; or sending second notification information to the display device to enable the display device to display the abnormal behavior type, the warning words or the videos of the abnormal areas, so that the driver of the motor vehicle is reminded of the current abnormal behavior and needs to decelerate or stop, and the effect of reminding the driver of pedestrians or non-motor vehicles is achieved.
In a specific embodiment, as shown in fig. 2, one equipment placing pole 21 is installed at every N meters (for example, 100 meters) on both sides of the road, the cameras 22 are installed on the cross arms (not shown) of the equipment placing pole 21, and one or more cameras 22 can be installed according to the needs and conditions; the display device 23 is installed on the same equipment setting pole 21, and the speaker 24 may or may not be installed on the same equipment setting pole 21 according to conditions.
In order to ensure the full scene monitoring, the installation height of the camera 22 and the interval between the two equipment installation rods 21 are controlled, so that the monitoring view field of the camera 22 can cover the road condition within N meters on one side of the road; and in order to avoid the influence of the shielding of the vehicle on the monitoring, a point position of the equipment placing rod 21 can be arranged on the other side of the road so as to enlarge the view range, and the point position can be opposite to or dislocated with the point position on the opposite side (as shown in fig. 2). Since the abnormal behavior of pedestrians or non-motor vehicles can be monitored only on the road sections within the monitoring field of view of the camera 22, if the "ghost probe" behavior of the whole road section monitoring is to be realized, points can be added at every N meters on all the road sections.
When abnormal behaviors in the current monitoring scene are judged, the speaker 24 is adopted to broadcast warning information to warn pedestrians to return and prompt passing vehicles to pay attention to avoidance; moreover, the warning words can be output on the display device 23, and the video stream of the road section with the ghost probe can be played in real time. For example, as shown in fig. 2, the pedestrian 25 is present at the front side of the vehicle 26a, and the speaker 24 can play voice messages such as "pedestrian please return" or "vehicle dodge" to remind the pedestrian 25 and the vehicles 26a-26b to prevent traffic accidents; the warning words displayed by the display device 23 include the type of the ghost probe, which is the first abnormal behavior type, that is, the pedestrian or the non-motor vehicle suddenly runs out of the head of the motor vehicle.
The embodiment provides a method for early warning abnormal behaviors of pedestrians/non-motor vehicles, which can be used for early warning and covering ghost probes in all road sections, namely monitoring ghost probe behaviors at any position on an appointed road section, evaluating potential abnormal behaviors in all directions and reducing the probability of danger; the abnormal behaviors are detected based on the dangerous heat point diagram, so that the potential abnormal behaviors can be effectively detected, the accuracy of early warning on the abnormal behaviors is improved, and the effectiveness of early warning is improved; in addition, voice reminding and picture reminding are combined, a more direct and more effective reminding mode is used for early warning pedestrians and drivers, the effectiveness of early warning is improved, the probability that the drivers or the pedestrians cannot receive early warning information is reduced, and the safety of roads is further improved.
Referring to fig. 3, fig. 3 is a schematic flow chart of another embodiment of the method for warning of abnormal behavior according to the present application, the method includes:
step 31: and processing the monitoring video data to obtain a first monitoring image.
Determining a second monitoring image in the monitoring video data, wherein the second monitoring image can be obtained by sampling the monitoring video data at intervals, or the second monitoring image is an image of each frame in the monitoring video data, or the monitoring video data is detected first, and the image with motor vehicles and pedestrians/non-motor vehicles is used as the second monitoring image; firstly, zooming the second monitoring image to obtain a zoomed image; then clipping the zoomed image to obtain a sampling image; and then, carrying out mean value removal and normalization processing on the sampling image to obtain a first monitoring image. It is understood that the prior art means for performing the de-averaging and normalization are not described herein.
Further, it may be determined whether the size of the zoomed image is larger than a preset size; and if the size of the scaled image is larger than the preset size, cutting the scaled image along the width direction and/or the length direction of the scaled image so as to enable the size of the sampling image to be equal to the preset size.
In a specific embodiment, the second monitoring image obtained by sampling is scaled in an equal proportion, so that a scaled image with the image size of m × k is obtained, m is a preset positive integer, m is less than or equal to k, and m and k are the number of pixel points; cutting the zoomed image to obtain m pixels in the central area of the zoomed image as a sampling image; and carrying out mean value removal and normalization processing on the sampling image to obtain a final first monitoring image for inputting the abnormal behavior detection network.
In other embodiments, the second monitoring image may also be scaled non-equally to generate a sampled image; and then, carrying out mean value removal and normalization processing on the sampling image to obtain a first monitoring image.
Step 32: and acquiring a data set, training the abnormal behavior detection network by adopting a training set, and verifying by adopting a verification set until a preset training termination condition is met to obtain the trained abnormal behavior detection network.
The data set comprises a training set and a verification set, the training set and the verification set comprise a plurality of training test images and label hot spot diagrams corresponding to the training test images, and the label hot spot diagrams are generated by setting danger probability values corresponding to abnormal behavior types for each position based on Gaussian distribution.
Further, a data set with abnormal behaviors is prepared, the data set comprises training test images of various abnormal behavior types and dangerous scenes, an abnormal area in each training test image can be manually selected, the center point of the abnormal area is used as the center point of Gaussian distribution, and corresponding danger probability values are set for each training test image according to the Gaussian distribution to obtain a label heat point diagram. Then, the training set is used for training the abnormal behavior detection network, and finally, the network parameter with the optimal performance on the verification set is selected as the network parameter of the abnormal behavior detection network.
It is understood that a common loss function can be used to calculate a loss value (denoted as a current loss value) between the training test image and the label hotspot graph corresponding to the training test image; the preset training stop condition may include: the loss value is converged, namely the difference value between the last loss value and the current loss value is smaller than a set value; judging whether the current loss value is smaller than a preset loss value, wherein the preset loss value is a preset loss threshold value; training times reach a set value (for example: 10000 times of training); or the accuracy obtained when the verification set is used for testing is greater than the set accuracy, and the like.
Step 33: and carrying out abnormal behavior detection on the first monitoring image to obtain a first dangerous heat point diagram.
The first monitoring image is input into an abnormal behavior detection network, the abnormal behavior detection network outputs a dangerous heat point diagram (namely a first dangerous heat point diagram) with the same size as the input image through calculation, and the value of each position in the first dangerous heat point diagram represents a dangerous probability value corresponding to the abnormal behavior type of the position.
Step 34: and clustering the first dangerous hot spot graph to obtain a clustering result.
Clustering the first dangerous hot spot graph by adopting a K-Means (K-Means) clustering method to obtain a clustering result, wherein the clustering result comprises at least one clustering cluster, and the clustering center of the clustering cluster comprises a center position and a dangerous probability value corresponding to the center position; specifically, a cluster includes at least one sample, data corresponding to each sample includes a position (i.e., an abscissa and an ordinate) and a corresponding risk probability value, a center position includes a center abscissa and a center ordinate, the center abscissa is a value obtained by averaging abscissas of all samples in the cluster, the center ordinate is a value obtained by averaging ordinates of all samples in the cluster, and the risk probability value corresponding to the center position is a value obtained by averaging risk probability values of all samples in the cluster.
In a specific embodiment, assume that the width of the first dangerous hotspot graph is ow, the height is oh, the upper left corner is the origin of coordinates, and the coordinates (m) arei,ni) Has a value of si，siRepresenting the danger probability value output by the abnormal behavior detection network, the first danger hotspot graph may be represented as a set of 3-dimensional data of size n (n ═ ow × oh), as follows:
wherein the content of the first and second substances,
and the data corresponds to the ith sample in the first dangerous heat point diagram.
For the first dangerous heat point diagram defined above, segmenting the first dangerous heat point diagram by using a K-means clustering method; specifically, for a given first dangerous heat point diagram, the sample set is divided into K cluster clusters according to the distance between each sample in the first dangerous heat point diagram, and the cluster clusters are assumed to be divided into C ═ C (C1,C2,…,CK)，C＝(m,n,s)TThe K clusters are obtained by minimizing the square error:
wherein E is the square error, μiIs a cluster CiMean vector of (u)i＝[μim,μin,μis]，μiThe calculation formula of (a) is as follows:
it will be appreciated that other clustering methods than K-means clustering may be used, such as: hierarchical clustering or density-based clustering methods, etc.
Step 35: and screening the clustering result to obtain a screening result.
Judging whether the danger probability value of the clustering center of the clustering cluster is greater than a preset danger threshold value or not; if the danger probability value of the cluster center is larger than a preset danger threshold value, putting the cluster into a screening result; and if the danger probability value of the cluster center is less than or equal to a preset danger threshold value, no processing is performed.
In a specific embodiment, when the optimal K cluster centers C ═ are obtained (C)1,C2,…,CK) Then, substituting the outputted K clustering centers into the following formula:
wherein the content of the first and second substances,
for a preset hazard threshold, s is the hazard probability value of the cluster center, δ (C)i) And judging whether the danger probability value of the clustering center of the ith clustering cluster is greater than a preset danger threshold value.
Through the above processing, the clustering center C ═ of Q abnormal regions at high risk can be finally obtained (C ═ C)1,C2,…,CQ)。
Step 36: based on the screening results, an abnormal region is determined.
The abnormal region comprises a plurality of sub-regions, and the sub-regions (namely regions where abnormal behaviors occur) are constructed based on the central position, wherein the sub-regions are regions taking the central position as the center; specifically, the sub-region is a rectangular region, and a region with a center position as a center and a set size is used as the sub-region; for example, suppose a subregion is denoted as D ═ (D)1,D2,…,DQ) Each sub-region having a width wdHeight of each subregion is hdThen the definition of the sub-regions is as follows:
wherein D isjIs the jth sub-region, (m)j,nj) Is the center position of the jth sub-region.
Step 37: sending first notification information to a loudspeaker, and controlling the loudspeaker to play warning information; and sending second notification information to the display device, and controlling the display device to play video data and display the abnormal behavior type.
The early warning information comprises warning information, abnormal behavior types and video data of abnormal areas, first notification information and second notification information can be generated, and the first notification information is sent to the loudspeaker so that the loudspeaker can play the warning information; and simultaneously sending second notification information to the display device so that the display device plays video data, displays abnormal behavior types or displays warning words.
The early warning scheme provided by the embodiment has the following advantages:
(1) the ghost probe early warning coverage of the whole road section can monitor the ghost probe behavior at any position on the specified road section.
The ghost probe early warning system in the related technology is limited to specific traffic areas such as zebra stripes, is not consistent with the site randomness of ghost probe behaviors, mostly needs to be respectively provided with early warning devices on a mobile phone of a pedestrian and a vehicle, is complex, is not beneficial to popularization, and also relates to the maintenance of a remote server and increases traffic management burden. In the scheme provided by the embodiment, the camera, the loudspeaker and the display device are combined into a set of monitoring and early warning system and are installed on two sides of the road at intervals, so that the whole road section monitoring and early warning of the specified road section can be realized, and only the specified road section is needed to be installed, so that the popularization and the maintenance are facilitated.
(2) The pedestrian and the driver are warned in a more direct and effective prompting mode.
In the related technology, early warning information is mostly sent to pedestrians or drivers through multimedia terminals such as mobile phones, and due to the time delay of the mode and the terminal reminding mode, the early warning information cannot be obtained by the pedestrians or the drivers at the first time, so that the effectiveness of receiving the early warning information is greatly reduced. In the scheme provided by the embodiment, the loudspeaker is adopted to broadcast the voice warning, the display device is used to display the early warning information, and the video of the abnormal area is played in real time, so that vehicles and pedestrians on the whole road section can acquire the early warning information at the first time, the early warning effectiveness is greatly improved, and the possibility of accidents is reduced.
(3) The abnormal behaviors are detected based on the full-graph analysis, and potential abnormal regions can be detected more effectively.
The related technology usually only depends on the distance and the speed calculated by the radar for early warning, is not suitable for multi-target complex environment, and has higher false alarm rate; for the driver of the motor vehicle, when the early warning system judges the distance, the distance between the pedestrian and the vehicle is continuously reduced, the judgment is finished, and then the early warning information is sent to the driver, so that the driver has difficulty in quickly making reactions such as speed reduction and the like in a short time, the pedestrian or the non-motor vehicle is easy to collide, and the time delay actually greatly reduces the effectiveness of early warning. In the scheme provided by the embodiment, the real-time video frame images are input into the abnormal behavior detection network, the whole graph information is integrated, the danger probability value of the potential danger condition in the current monitoring scene is predicted, and the output dangerous heat point graph is clustered, so that the potential abnormal danger phenomena in a plurality of areas can be early warned and displayed, the safety of the road is improved, and the early warning of the ghost probe phenomenon is more efficient.
Referring to fig. 4, fig. 4 is a schematic structural diagram of an embodiment of the warning apparatus provided in the present application, in which the warning apparatus 40 includes a memory 41 and a processor 42 that are connected to each other, the memory 41 is used for storing a computer program, and the computer program is used for implementing the warning method for abnormal behavior in the foregoing embodiment when being executed by the processor 42.
Referring to fig. 5, fig. 5 is a schematic structural diagram of an embodiment of a monitoring and forewarning system provided in the present application, where the monitoring and forewarning system 50 includes a forewarning device 51, a speaker 52, and a display device 53, and the forewarning device 51 is connected to the speaker 52 and the display device 53 and is configured to send notification information to the speaker 52 and the display device 53, where the forewarning device 51 is the forewarning device in the above embodiment.
The scheme provided by the embodiment can be applied to the field of video monitoring in artificial intelligence, in particular to the field of vehicle early warning of ghost probes; the camera and the display device are combined into a set of monitoring and early warning system, and point locations are installed at two sides of a road at intervals, so that the whole-section monitoring and early warning of a specified road section can be realized; moreover, the voice warning is broadcasted by adopting the loudspeaker, the early warning information is displayed by the display device, and the video of the abnormal area is played in real time, so that the early warning information can be obtained by vehicles and pedestrians on the whole road section at the first time, the early warning effectiveness is greatly improved, and the possibility of accidents is greatly reduced; in addition, real-time video frame images are input into the abnormal behavior detection network, the whole image information is integrated, the potential danger probability value in the current monitoring scene is predicted, the output dangerous heat point images are clustered, the potential abnormal danger phenomena in a plurality of areas can be pre-warned and displayed, the effect of killing dangers in the bud stage is achieved, and the pre-warning efficiency for the ghost probe phenomenon is higher.
Referring to fig. 6, fig. 6 is a schematic structural diagram of an embodiment of a computer-readable storage medium 60 provided in the present application, where the computer-readable storage medium is used for storing a computer program 61, and when the computer program 61 is executed by a processor, the computer program is used for implementing the method for early warning of abnormal behavior in the foregoing embodiment.
The computer-readable storage medium 60 may be a server, a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and various media capable of storing program codes.
In the several embodiments provided in the present application, it should be understood that the disclosed method and apparatus may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, a division of modules or units is merely a logical division, and an actual implementation may have another division, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed.
Units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The above description is only an example of the present application and is not intended to limit the scope of the present application, and all modifications of equivalent structures and equivalent processes, which are made by the contents of the specification and the drawings, or which are directly or indirectly applied to other related technical fields, are intended to be included within the scope of the present application.
Claims (12)
1. An early warning method for abnormal behaviors is characterized by comprising the following steps:
processing the monitoring video data to obtain a first monitoring image;
detecting abnormal behaviors of the first monitoring image to obtain a first dangerous hotspot graph, wherein the first dangerous hotspot graph comprises the abnormal behavior type of each pixel position in the first monitoring image and a corresponding dangerous probability value;
processing the first dangerous hot spot diagram to obtain an abnormal area with abnormal behaviors;
and generating early warning information based on the abnormal area.
2. The method for warning of abnormal behavior according to claim 1, wherein the step of processing the first dangerous heat point map to obtain an abnormal region with abnormal behavior comprises:
clustering the first dangerous hot spot graph to obtain a clustering result;
screening the clustering result to obtain a screening result;
and determining the abnormal region based on the screening result.
3. The method for early warning of abnormal behavior according to claim 2, wherein the size of the first dangerous hot spot map is the same as the size of the monitoring image, and the step of performing clustering processing on the first dangerous hot spot map to obtain a clustering result comprises:
clustering the first dangerous hot spot graph by adopting a K-means clustering method to obtain a clustering result;
the clustering result comprises at least one clustering cluster, and the clustering center of the clustering cluster comprises a center position and a danger probability value corresponding to the center position.
4. The method of claim 3, wherein the abnormal region comprises a plurality of sub-regions, and the step of determining the abnormal region based on the screening result comprises:
constructing the sub-region based on the central location;
wherein the sub-region is a region centered on the center position.
5. The method for early warning of abnormal behavior according to claim 3, wherein the step of screening the clustering result to obtain a screening result comprises:
judging whether the danger probability value of the clustering center of the clustering cluster is larger than a preset danger threshold value or not;
and if so, putting the cluster into the screening result.
6. The method of claim 1, wherein the warning information includes warning information, the abnormal behavior type, and video data of the abnormal region, and the step of generating the warning information based on the abnormal region includes:
sending first notification information to a loudspeaker, and controlling the loudspeaker to play warning information;
and sending second notification information to a display device, and controlling the display device to play the video data and display the abnormal behavior type.
7. The method for warning of abnormal behavior according to claim 1, wherein before the step of detecting abnormal behavior of the first monitoring image, the method further comprises:
acquiring a data set, wherein the data set comprises a training set and a verification set;
training the abnormal behavior detection network by adopting the training set, and verifying by adopting the verification set until a preset training termination condition is met to obtain a trained abnormal behavior detection network;
the training set and the verification set comprise a plurality of training test images and label hot spot maps corresponding to the training test images, and the label hot spot maps are generated by setting danger probability values corresponding to abnormal behavior types for each position based on Gaussian distribution.
8. The method for early warning of abnormal behavior as claimed in claim 1, wherein the step of processing the monitoring video data to obtain the first monitoring image comprises:
determining a second monitoring image in the monitoring video data;
zooming the second monitoring image to obtain a zoomed image;
clipping the zoomed image to obtain a sampling image;
and carrying out mean value removal and normalization processing on the sampling image to obtain the first monitoring image.
9. The method of claim 8, wherein the step of cropping the scaled image to obtain a sampled image comprises:
judging whether the size of the zoomed image is larger than a preset size or not;
if yes, the scaling image is cut along the width direction and/or the length direction of the scaling image, so that the size of the sampling image is equal to the preset size.
10. An early warning apparatus, comprising a memory and a processor connected to each other, wherein the memory is used for storing a computer program, and the computer program is used for implementing the early warning method of abnormal behavior according to any one of claims 1 to 9 when the computer program is executed by the processor.
11. A monitoring and early warning system is characterized by comprising an early warning device, a loudspeaker and a display device, wherein the early warning device is connected with the loudspeaker and the display device and used for sending notification information to the loudspeaker and the display device, and the early warning device is the early warning device in claim 10.
12. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, is adapted to implement the method of warning of abnormal behavior of any one of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110874427.4A CN113657211A (en) | 2021-07-30 | 2021-07-30 | Early warning method, device and system for abnormal behavior and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110874427.4A CN113657211A (en) | 2021-07-30 | 2021-07-30 | Early warning method, device and system for abnormal behavior and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113657211A true CN113657211A (en) | 2021-11-16 |
Family
ID=78478183
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110874427.4A Pending CN113657211A (en) | 2021-07-30 | 2021-07-30 | Early warning method, device and system for abnormal behavior and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113657211A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114648879A (en) * | 2022-05-18 | 2022-06-21 | 浙江大华技术股份有限公司 | Abnormal area monitoring method and device based on dangerous goods and storage medium |
| CN114724246A (en) * | 2022-04-11 | 2022-07-08 | 中国人民解放军东部战区总医院 | Dangerous behavior identification method and device |
| CN114912719A (en) * | 2022-07-15 | 2022-08-16 | 北京航空航天大学 | Heterogeneous traffic individual trajectory collaborative prediction method based on graph neural network |
| CN116110006A (en) * | 2023-04-13 | 2023-05-12 | 武汉商学院 | Scenic spot tourist abnormal behavior identification method for intelligent tourism system |
-
2021
- 2021-07-30 CN CN202110874427.4A patent/CN113657211A/en active Pending
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114724246A (en) * | 2022-04-11 | 2022-07-08 | 中国人民解放军东部战区总医院 | Dangerous behavior identification method and device |
| CN114724246B (en) * | 2022-04-11 | 2024-01-30 | 中国人民解放军东部战区总医院 | Dangerous behavior identification method and device |
| CN114648879A (en) * | 2022-05-18 | 2022-06-21 | 浙江大华技术股份有限公司 | Abnormal area monitoring method and device based on dangerous goods and storage medium |
| CN114648879B (en) * | 2022-05-18 | 2022-08-16 | 浙江大华技术股份有限公司 | Abnormal area monitoring method and device based on dangerous goods and storage medium |
| CN114912719A (en) * | 2022-07-15 | 2022-08-16 | 北京航空航天大学 | Heterogeneous traffic individual trajectory collaborative prediction method based on graph neural network |
| CN116110006A (en) * | 2023-04-13 | 2023-05-12 | 武汉商学院 | Scenic spot tourist abnormal behavior identification method for intelligent tourism system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113657211A (en) | Early warning method, device and system for abnormal behavior and storage medium | |
| US11380105B2 (en) | Identification and classification of traffic conflicts | |
| CN110164130B (en) | Traffic incident detection method, device, equipment and storage medium | |
| CN112885144B (en) | Early warning method and system for vehicle crash event in construction operation area | |
| CN109191829B (en) | road safety monitoring method and system, and computer readable storage medium | |
| CN109934075A (en) | Accident detection method, apparatus, system and electronic equipment | |
| US20220139090A1 (en) | Systems and methods for object monitoring | |
| CN113676702B (en) | Video stream-based target tracking and monitoring method, system, device and storage medium | |
| JP2023527265A (en) | Method and device for detecting traffic abnormality, electronic device, storage medium and computer program | |
| CN112766069A (en) | Vehicle illegal parking detection method and device based on deep learning and electronic equipment | |
| CN109996212A (en) | Vehicle blind zone information acquisition method, car-mounted terminal and storage equipment | |
| CN110782677A (en) | Illegal vehicle snapshot warning method and device | |
| CN111427063B (en) | Mobile device traffic control method, device, equipment, system and medium | |
| CN112084892B (en) | Road abnormal event detection management device and method thereof | |
| CN113658427A (en) | Road condition monitoring method, system and equipment based on vision and radar | |
| CN111746526B (en) | Early warning method, device and equipment for rear unmanned vehicle and vehicle | |
| CN116311946B (en) | Method, system, terminal and storage medium for displaying real-time traffic situation | |
| CN112633228A (en) | Parking detection method, device, equipment and storage medium | |
| CN112906664A (en) | Neural network-based method and system for detecting abnormal events in tunnel | |
| CN117169873A (en) | Beyond-view bird's eye view sensing method and device, target sensing device, equipment and medium | |
| EP0403193A2 (en) | Method and apparatus for traffic monitoring | |
| WO2020210960A1 (en) | Method and system for reconstructing digital panorama of traffic route | |
| CN115270400A (en) | Test scene construction method for automatic driving simulation and readable storage medium | |
| CN115713745A (en) | Obstacle detection method, electronic device, and storage medium | |
| CN111932889B (en) | Traffic dispersion system based on vehicle-road cooperation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |