CN112200044A - Abnormal behavior detection method and device and electronic equipment - Google Patents

Abnormal behavior detection method and device and electronic equipment Download PDF

Info

Publication number
CN112200044A
CN112200044A CN202011058838.8A CN202011058838A CN112200044A CN 112200044 A CN112200044 A CN 112200044A CN 202011058838 A CN202011058838 A CN 202011058838A CN 112200044 A CN112200044 A CN 112200044A
Authority
CN
China
Prior art keywords
vehicle
pedestrian
area
module
key point
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202011058838.8A
Other languages
Chinese (zh)
Other versions
CN112200044B (en
Inventor
蔚勇
刘树明
朱小龙
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Navinfo Co Ltd
Original Assignee
Navinfo Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Navinfo Co Ltd filed Critical Navinfo Co Ltd
Priority to CN202011058838.8A priority Critical patent/CN112200044B/en
Priority claimed from CN202011058838.8A external-priority patent/CN112200044B/en
Publication of CN112200044A publication Critical patent/CN112200044A/en
Application granted granted Critical
Publication of CN112200044B publication Critical patent/CN112200044B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G06V20/54—Surveillance or monitoring of activities, e.g. for recognising suspicious objects of traffic, e.g. cars on the road, trains or boats
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/08—Detecting or categorising vehicles

Abstract

The application provides an abnormal behavior detection method, an abnormal behavior detection device and electronic equipment, wherein the method comprises the following steps: acquiring data to be identified, wherein a target in the data to be identified comprises a vehicle; determining an identification frame of the vehicle through an identification frame detection module of a key point detection network model, and detecting key points of the vehicle in the identification frame of the vehicle through the key point detection module of the key point detection network model; carrying out key point tracking on key points of the vehicle, and determining a motion track of the vehicle; and determining whether the vehicle has abnormal behaviors according to the motion trail of the vehicle and the area where the vehicle is located. The method is suitable for various behavior scenes to detect various abnormal behaviors, can adapt to different surrounding environments, and is good in robustness of abnormal behavior detection.

Description

Abnormal behavior detection method and device and electronic equipment
Technical Field
The present disclosure relates to the field of abnormal behavior detection technologies, and in particular, to an abnormal behavior detection method and apparatus, and an electronic device.
Background
At present, intelligent video monitoring becomes one of the very active hot techniques in the field of artificial intelligence, and is an important application of the computer vision technology in the field of security protection. Traffic flow peaks or people flow peaks often exist in public areas such as traffic intersections, railway stations, subway stations and squares, and are particularly important for detecting abnormal behaviors of vehicles and pedestrians, avoiding traffic accidents and timely discovering traffic accidents.
In the prior art, for detecting abnormal traffic behaviors, a conventional algorithm is usually adopted to detect abnormal behaviors, which mainly includes traffic data acquisition, then traffic data is detected by a background difference method to determine a moving vehicle, and then the moving vehicle is compared with a predefined behavior to determine whether the behavior of the moving vehicle is the predefined behavior, so that the abnormal behavior of the vehicle is detected.
However, in the prior art, the detection of abnormal behaviors by using a conventional algorithm is limited to various behavior scenes, and the robustness of the detection of abnormal behaviors is poor.
Disclosure of Invention
The application provides an abnormal behavior detection method, an abnormal behavior detection device and electronic equipment, so that abnormal behavior detection is achieved, the method and the device are suitable for various behavior scenes, and robustness of abnormal behavior detection is good.
In a first aspect, an embodiment of the present application provides an abnormal behavior detection method, including:
acquiring data to be identified, wherein a target in the data to be identified comprises a vehicle; determining an identification frame of the vehicle through an identification frame detection module of the key point detection network model, and detecting key points of the vehicle in the identification frame of the vehicle by using the key point detection module of the key point detection network model; carrying out key point tracking on key points of the vehicle to determine a motion track of the vehicle; and determining whether the vehicle has abnormal behaviors according to the motion trail of the vehicle and the area where the vehicle is located.
In the embodiment of the application, the key point detection is carried out on the vehicle in the data to be identified through the key point detection network model, the motion track of the vehicle is determined according to the key points of the vehicle, and then the motion track of the vehicle and the area where the vehicle is located are utilized to determine whether abnormal behaviors exist in the vehicle. Compared with the prior art that the background difference method is adopted to detect the traffic data to determine the moving vehicle and then the abnormal behavior is detected by comparing the detected moving vehicle with the predefined behavior, the accuracy and flexibility of abnormal behavior detection can be improved.
In one possible implementation, determining whether the vehicle has abnormal behavior according to the motion track of the vehicle and the area where the vehicle is located includes:
if the area where the vehicle is located is a lane area and the motion trail of the vehicle is different from the specified driving direction of the lane area, determining that the vehicle has a retrograde motion behavior; and/or determining that the vehicle has a behavior of exiting the road surface if the area where the vehicle is located is a non-road area and the motion track of the vehicle does not change within a first preset time; and/or if the area where the vehicle is located is the central intersection area and the motion track of the vehicle does not change within a second preset time, determining that the vehicle has the behavior of stopping at the intersection; and/or if a first vehicle with a changed motion track exists in an area outside the current passable lane area and the vehicle track of the first vehicle passes through the central intersection area, determining that the first vehicle has a behavior of running a red light.
In the embodiment of the application, the judgment of the abnormal behavior of the vehicle is realized.
In a possible implementation manner, the target includes a pedestrian, and the abnormal behavior detection method provided in the embodiment of the present application may further include:
determining a pedestrian identification frame and a pedestrian key point through a key point detection network model; inputting the identification frame of the pedestrian and the key points of the pedestrian into a behavior discrimination network model to determine the human body posture of the pedestrian; and determining whether the vehicle has abnormal behaviors or not according to the human body posture, the area where the pedestrian is located and the recognition frame of the vehicle.
In the embodiment of the application, the behaviors of the vehicle are judged by utilizing the human body posture of the pedestrian, the area where the pedestrian is located and the recognition frame of the vehicle, so that the applicability of detecting the abnormal behaviors of the vehicle can be improved.
In one possible implementation, determining whether the vehicle has abnormal behavior according to the human body posture, the area where the pedestrian is located and the recognition frame of the vehicle comprises:
and if the overlapping time of the identification frames of at least two second vehicles is detected to exceed the third preset time, and the pedestrian with the second preset posture is detected in the preset range outside the identification frames of the second vehicles, determining that the second vehicles have collision behaviors, wherein the second preset posture comprises any one or more of an upright posture, a lying posture or a squatting posture.
In the embodiment of the application, the judgment of whether the vehicle has collision behavior or not is realized by combining the recognition frame of the vehicle and the human body posture of the pedestrian, and the reliability of abnormal behavior detection is improved.
In a possible implementation, before acquiring the data to be identified, the method further includes:
acquiring data to be calibrated; and calibrating the picture area in the data to be calibrated so as to divide the picture area into different areas.
According to the method and the device, before the data to be identified is obtained, the picture area in the data to be calibrated is accurately calibrated, so that the reliability of abnormal behavior detection can be further improved.
In one possible embodiment, calibrating the picture area in the data to be calibrated includes:
detecting lane lines, pedestrian crossing lines and traffic signal lamps in the data to be calibrated; dividing the picture area into a lane area, a central intersection area, a pedestrian crossing area and a non-road area according to the lane line and the pedestrian crossing line; determining the vehicle driving direction and the pedestrian track in the data to be calibrated; determining the vehicle driving direction exceeding a preset proportion in the lane area as a specified driving direction of the lane area; and when the pedestrian track change exists in the pedestrian crossing area, the traffic signal lamp with the green signal lamp on is determined as the traffic signal lamp of the pedestrian crossing area.
In a possible implementation, before acquiring the data to be identified, the method further includes:
acquiring a training data sample; constructing a network structure of a key point detection network model; and training the training data sample by using the network structure of the key point detection network model to generate the trained key point detection network model.
In one possible implementation, the network structure of the key point detection network model comprises a two-stage detection network module, an identification frame detection module and a key point detection module; the output of the two-stage detection network module is respectively connected with the input of the identification frame detection module and the input of the key point detection module, the output of the identification frame detection module is connected with the input of the key point detection module, and the identification frame detection module sequentially comprises a box-pooler module, a box-head module, a box-classes module and a box-boxes module; the key point detection module sequentially comprises a keypoint-pooler module, a keypoint-head module and a keypoint-location module; the identification frame detection module is used for identifying an identification frame of the target, and the key point detection module is used for detecting key points of the target.
In a possible implementation manner, the abnormal behavior detection method provided in the embodiment of the present application further includes:
if the target has abnormal behaviors, alarm information is pushed to the user terminal, and the alarm information comprises the abnormal behaviors of the vehicle and the position of the vehicle.
In the embodiment of the application, the abnormal behavior of the vehicle and the position of the vehicle are pushed to the user terminal, so that the user can timely and accurately process the abnormal vehicle.
The apparatus, the electronic device, the computer-readable storage medium, and the computer program product provided in the embodiments of the present application are described below, and contents and effects thereof may refer to the abnormal behavior detection method provided in the embodiments of the present application, and are not described again.
In a second aspect, an embodiment of the present application provides an abnormal behavior detection apparatus, including:
the acquisition module is used for acquiring data to be identified, and targets in the data to be identified comprise vehicles.
And the detection module is used for determining the identification frame of the vehicle through the identification frame detection module for detecting the network model through the key points, and detecting the key points of the vehicle in the identification frame of the vehicle by using the key point detection module for detecting the network model through the key points.
The determining module is used for tracking key points of the vehicle and determining the motion track of the vehicle; and determining whether the vehicle has abnormal behaviors according to the motion trail of the vehicle and the area where the vehicle is located.
In a possible implementation, the determining module is specifically configured to:
if the area where the vehicle is located is a lane area and the motion trail of the vehicle is different from the specified driving direction of the lane area, determining that the vehicle has a retrograde motion behavior; and/or determining that the vehicle has a behavior of exiting the road surface if the area where the vehicle is located is a non-road area and the motion track of the vehicle does not change within a first preset time; and/or if the area where the vehicle is located is the central intersection area and the motion track of the vehicle does not change within a second preset time, determining that the vehicle has the behavior of stopping at the intersection; and/or if a first vehicle with a changed motion track exists in an area outside the current passable lane area and the vehicle track of the first vehicle passes through the central intersection area, determining that the first vehicle has a behavior of running a red light.
In one possible implementation, the target includes a pedestrian, and the determination module is further configured to:
and if the traffic signal lamp of the pedestrian crossing area is the red light and the motion trail of the pedestrian passes through the pedestrian crossing area, determining that the pedestrian has the behavior of running the red light.
In one possible embodiment, the target comprises a pedestrian, and the determining module further comprises: inputting the key points of the pedestrians and the recognition frames of the pedestrians into a behavior discrimination network model to determine the human body posture of the pedestrians; and determining whether the pedestrian has abnormal behaviors according to the human body posture and the region where the pedestrian is located.
In a possible implementation, the determining module is specifically configured to:
if the area where the pedestrian is located is any one of the lane area, the central intersection area or the pedestrian crossing area, and the posture of the pedestrian is a first preset posture, it is determined that the pedestrian has abnormal behaviors, and the first preset posture comprises a lying posture or a squatting posture.
In one possible implementation, the target includes a pedestrian, and the determination module is further configured to:
determining a pedestrian identification frame and a pedestrian key point through a key point detection network model; inputting the identification frame of the pedestrian and the key points of the pedestrian into a behavior discrimination network model to determine the human body posture of the pedestrian; and determining whether the vehicle has abnormal behaviors or not according to the human body posture, the area where the pedestrian is located and the recognition frame of the vehicle.
In a possible implementation, the determining module is specifically configured to:
and if the overlapping time of the identification frames of at least two second vehicles is detected to exceed the third preset time, and the pedestrian with the second preset posture is detected in the preset range outside the identification frames of the second vehicles, determining that the second vehicles have collision behaviors, wherein the second preset posture comprises any one or more of an upright posture, a lying posture or a squatting posture.
In a possible implementation manner, the abnormal behavior detection apparatus provided in the embodiment of the present application further includes a calibration module.
The calibration module is used for acquiring data to be calibrated and calibrating the picture area in the data to be calibrated so as to divide the picture area into different areas.
In a possible implementation, the calibration module is specifically configured to:
detecting lane lines, pedestrian crossing lines and traffic signal lamps in the data to be calibrated; dividing the picture area into a lane area, a central intersection area, a pedestrian crossing area and a non-road area according to the lane line and the pedestrian crossing line; determining the vehicle driving direction and the pedestrian track in the data to be calibrated; determining the vehicle driving direction exceeding a preset proportion in the lane area as a specified driving direction of the lane area; and when the pedestrian track change exists in the pedestrian crossing area, the traffic signal lamp with the green signal lamp on is determined as the traffic signal lamp of the pedestrian crossing area.
In a possible implementation manner, the abnormal behavior detection apparatus provided in the embodiment of the present application further includes a training module.
The training module is specifically used for acquiring training data samples; constructing a network structure of a key point detection network model; and training the training data sample by using the network structure of the key point detection network model to generate the trained key point detection network model.
In one possible implementation, the network structure of the key point detection network model comprises a two-stage detection network module, an identification frame detection module and a key point detection module; the output of the two-stage detection network module is respectively connected with the input of the identification frame detection module and the input of the key point detection module, the output of the identification frame detection module is connected with the input of the key point detection module, and the identification frame detection module sequentially comprises a box-pooler module, a box-head module, a box-classes module and a box-boxes module; the key point detection module sequentially comprises a keypoint-pooler module, a keypoint-head module and a keypoint-location module; the identification frame detection module is used for identifying an identification frame of the target, and the key point detection module is used for detecting key points of the target.
In a possible implementation manner, the abnormal behavior detection apparatus provided in the embodiment of the present application further includes:
the pushing module is used for pushing alarm information to the user terminal if the vehicle has abnormal behaviors, wherein the alarm information comprises the abnormal behaviors of the vehicle and the position of the vehicle.
In a third aspect, an embodiment of the present application provides an electronic device, including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein
The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method as provided by the first aspect or the first aspect realizable manner.
In a fourth aspect, embodiments of the present application provide a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the method as provided in the first aspect or the first aspect implementable manner.
In a fifth aspect, an embodiment of the present application provides a computer program product, including: executable instructions for implementing the method as provided in the first aspect or the first aspect alternatives.
According to the abnormal behavior detection method, the abnormal behavior detection device and the electronic equipment, data to be recognized are obtained, targets in the data to be recognized comprise vehicles, then the recognition frame of the vehicles is determined through a recognition frame detection module of a key point detection network model, key points of the vehicles in the recognition frame of the vehicles are detected through the key point detection module of the key point detection network model, and finally key point tracking is carried out on the key points of the vehicles to determine the motion tracks of the vehicles; and determining whether the vehicle has abnormal behaviors according to the motion trail of the vehicle and the area where the vehicle is located. In the embodiment of the application, the vehicle in the data to be recognized is subjected to key point detection through the key point detection network model, the motion track of the vehicle is determined according to the key points of the vehicle, and then whether abnormal behaviors exist in the vehicle is determined by utilizing the motion track of the vehicle and the area where the vehicle is located. Compared with the prior art that the background difference method is adopted to detect the traffic data to determine the moving vehicle and then the abnormal behavior is detected by comparing the detected moving vehicle with the predefined behavior, the accuracy and flexibility of abnormal behavior detection can be improved.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to these drawings without inventive exercise.
FIG. 1 is an exemplary application scenario architecture diagram provided by an embodiment of the present application;
fig. 2 is a schematic flowchart of an abnormal behavior detection method according to an embodiment of the present application;
FIG. 3 is a schematic diagram of key points of a vehicle provided by an embodiment of the present application;
FIG. 4 is a schematic diagram of key points of a pedestrian provided by an embodiment of the present application;
FIG. 5 is a schematic view of a traffic intersection provided by an embodiment of the present application;
fig. 6 is a schematic flowchart of an abnormal behavior detection method according to yet another embodiment of the present application;
FIG. 7 is a schematic diagram of a network structure of a key point detection network model according to an embodiment of the present application;
fig. 8 is a schematic structural diagram of an abnormal behavior detection apparatus according to an embodiment of the present application;
fig. 9 is a schematic structural diagram of an electronic device provided in an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The terms "first," "second," "third," "fourth," and the like in the description and in the claims of the present application and in the drawings described above, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the application described herein are, for example, capable of operation in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
At present, intelligent video monitoring becomes one of the very active hot techniques in the field of artificial intelligence, and is an important application of the computer vision technology in the field of security protection. The traffic flow peak or the people flow peak often exists in public areas such as traffic intersections, railway stations, subway stations, squares and the like, and the detection of the abnormal behaviors of vehicles and pedestrians is particularly important for avoiding accidents and timely finding the accidents. In the prior art, for detecting abnormal traffic behaviors, a conventional algorithm is usually adopted to detect abnormal behaviors, which mainly includes traffic data acquisition, then traffic data is detected by a background difference method to determine a moving vehicle, and then the moving vehicle is compared with a predefined behavior to determine whether the behavior of the moving vehicle is the predefined behavior, so that the abnormal behavior of the vehicle is detected. However, in the prior art, the detection of abnormal behaviors by using a conventional algorithm is limited to various behavior scenes, and the robustness of the detection of abnormal behaviors is poor.
The abnormal behavior detection method, the abnormal behavior detection device and the electronic equipment have the inventive concept that the key point detection is performed on the target in the data to be recognized through the key point detection network model, and whether the abnormal behavior exists in the target is determined according to the key point of the target and the area where the target is located. In addition, by using the key point tracking and the track calculation of the target, more detailed and more accurate information such as the position, the behavior and the like of the target can be obtained, the detection of the behavior of the target is more convenient and accurate, and the method is not limited to the condition that the collection weather of the data to be identified is rainy days, sunny days, cloudy days, daytime or nighttime and the like.
An exemplary application scenario of the embodiments of the present application is described below.
The abnormal behavior detection method provided by the embodiment of the present application can be executed by the abnormal behavior detection device provided by the embodiment of the present application, and the abnormal behavior detection device provided by the embodiment of the present application can be integrated on the terminal device, or the abnormal behavior detection device can be the terminal device itself. The specific type of the terminal device is not limited in the embodiment of the application, for example, the terminal device may be a camera, a personal computer, a tablet computer, a wearable device, a vehicle-mounted terminal, a monitoring device, and the like. Fig. 1 is an exemplary application scenario architecture diagram provided in an embodiment of the present application, and as shown in fig. 1, the architecture mainly includes: a terminal device 01 (personal computer) and a monitoring device 02. In a possible implementation manner, the abnormal behavior detection method provided in this embodiment of the present application may be applied to the monitoring device 02, acquire data to be identified through a camera of the monitoring device 02, process the data to be identified through a graphics processor in the monitoring device, and then send a processing result to the terminal device 01. In another possible implementation manner, the embodiment of the present application may be applied to the terminal device 01, and the data to be identified is received and then processed by the monitoring device 02. In yet another possible implementation manner, the application scene architecture diagram provided in this embodiment of the present application may further include a server 03, the terminal device 01 obtains data to be identified through the monitoring device 02, then processes the data to be identified through the server 03, and sends a processing result to the terminal device 01, and the terminal device 01 receives the processing result sent by the server 03. The embodiments of the present application are merely examples, and are not limited thereto.
It should be noted that, the embodiment of the present application is described only by taking an application in a traffic intersection as an example, and the embodiment of the present application may also be applied in scenes such as a train station, a market, a hospital, a school, a subway entrance, and the like to detect abnormal behaviors of pedestrians or vehicles.
Fig. 2 is a schematic flow diagram of an abnormal behavior detection method according to an embodiment of the present application, where the method may be executed by an abnormal behavior detection apparatus, and the apparatus may be implemented in a software and/or hardware manner, and the abnormal behavior detection method is described below with a terminal device as an execution subject, as shown in fig. 2, the abnormal behavior detection method according to the embodiment of the present application may include:
step S101: and acquiring data to be identified.
The data to be recognized can include images to be recognized or videos to be recognized, and the images to be recognized or the videos to be recognized can be obtained through a camera of the monitoring device. The data to be recognized includes a monitoring area, and the embodiment of the present application may be configured to detect abnormal behaviors in the monitoring area, where a target in the data to be recognized may include a vehicle, the abnormal behavior may include an abnormal behavior of the vehicle, the target in the data to be recognized may also include a pedestrian, and the abnormal behavior may include an abnormal behavior of the pedestrian or an abnormal behavior of the vehicle, which is not limited in the embodiment of the present application.
Step S102: and determining the identification frame of the vehicle by using an identification frame detection module of the key point detection network model, and detecting the key points of the vehicle in the identification frame of the vehicle by using the key point detection module of the key point detection network model.
After the data to be identified are obtained, the key point detection is carried out on the vehicles in the data to be identified through the key point detection network model. The embodiment of the application does not limit the network structure of the key point detection network model, and only needs to detect the key points of the targets in the data to be identified. In a possible implementation manner, the key point detecting network model may include an identification frame detecting module and a key point detecting module, and performing key point detection on a target in data to be identified through the key point detecting network model to obtain a key point of the target, which may include: and determining the identification frame of the vehicle by using an identification frame detection module of the key point detection network model, and detecting the key points of the vehicle in the identification frame of the vehicle by using the key point detection module of the key point detection network model.
According to the embodiment of the application, the key point of the target can be obtained through the key point detection network module, the identification frame of the target can be obtained, the subsequent detection of the behavior of the target by combining the key point and the target frame is facilitated, and the reliability of abnormal behavior detection can be further improved.
The key point detection network model can directly adopt a key point detection network model which is trained by other terminal equipment or a server, and can also be trained by the terminal equipment. In a possible implementation manner, before acquiring the data to be identified, the abnormal behavior detection method provided in the embodiment of the present application may further include:
acquiring a training data sample; constructing a network structure of a key point detection network model; and training the training data sample by using the network structure of the key point detection network model to generate the trained key point detection network model.
The method comprises the steps of obtaining a training data sample, obtaining traffic data through a camera or monitoring equipment, wherein the traffic data can comprise training images or training videos, and then carrying out key point marking on the traffic data to generate the training data sample. And inputting the training data sample into the network structure of the constructed key point detection network model, and finally training the training data sample by using the network structure of the key point detection network model to generate the trained key point detection network model. The embodiments of the present application are merely examples, and are not limited thereto.
For the key points of the targets, the number, the positions, and the like of the key points required for different targets may be different, and in one possible implementation, fig. 3 is a schematic diagram of the key points of the vehicle provided in an embodiment of the present application, and fig. 4 is a schematic diagram of the key points of the pedestrian provided in an embodiment of the present application. As shown in fig. 3, the key points of the vehicle may include a roof front left key point 1, a roof front right key point 2, a front vehicle left key point 3, a front vehicle right key point 4, a body front left key point 5, a body front right key point 6, a left side mirror key point 7, a right side mirror key point 8 (not shown), a roof rear left key point 9, a roof rear right key point 10, a rear vehicle left key point 11 (not shown), a rear vehicle right key point 12 (not shown), a body rear left key point 13, and a body right key point 14 (not shown). The embodiment of the present application is only an example, and the number and the positions of the key points of the vehicle are not limited to this, and for example, the key points may be provided at four wheels of the vehicle. As shown in fig. 4, the key points of the pedestrian may include left eye 1, right eye 2, left ear 3, right ear 4, left shoulder 5, right shoulder 6, left elbow 7, right elbow 8, left hand 9, right hand 10, left hip 11, right hip 12, left knee 13, right knee 14, left foot 15, right foot 16, nose 17. The embodiment of the present application is only an example, and the number and the positions of the key points of the pedestrian are not limited thereto.
Step S103: and carrying out key point tracking on key points of the vehicle and determining the motion track of the vehicle.
Step S104: and determining whether the vehicle has abnormal behaviors according to the motion trail of the vehicle and the area where the vehicle is located.
After determining the key points of the vehicle, in one possible embodiment, it may be determined whether there is an abnormal behavior of the vehicle according to the key points of the vehicle and the area where the vehicle is located. The behavior, action, etc. of the vehicle may be determined by key points of the vehicle. The embodiment of the application does not limit the specific implementation manner of determining whether the vehicle has the abnormal behavior according to the key point of the vehicle and the area where the vehicle is located. In different areas, normal behaviors of pedestrians or vehicles may be inconsistent, whether the current behavior of the vehicle is normal in the area where the vehicle is located can be judged through key points of the vehicle and the area where the vehicle is located, if not, it is determined that the vehicle has an abnormal behavior, and if so, it is determined that the vehicle does not have the abnormal behavior. For example, the driving direction of the vehicle and the area where the vehicle is located are determined by acquiring the position of a camera of the data to be identified and the detected key points of the vehicle, and if the driving direction of the vehicle is not consistent with the current driving direction of the vehicle in the area, the vehicle may have a reverse behavior.
In another possible embodiment, determining whether the vehicle has abnormal behavior according to the key points of the vehicle and the area where the vehicle is located may be implemented by steps S103 and S104.
After the key points of the vehicle are determined, the key points of the vehicle can be tracked, and the specific implementation manner of the key point tracking of the key points of the vehicle in the embodiment of the application is not limited as long as the motion track of the vehicle can be determined. For example, the keypoint tracking may be implemented by a recurrent neural network, and this is merely an example of the embodiment of the present application.
After the movement trace of the vehicle is determined, whether abnormal behavior of the vehicle exists may be determined according to the movement trace of the vehicle and the area where the vehicle is located. The abnormal behaviors in the area where the vehicle is located may be different for different vehicles and different application scenes, and the abnormal behaviors may be specifically set according to different application scenes, so that the abnormal behaviors are determined according to the motion track of the vehicle and the area where the vehicle is located.
The following describes a determination of an abnormal behavior that may exist in a vehicle, and the embodiment of the present application is only used as an example and is not limited thereto.
In one possible implementation, determining whether the vehicle has abnormal behavior according to the motion track of the vehicle and the area where the vehicle is located includes:
if the area where the vehicle is located is a lane area and the motion trail of the vehicle is different from the specified driving direction of the lane area, determining that the vehicle has a retrograde motion behavior; and/or determining that the vehicle has a behavior of exiting the road surface if the area where the vehicle is located is a non-road area and the motion track of the vehicle does not change within a first preset time; and/or if the area where the vehicle is located is the central intersection area and the motion track of the vehicle does not change within a second preset time, determining that the vehicle has the behavior of stopping at the intersection; and/or if a first vehicle with a changed motion track exists in an area outside the current passable lane area and the vehicle track of the first vehicle passes through the central intersection area, determining that the first vehicle has a behavior of running a red light.
For convenience of introduction, fig. 5 is a schematic diagram of a traffic intersection provided in an embodiment of the present application, as shown in fig. 5, where the traffic intersection shown in fig. 5 includes four roads, and the embodiment of the present application only takes a road 1 and a road 2 as an example, where the road 1 and the road 2 respectively include two lane areas, the road 1 includes a lane area 1 and a lane area 4, and the road 2 includes a lane area 2 and a lane area 3, and an arrow in the lane area indicates a specified driving direction of the lane area. If the vehicle is in the lane area 1 and the motion trail of the vehicle is different from the prescribed driving direction of the lane area 1, for example, the driving direction is opposite or the vehicle turns around, the vehicle is determined to have a reverse driving behavior.
The traffic intersection further comprises at least one traffic signal lamp, which is described by taking one traffic signal lamp as an example, the traffic signal lamp is used for indicating a current passable lane, for example, when the current traffic signal lamp indicates that the road 1 and the road 2 can pass through, the current passable lane areas are a lane area 1, a lane area 2, a lane area 3 and a lane area 4, and if at this time, the motion track of the vehicle changes and passes through a central intersection area in other lanes, it is determined that the vehicle has a behavior of running a red light.
The central intersection area is an important area of a road, the vehicle cannot stay for a long time without special conditions, otherwise traffic jam may be caused, and therefore if the vehicle is detected to be in the central intersection area and the motion track is not changed within the second preset time, it is determined that the vehicle has an intersection stay behavior.
The method comprises the steps that a non-road area is further included outside a lane area, a central intersection area and a pedestrian crossing area, if the vehicle stays in the non-road area for a long time, the vehicle possibly breaks down to cause the behavior of driving out of the road surface, and therefore the vehicle can be determined to have the behavior of driving out of the road surface when the vehicle is in the non-road area and the motion track of the vehicle is not changed within first preset time. The embodiments of the present application are merely examples, and are not limited thereto.
If the abnormal behavior of the target is detected, the target needs to be reminded or timely processed, for example, a pedestrian or a vehicle violates a traffic rule, the pedestrian can be reminded, and if a traffic accident or an unexpected condition of the pedestrian occurs, a traffic department, a medical department and the like can be informed to timely process the abnormal behavior. In order to solve the above technical problem, in a possible implementation manner, the abnormal behavior detection method provided in an embodiment of the present application may further include:
if the vehicle has abnormal behaviors, alarm information is pushed to the user terminal, and the alarm information comprises the abnormal behaviors of the vehicle and the position of the vehicle.
The specific implementation manner of pushing the alarm information to the user terminal is not limited in the embodiments of the present application, for example, an audio player or a video player may be arranged at a traffic intersection, and a behavior violating a traffic rule is played by the audio player or a behavior violating the traffic rule is played by the video player. For another example, the alarm information can be pushed to a terminal device of a traffic department to monitor a traffic intersection. The embodiment of the application does not limit the specific information included in the warning information, for example, the warning information may include an abnormal behavior of the vehicle and a position of the vehicle. For example, at an intersection between a road a and a road B, there is a collision between the vehicle a and the vehicle B, and this is merely taken as an example and not a limitation. In the embodiment of the application, the abnormal behavior of the vehicle and the position of the vehicle are pushed to the user terminal, so that the user can process the vehicle timely and accurately.
In the embodiment of the application, the vehicle in the data to be recognized is subjected to key point detection through the key point detection network model, the motion track of the vehicle is determined according to the key points of the vehicle, and then whether abnormal behaviors exist in the vehicle is determined by utilizing the motion track of the vehicle and the area where the vehicle is located. In addition, by using the key point tracking and the track calculation of the target, compared with the traditional method for detecting the target frame, the method can obtain more detailed and accurate information such as the position, the behavior and the like of the target based on the key point detection, is more convenient and accurate to detect the behavior of the target, not only improves the precision, but also simplifies the flow.
In a possible implementation manner, the target may also be a pedestrian, and the determination of the abnormal behavior that may exist in the pedestrian is described below by taking the example that the target includes the pedestrian, and the example of the application is only taken as an example and is not limited thereto.
Taking a traffic intersection as an example, for example, a pedestrian should stand in a non-lane area to wait when a red signal light of a traffic signal light corresponding to a sidewalk area is on, and can pass through the sidewalk area until a green signal light of the traffic signal light is on. If the pedestrian is detected to pass through the sidewalk area or stay in the sidewalk area when a red signal lamp of a traffic signal lamp corresponding to the sidewalk area is on, it can be determined that the pedestrian has an abnormal behavior, and the abnormal behavior is that the pedestrian runs a red light. For another example, the behaviors of the pedestrian at the traffic intersection are generally standing and walking, and if the behaviors of lying, squatting and the like of the pedestrian are detected in the sidewalk area, it can be determined that the pedestrian has an abnormal behavior, and the abnormal behavior may be the conditions of collision of the pedestrian, sudden diseases of the pedestrian and the like and needs to be processed in time. The embodiments of the present application are merely examples, and are not limited thereto.
In yet another possible implementation, determining whether the pedestrian has abnormal behavior according to the motion trail of the pedestrian and the area where the pedestrian is located includes:
and if the traffic signal lamp of the pedestrian crossing area is the red light and the motion trail of the pedestrian passes through the pedestrian crossing area, determining that the pedestrian has the behavior of running the red light.
As shown in fig. 5, the traffic intersection includes four sidewalks, this embodiment of the application only uses sidewalk 1 and sidewalk 2 as an example, the area where sidewalk 1 and sidewalk 2 are located is the pedestrian crossing area, there is respective traffic signal lamp in the pedestrian crossing area, the pedestrian passes or waits through observing the traffic signal lamp that this pedestrian crossing corresponds, if the traffic signal lamp of the pedestrian crossing area where the pedestrian is located is the red light, and the motion trail of the pedestrian passes the pedestrian crossing area, then it is determined that the pedestrian has the behavior of running the red light.
On the basis of the foregoing embodiment, taking an example that the target includes a pedestrian and a vehicle, in a possible implementation manner, fig. 6 is a schematic flowchart of an abnormal behavior detection method provided in yet another embodiment of the present application, and as shown in fig. 6, the abnormal behavior detection method provided in the embodiment of the present application may further include:
step S201: and determining the identification frame of the pedestrian and the key points of the pedestrian through the key point detection network model.
Step S202: and inputting the identification frame of the pedestrian and the key points of the pedestrian into the behavior discrimination network model so as to determine the human body posture of the pedestrian.
Step S203: and determining whether the vehicle has abnormal behaviors or not according to the human body posture, the area where the pedestrian is located and the recognition frame of the vehicle.
The network model can be used for determining the identification frame of the pedestrian and the key points of the pedestrian through the key point detection network model, and the behavior discrimination network model is used for identifying the human body posture of the pedestrian, wherein the human body posture comprises an upright posture, a stooped posture, a lying posture, a squatting posture, a putting posture and the like. Before inputting the key points of the pedestrian and the recognition frame of the pedestrian into the behavior discrimination network model, the embodiment of the application may further include: the embodiment of the present application does not limit the process of constructing and training the network structure of the behavior discrimination network model, and is not described in detail again. The human body posture can be directly obtained through the behavior discrimination network model, and a database and subsequent logic judgment do not need to be established in advance, so that the method is more accurate and quicker.
After the human body posture of the pedestrian is determined through the behavior discrimination network model, in one possible implementation mode, whether the pedestrian has abnormal behaviors or not can be determined according to the human body posture and the area where the pedestrian is located, and in another possible implementation mode, whether the vehicle has abnormal behaviors or not can also be determined according to the human body posture, the area where the pedestrian is located and the recognition frame of the vehicle.
The embodiment of the application does not limit the specific implementation mode for determining whether the pedestrian has abnormal behaviors according to the human body posture and the area where the pedestrian is located. For example, the correspondence between the region and the behavior may be established, and then if it is detected that the pedestrian has a behavior other than the behavior corresponding to the region in the region, it may be determined that the pedestrian has an abnormal behavior.
In one possible implementation, the determining whether the pedestrian has abnormal behavior according to the human body posture and the area where the pedestrian is located comprises the following steps: if the area where the pedestrian is located is any one of the lane area, the central intersection area or the pedestrian crossing area, and the posture of the pedestrian is a first preset posture, it is determined that the pedestrian has abnormal behaviors, and the first preset posture comprises a lying posture or a squatting posture.
With reference to fig. 5, if the area where the pedestrian is located is a lane area, a central intersection area or a pedestrian crossing area, the normal behavior of the pedestrian may be a behavior of walking upright, standing still, lowering head, etc., and if there is an abnormal behavior of a lying posture or a squatting posture in the posture of the pedestrian, it may indicate that the pedestrian has an accident or an accident, etc., and needs to be handled in time. The first preset posture can be other postures besides a lying posture or a squatting posture, and the first preset posture is not limited in the embodiment of the application.
In the embodiment of the application, whether abnormal behaviors exist in the pedestrian or not is judged through the human body posture of the pedestrian and the region where the pedestrian is located, so that the detection of the abnormal behaviors of the pedestrian is realized, and the detection accuracy is improved.
In another possible implementation, determining whether the vehicle has abnormal behavior according to the human body posture, the area where the pedestrian is located and the recognition frame of the vehicle may include:
and if the overlapping time of the identification frames of at least two second vehicles is detected to exceed the third preset time, and the pedestrian with the second preset posture is detected in the preset range outside the identification frames of the second vehicles, determining that the second vehicles have collision behaviors, wherein the second preset posture comprises any one or more of an upright posture, a lying posture or a squatting posture.
If it is detected that the identification frames of the second vehicles overlap for a long time, the second vehicles may have traffic accidents such as friction and collision, or the vehicles ahead may be in a standstill due to traffic congestion. By combining a preset range outside the identification frame of the second vehicle, for example, the identification frame is expanded by 1.2 times, the size of the preset range is not limited in the embodiment of the present application. If the pedestrian with the second preset posture is detected in the preset range outside the identification frame of the second vehicle, it can be determined that the second vehicles have collision behaviors. The specific posture of the second preset posture is not limited in the embodiments of the present application, for example, the second preset posture includes any one or a combination of a plurality of upright postures, lying postures, or squatting postures.
In the embodiment of the application, the judgment of whether the vehicle has collision behavior or not is realized by combining the recognition frame of the vehicle and the human body posture of the pedestrian, and the reliability of abnormal behavior detection is improved.
On the basis of the foregoing embodiment, in a possible implementation manner, before acquiring the data to be identified, the method further includes:
acquiring data to be calibrated; and calibrating the picture area in the data to be calibrated so as to divide the picture area into different areas.
The data to be calibrated can be a video to be calibrated or an image to be calibrated, and the data to be calibrated is obtained through the camera after the camera is installed and fixed. For different application scenarios, the requirements for the data to be calibrated may be different, for example, for a traffic intersection scenario, the data to be calibrated may need to include traffic data when a traffic light changes multiple times. For different application scenarios, the number and types of regions divided into the picture region in the data to be calibrated may also be different, which is not limited in the embodiment of the present application.
According to the method and the device, before the data to be identified is obtained, the picture area in the data to be calibrated is accurately calibrated, so that the reliability of abnormal behavior detection can be further improved.
Taking the data to be calibrated as the traffic data as an example, in one possible implementation manner, calibrating the picture area in the data to be calibrated includes:
detecting lane lines, pedestrian crossing lines and traffic signal lamps in the data to be calibrated; dividing the picture area into a lane area, a central intersection area, a pedestrian crossing area and a non-road area according to the lane line and the pedestrian crossing line; determining the vehicle driving direction and the pedestrian track in the data to be calibrated; determining the vehicle driving direction exceeding a preset proportion in the lane area as a specified driving direction of the lane area; and when the pedestrian track change exists in the pedestrian crossing area, the traffic signal lamp with the green signal lamp on is determined as the traffic signal lamp of the pedestrian crossing area.
As shown in fig. 5, lane lines, pedestrian crossing lines, and traffic lights in the data to be calibrated are detected, and then each lane in the screen area can be divided into lane areas, each pedestrian crossing can be divided into pedestrian crossing areas, and a central intersection area and a non-road area can be determined through the corner points of the lane lines and the identification frames of the pedestrian crossing lines. After the lane area and the pedestrian crossing area are determined, the specified driving direction of the lane area and the traffic signal lamp of the pedestrian crossing area can be determined according to the vehicle driving direction and the pedestrian track in the data to be calibrated.
In the embodiment of the application, the key points of the target are tracked, the motion track of the target is determined, and whether abnormal behaviors exist in the target is determined according to the motion track of the target and the area where the target is located.
In order to realize the identification of the identification frame, the category, and the key point of the target, in a possible implementation manner, fig. 7 is a schematic diagram of a network structure of a key point detection network model provided in an embodiment of the present application, and as shown in fig. 7, the network structure of the key point detection network model includes a two-stage detection network module, an identification frame detection module, and a key point detection module; the output of the two-stage detection network module is respectively connected with the input of the identification frame detection module and the input of the key point detection module, the output of the identification frame detection module is connected with the input of the key point detection module, and the identification frame detection module sequentially comprises a box-pooler module, a box-head module, a box-classes module and a box-boxes module; the key point detection module sequentially comprises a keypoint-pooler module, a keypoint-head module and a keypoint-location module; the identification frame detection module is used for identifying the identification frame and the category of the target, and the key point detection module is used for detecting the key point of the target.
Inputting a training data sample into a two-stage detection network for feature recognition and feature extraction, then inputting the training data sample into a recognition frame detection module, and classifying targets through a box-pooler module, a box-head module and a box-classes module in the recognition frame detection module to obtain the category of the targets; the detection of the identification frame of the target is realized through a box-pooler module, a box-head module and a box-boxes module in the identification frame detection module so as to obtain the identification frame of the target. The key point of the target is obtained by inputting the identification frame of the target, the category of the target and the output result of the two-stage detection network into the key point detection module and processing the keypoint-pooler module, the keypoint-head module and the keypoint-location module in the key point detection module.
The following are embodiments of the apparatus of the present application that may be used to perform embodiments of the method of the present application. For details which are not disclosed in the embodiments of the apparatus of the present application, reference is made to the embodiments of the method of the present application.
Fig. 8 is a schematic structural diagram of an abnormal behavior detection apparatus provided in an embodiment of the present application, where the apparatus may be implemented in a software and/or hardware manner, for example, may be implemented by a terminal device, as shown in fig. 8, the abnormal behavior detection apparatus provided in the embodiment of the present application may include: an acquisition module 41, a detection module 42 and a determination module 43.
The acquiring module 41 is configured to acquire data to be identified, where an object in the data to be identified includes a vehicle.
And the detection module 42 is configured to determine the identification frame of the vehicle through the identification frame detection module for detecting the network model by using the key point, and detect the key point of the vehicle in the identification frame of the vehicle by using the key point detection module for detecting the network model by using the key point.
The determining module 43 is configured to perform key point tracking on key points of the vehicle, and determine a motion trajectory of the vehicle; and determining whether the vehicle has abnormal behaviors according to the motion trail of the vehicle and the area where the vehicle is located.
In a possible implementation, the determining module 43 is specifically configured to:
if the area where the vehicle is located is a lane area and the motion trail of the vehicle is different from the specified driving direction of the lane area, determining that the vehicle has a retrograde motion behavior; and/or determining that the vehicle has a behavior of exiting the road surface if the area where the vehicle is located is a non-road area and the motion track of the vehicle does not change within a first preset time; and/or if the area where the vehicle is located is the central intersection area and the motion track of the vehicle does not change within a second preset time, determining that the vehicle has the behavior of stopping at the intersection; and/or if a first vehicle with a changed motion track exists in an area outside the current passable lane area and the vehicle track of the first vehicle passes through the central intersection area, determining that the first vehicle has a behavior of running a red light.
In one possible embodiment, the target includes a pedestrian, and the determining module 43 is further configured to:
and if the traffic signal lamp of the pedestrian crossing area is the red light and the motion trail of the pedestrian passes through the pedestrian crossing area, determining that the pedestrian has the behavior of running the red light.
In one possible embodiment, the target comprises a pedestrian, and the determining module 43 further comprises: inputting the key points of the pedestrians and the recognition frames of the pedestrians into a behavior discrimination network model to determine the human body posture of the pedestrians; and determining whether the pedestrian has abnormal behaviors according to the human body posture and the region where the pedestrian is located.
In a possible implementation, the determining module 43 is specifically configured to:
if the area where the pedestrian is located is any one of the lane area, the central intersection area or the pedestrian crossing area, and the posture of the pedestrian is a first preset posture, it is determined that the pedestrian has abnormal behaviors, and the first preset posture comprises a lying posture or a squatting posture.
In one possible embodiment, the target includes a pedestrian, and the determining module 43 is further configured to:
determining a pedestrian identification frame and a pedestrian key point through a key point detection network model; inputting the identification frame of the pedestrian and the key points of the pedestrian into a behavior discrimination network model to determine the human body posture of the pedestrian; and determining whether the vehicle has abnormal behaviors or not according to the human body posture, the area where the pedestrian is located and the recognition frame of the vehicle.
In a possible implementation, the determining module 43 is specifically configured to:
and if the overlapping time of the identification frames of at least two second vehicles is detected to exceed the third preset time, and the pedestrian with the second preset posture is detected in the preset range outside the identification frames of the second vehicles, determining that the second vehicles have collision behaviors, wherein the second preset posture comprises any one or more of an upright posture, a lying posture or a squatting posture.
In a possible implementation manner, as shown in fig. 8, the abnormal behavior detection apparatus provided in the embodiment of the present application further includes a calibration module 44.
The calibration module 44 is configured to obtain data to be calibrated, and calibrate a picture area in the data to be calibrated, so as to divide the picture area into different areas.
In one possible implementation, the calibration module 44 is specifically configured to:
detecting lane lines, pedestrian crossing lines and traffic signal lamps in the data to be calibrated; dividing the picture area into a lane area, a central intersection area, a pedestrian crossing area and a non-road area according to the lane line and the pedestrian crossing line; determining the vehicle driving direction and the pedestrian track in the data to be calibrated; determining the vehicle driving direction exceeding a preset proportion in the lane area as a specified driving direction of the lane area; and when the pedestrian track change exists in the pedestrian crossing area, the traffic signal lamp with the green signal lamp on is determined as the traffic signal lamp of the pedestrian crossing area.
In a possible implementation manner, as shown in fig. 8, the abnormal behavior detection apparatus provided in the embodiment of the present application further includes a training module 45.
A training module 45, specifically configured to obtain training data samples; constructing a network structure of a key point detection network model; and training the training data sample by using the network structure of the key point detection network model to generate the trained key point detection network model.
In one possible implementation, the network structure of the key point detection network model comprises a two-stage detection network module, an identification frame detection module and a key point detection module; the output of the two-stage detection network module is respectively connected with the input of the identification frame detection module and the input of the key point detection module, the output of the identification frame detection module is connected with the input of the key point detection module, and the identification frame detection module sequentially comprises a box-pooler module, a box-head module, a box-classes module and a box-boxes module; the key point detection module sequentially comprises a keypoint-pooler module, a keypoint-head module and a keypoint-location module; the identification frame detection module is used for identifying an identification frame of the target, and the key point detection module is used for detecting key points of the target.
In a possible implementation manner, the abnormal behavior detection apparatus provided in the embodiment of the present application further includes:
and the pushing module 46 is configured to push alarm information to the user terminal if the vehicle has an abnormal behavior, where the alarm information includes the abnormal behavior of the vehicle and a position of the vehicle.
The device embodiments provided in the present application are merely schematic, and the module division in fig. 8 is only one logic function division, and there may be another division manner in actual implementation. For example, multiple modules may be combined or may be integrated into another system. The coupling of the various modules to each other may be through interfaces that are typically electrical communication interfaces, but mechanical or other forms of interfaces are not excluded. Thus, modules described as separate components may or may not be physically separate, may be located in one place, or may be distributed in different locations on the same or different devices.
Fig. 9 is a schematic structural diagram of an electronic device provided in an embodiment of the present application, where the electronic device may be a terminal device in the foregoing embodiment, and as shown in fig. 9, the electronic device includes:
a processor 61, a memory 62, a transceiver 63 and a computer program; wherein the transceiver 63 enables data transmission with other devices, a computer program is stored in the memory 62 and configured to be executed by the processor 61, the computer program comprising instructions for performing the above abnormal behavior detection method, the contents and effects of which refer to the method embodiments.
In addition, embodiments of the present application further provide a computer-readable storage medium, in which computer-executable instructions are stored, and when at least one processor of the user equipment executes the computer-executable instructions, the user equipment performs the above-mentioned various possible methods.
Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media may be any available media that can be accessed by a general purpose or special purpose computer. An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. Of course, the storage medium may also be integral to the processor. The processor and the storage medium may reside in an ASIC. Additionally, the ASIC may reside in user equipment. Of course, the processor and the storage medium may reside as discrete components in a communication device.
Those of ordinary skill in the art will understand that: all or a portion of the steps of implementing the above-described method embodiments may be performed by hardware associated with program instructions. The program may be stored in a computer-readable storage medium. When executed, the program performs steps comprising the method embodiments described above; and the aforementioned storage medium includes: various media that can store program codes, such as ROM, RAM, magnetic or optical disks.
Finally, it should be noted that: the above embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present application.

Claims (10)

1. An abnormal behavior detection method, comprising:
acquiring data to be identified, wherein a target in the data to be identified comprises a vehicle;
determining an identification frame of the vehicle through an identification frame detection module of a key point detection network model, and detecting key points of the vehicle in the identification frame of the vehicle through the key point detection module of the key point detection network model;
carrying out key point tracking on key points of the vehicle, and determining a motion track of the vehicle;
and determining whether the vehicle has abnormal behaviors according to the motion trail of the vehicle and the area where the vehicle is located.
2. The method of claim 1, wherein the object comprises a pedestrian, the method further comprising:
determining the identification frame of the pedestrian and the key points of the pedestrian through the key point detection network model;
inputting the identification frame of the pedestrian and the key points of the pedestrian into a behavior discrimination network model to determine the human body posture of the pedestrian;
and determining whether the vehicle has abnormal behaviors or not according to the human body posture, the area where the pedestrian is located and the recognition frame of the vehicle.
3. The method according to claim 2, wherein the determining whether the vehicle has abnormal behavior according to the human body posture, the area where the pedestrian is located and the recognition frame of the vehicle comprises:
if the overlapping time of the identification frames of the at least two second vehicles exceeds the third preset time and the pedestrian with the second preset posture is detected in the preset range outside the identification frames of the second vehicles, the collision behavior of the second vehicles is determined, and the second preset posture comprises any one or more of the combination of the upright posture, the lying posture and the squatting posture.
4. The method according to any one of claims 1-3, further comprising, prior to obtaining the data to be identified:
acquiring data to be calibrated;
and calibrating the picture area in the data to be calibrated so as to divide the picture area into different areas.
5. The method according to claim 4, wherein the calibrating the picture area in the data to be calibrated comprises:
detecting lane lines, pedestrian crossing lines and traffic signal lamps in the data to be calibrated;
dividing the picture area into a lane area, a central intersection area, a pedestrian crossing area and a non-road area according to the lane line and the pedestrian crossing line;
determining the vehicle driving direction and the pedestrian track in the data to be calibrated;
determining the vehicle driving direction exceeding a preset proportion in the lane area as a specified driving direction of the lane area;
and when the pedestrian track change exists in the pedestrian crossing area, determining the traffic signal lamp with the green signal lamp as the traffic signal lamp of the pedestrian crossing area.
6. The method according to any one of claims 1-3, further comprising, prior to said obtaining data to be identified:
acquiring a training data sample;
constructing a network structure of the key point detection network model;
and training the training data sample by using the network structure of the key point detection network model to generate a trained key point detection network model.
7. The method of claim 6, wherein the network structure of the key point detection network model comprises a two-stage detection network module, a recognition box detection module, and a key point detection module;
the output of the two-stage detection network module is respectively connected with the input of the identification frame detection module and the input of the key point detection module, the output of the identification frame detection module is connected with the input of the key point detection module, and the identification frame detection module sequentially comprises a box-pooler module, a box-head module, a box-classes module and a box-boxes module; the key point detection module sequentially comprises a keypoint-pooler module, a keypoint-head module and a keypoint-location module;
the identification frame detection module is used for identifying an identification frame of a target, and the key point detection module is used for detecting key points of the target.
8. The method according to any one of claims 1-3, further comprising:
if the vehicle has abnormal behaviors, alarm information is pushed to a user terminal, and the alarm information comprises the abnormal behaviors of the vehicle and the position of the vehicle.
9. An abnormal behavior detection apparatus, comprising:
the system comprises an acquisition module, a recognition module and a processing module, wherein the acquisition module is used for acquiring data to be recognized, and a target in the data to be recognized comprises a vehicle;
the detection module is used for determining the identification frame of the vehicle through the identification frame detection module of the key point detection network model, and detecting the key points of the vehicle in the identification frame of the vehicle by using the key point detection module of the key point detection network model;
the determining module is used for tracking key points of the vehicle and determining the motion track of the vehicle; and determining whether the vehicle has abnormal behaviors according to the motion trail of the vehicle and the area where the vehicle is located.
10. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein
The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-8.
CN202011058838.8A 2020-09-30 Abnormal behavior detection method and device and electronic equipment Active CN112200044B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011058838.8A CN112200044B (en) 2020-09-30 Abnormal behavior detection method and device and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011058838.8A CN112200044B (en) 2020-09-30 Abnormal behavior detection method and device and electronic equipment

Publications (2)

Publication Number Publication Date
CN112200044A true CN112200044A (en) 2021-01-08
CN112200044B CN112200044B (en) 2024-04-30

Family

ID=

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112907949A (en) * 2021-01-20 2021-06-04 北京百度网讯科技有限公司 Traffic anomaly detection method, model training method and device
CN113033471A (en) * 2021-04-15 2021-06-25 北京百度网讯科技有限公司 Traffic abnormality detection method, apparatus, device, storage medium, and program product
WO2022267387A1 (en) * 2021-06-21 2022-12-29 梅卡曼德（北京）机器人科技有限公司 Image recognition method and apparatus, electronic device, and storage medium
CN115909286A (en) * 2023-02-17 2023-04-04 浙江口碑网络技术有限公司 Data processing method and device
WO2023206236A1 (en) * 2022-04-28 2023-11-02 华为技术有限公司 Method for detecting target and related device

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106652465A (en) * 2016-11-15 2017-05-10 成都通甲优博科技有限责任公司 Method and system for identifying abnormal driving behavior on road
WO2017157119A1 (en) * 2016-03-18 2017-09-21 中兴通讯股份有限公司 Method and device for identifying abnormal behavior of vehicle
US20180297593A1 (en) * 2015-12-21 2018-10-18 Bayerische Motoren Werke Aktiengesellschaft Method for Collision Avoidance of a Motor Vehicle with an Emergency Vehicle and a Related System and Motor Vehicle
KR20190084594A (en) * 2018-01-08 2019-07-17 현대모비스 주식회사 Apparatus and method tracking object based on 3 dimension images
CN110178167A (en) * 2018-06-27 2019-08-27 潍坊学院 Crossing video frequency identifying method violating the regulations based on video camera collaboration relay
CN110503083A (en) * 2019-08-30 2019-11-26 北京妙医佳健康科技集团有限公司 A kind of critical point detection method, apparatus and electronic equipment
CN111626189A (en) * 2020-05-26 2020-09-04 浙江大华技术股份有限公司 Road surface abnormity detection method and device, electronic equipment and storage medium
WO2020187257A1 (en) * 2019-03-18 2020-09-24 长城汽车股份有限公司 Vehicle abnormal lane change control method, device and system

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180297593A1 (en) * 2015-12-21 2018-10-18 Bayerische Motoren Werke Aktiengesellschaft Method for Collision Avoidance of a Motor Vehicle with an Emergency Vehicle and a Related System and Motor Vehicle
WO2017157119A1 (en) * 2016-03-18 2017-09-21 中兴通讯股份有限公司 Method and device for identifying abnormal behavior of vehicle
CN106652465A (en) * 2016-11-15 2017-05-10 成都通甲优博科技有限责任公司 Method and system for identifying abnormal driving behavior on road
KR20190084594A (en) * 2018-01-08 2019-07-17 현대모비스 주식회사 Apparatus and method tracking object based on 3 dimension images
CN110178167A (en) * 2018-06-27 2019-08-27 潍坊学院 Crossing video frequency identifying method violating the regulations based on video camera collaboration relay
WO2020187257A1 (en) * 2019-03-18 2020-09-24 长城汽车股份有限公司 Vehicle abnormal lane change control method, device and system
CN110503083A (en) * 2019-08-30 2019-11-26 北京妙医佳健康科技集团有限公司 A kind of critical point detection method, apparatus and electronic equipment
CN111626189A (en) * 2020-05-26 2020-09-04 浙江大华技术股份有限公司 Road surface abnormity detection method and device, electronic equipment and storage medium

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112907949A (en) * 2021-01-20 2021-06-04 北京百度网讯科技有限公司 Traffic anomaly detection method, model training method and device
CN112907949B (en) * 2021-01-20 2022-11-22 北京百度网讯科技有限公司 Traffic anomaly detection method, model training method and device
CN113033471A (en) * 2021-04-15 2021-06-25 北京百度网讯科技有限公司 Traffic abnormality detection method, apparatus, device, storage medium, and program product
WO2022267387A1 (en) * 2021-06-21 2022-12-29 梅卡曼德（北京）机器人科技有限公司 Image recognition method and apparatus, electronic device, and storage medium
WO2023206236A1 (en) * 2022-04-28 2023-11-02 华为技术有限公司 Method for detecting target and related device
CN115909286A (en) * 2023-02-17 2023-04-04 浙江口碑网络技术有限公司 Data processing method and device

Similar Documents

Publication Publication Date Title
US9953245B2 (en) Multi-camera vehicle identification system
CN112700470B (en) Target detection and track extraction method based on traffic video stream
CN106462727A (en) Systems and methods for lane end recognition
JP2006208223A (en) Vehicle position recognition device and vehicle position recognition method
CN103116987A (en) Traffic flow statistic and violation detection method based on surveillance video processing
CN111932901A (en) Road vehicle tracking detection apparatus, method and storage medium
CN111160220B (en) Deep learning-based parcel detection method and device and storage medium
Zhang et al. A graded offline evaluation framework for intelligent vehicle’s cognitive ability
CN112613668A (en) Scenic spot dangerous area management and control method based on artificial intelligence
US11181911B2 (en) Control transfer of a vehicle
Malinovskiy et al. Model‐free video detection and tracking of pedestrians and bicyclists
Coronado et al. Detection and classification of road signs for automatic inventory systems using computer vision
CN112528944A (en) Image identification method and device, electronic equipment and storage medium
Yun et al. Video-based detection and analysis of driver distraction and inattention
CN112200044B (en) Abnormal behavior detection method and device and electronic equipment
Philipsen et al. Day and night-time drive analysis using stereo vision for naturalistic driving studies
CN112200044A (en) Abnormal behavior detection method and device and electronic equipment
Choi et al. State Machine and Downhill Simplex Approach for Vision‐Based Nighttime Vehicle Detection
TWI451990B (en) System and method for lane localization and markings
CN113223276B (en) Pedestrian hurdling behavior alarm method and device based on video identification
CN116152753A (en) Vehicle information identification method and system, storage medium and electronic device
CN113076821A (en) Event detection method and device
Dopfer et al. What can we learn from accident videos?
Fujiyoshi et al. Road observation and information providing system for supporting mobility of pedestrian
CN116912517B (en) Method and device for detecting camera view field boundary

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant