WO2021047306A1 - 一种异常行为判定方法、装置、终端及可读存储介质 - Google Patents

一种异常行为判定方法、装置、终端及可读存储介质 Download PDF

Info

Publication number
WO2021047306A1
WO2021047306A1 PCT/CN2020/104520 CN2020104520W WO2021047306A1 WO 2021047306 A1 WO2021047306 A1 WO 2021047306A1 CN 2020104520 W CN2020104520 W CN 2020104520W WO 2021047306 A1 WO2021047306 A1 WO 2021047306A1
Authority
WO
WIPO (PCT)
Prior art keywords
abnormal behavior
behavior
suspected
suspected abnormal
current state
Prior art date
Application number
PCT/CN2020/104520
Other languages
English (en)
French (fr)
Inventor
尹力
Original Assignee
中兴通讯股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中兴通讯股份有限公司 filed Critical 中兴通讯股份有限公司
Publication of WO2021047306A1 publication Critical patent/WO2021047306A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Definitions

  • the embodiment of the present invention relates to but not limited to the technical field of video surveillance, and specifically relates to but not limited to an abnormal behavior determination method, device, terminal and readable storage medium
  • the intelligent monitoring device obtains image data from the front end;
  • the back-end GPU cluster receives image data and performs detection through a convolutional neural network
  • the image data obtained by multiple cameras on the front end are all uninterruptedly sent to the back end high-definition image data, allowing the back end to perform information analysis and abnormal behavior detection. This not only increases the complexity of on-site deployment, but also requires more space and hardware resources, and the processing speed is slow.
  • the method, device, terminal, and readable storage medium for determining abnormal behaviors provided by the embodiments of the present invention are intended to solve at least to a certain extent, inaccurate determination of abnormal behaviors in some situations, large resource occupation, and slow processing speed. Technical issues.
  • an embodiment of the present invention provides an abnormal behavior determination method, including: acquiring behavior data in surveillance videos; determining at least one suspected abnormal behavior from the behavior data; acquiring the current state of the area where the suspected abnormal behavior is located Image; determine whether the suspected abnormal behavior is an abnormal behavior according to the current state image.
  • An embodiment of the present invention also provides an abnormal behavior determination device, which includes: at least one video monitoring module and a data processing module, wherein: the video monitoring module includes: a data acquisition module for acquiring behavior data in a surveillance video; data The preliminary processing module is used to determine at least one suspected abnormal behavior from the behavior data; the image acquisition module is used to obtain the current state image of the area where the suspected abnormal behavior is located; the data processing module is used to The state image determines whether the suspected abnormal behavior is an abnormal behavior.
  • the video monitoring module includes: a data acquisition module for acquiring behavior data in a surveillance video; data The preliminary processing module is used to determine at least one suspected abnormal behavior from the behavior data; the image acquisition module is used to obtain the current state image of the area where the suspected abnormal behavior is located; the data processing module is used to The state image determines whether the suspected abnormal behavior is an abnormal behavior.
  • the embodiment of the present invention also provides an abnormal behavior determination terminal, including a processor, a memory, and a communication bus; the communication bus is used to realize the connection and communication between the processor and the memory; the processor is used to execute the information stored in the memory
  • an abnormal behavior determination terminal including a processor, a memory, and a communication bus; the communication bus is used to realize the connection and communication between the processor and the memory; the processor is used to execute the information stored in the memory
  • One or more computer programs to implement the steps of the abnormal behavior determination method described in any one of the above.
  • the embodiment of the present invention also provides a readable storage medium, the readable storage medium stores one or more computer programs, and the one or more computer programs can be executed by one or more processors to realize the above Any one of the steps of the abnormal behavior determination method.
  • FIG. 1 is a schematic flowchart of a method for determining abnormal behavior in the background art of the present invention
  • FIG. 2 is a schematic flowchart of an abnormal behavior determination method according to Embodiment 1 of the present invention.
  • FIG. 3 is a schematic flowchart of a specific embodiment of an abnormal behavior determination method according to Embodiment 1 of the present invention.
  • FIG. 4 is a structural diagram of an abnormal behavior judging device provided by Embodiment 2 of the present invention.
  • FIG. 5 is a structural diagram of another abnormal behavior judging device provided by the second embodiment of the present invention.
  • FIG. 6 is a structural diagram of another abnormal behavior judging device provided by the second embodiment of the present invention.
  • FIG. 7 is a structural diagram of another abnormal behavior judging device provided by the second embodiment of the present invention.
  • FIG. 8 is a schematic structural diagram of a terminal provided in Embodiment 3 of the present invention.
  • an abnormal behavior determination method provided in this embodiment includes:
  • S202 Determine at least one suspected abnormal behavior from the behavior data
  • S204 Determine whether the suspected abnormal behavior is an abnormal behavior according to the current state image.
  • the abnormal behavior in this embodiment can be specific behaviors in certain scenarios, such as cheating in an examination room, taking pictures in a secret meeting room, lighting an open fire or smoking in a dangerous area, etc.; it can also be specific behaviors. Refers to behaviors, such as: running, waving, clapping, etc.; it can also be the facial expressions of some characters, such as frowning, pouting, and sticking out tongue.
  • the abnormal behavior can be determined by preset rules, or it can be determined by the user according to requirements.
  • the surveillance area of the surveillance video may be a school, a hospital, a square, a conference room, a park, a scenic spot, etc.
  • the surveillance area is not limited in this embodiment.
  • the behavior data in the surveillance video may be acquired in real time, or may be acquired at a certain time interval, for example, every 1 minute, the behavior data in the surveillance video is acquired. It may also be that the behavior data in the surveillance video is acquired in real time within a preset time period. For example, if the monitoring area is a conference room, it can be set to obtain real-time monitoring video behavior data in real time during the meeting time period, and stop obtaining it after the meeting.
  • the behavior data can include the position of a specific subject, the appearance of a specific object, motion data, light changes, color changes, and so on. For example, you can obtain a certain frame or a few frames or all frames in the surveillance video and analyze the position of each preset object as the behavior data. For example, analyze the position of the vehicle in a traffic intersection as the behavior data. When the position of a car exceeds the zebra crossing and there are pedestrians on the zebra crossing, the behavior data can be used to determine that the behavior of the vehicle crossing the zebra crossing is suspected to be abnormal; the brightness data of the image in the surveillance video can be obtained and analyzed as the behavior data, for example, In the kitchen surveillance video, the brightness data of each area in the surveillance screen is obtained.
  • the behavior in which the brightness of the non-stove area exceeds the preset range is a suspected abnormal behavior; for example, in a surveillance video of a square, images of multiple people moving are captured, and the moving state of each person can be used as behavior data. For example, in the surveillance video screen of a non-smoking place, the objects in the screen are identified, and each object is the behavior data. When the suspected smoke or lighter appears in the object, it is determined that there is suspected abnormal behavior in the screen.
  • the acquisition of specific behavior data can be set by those skilled in the art according to the characteristics of the abnormal behavior that needs to be determined.
  • the method further includes: normalizing the current state image.
  • normalizing the current state image includes but is not limited to adjusting the size of the current state image to a preset size.
  • the acquired current state image of the area where the abnormal behavior is located may be a picture , Its size may be 480*270 pixels.
  • normalizing the current state image also includes unifying the format of the current state image, for example, saving them as JPG format, mp4 format, etc.
  • performing normalization processing on the current state image further includes adjusting parameters such as brightness and contrast of the current state image.
  • the current state image is an image that includes a subject of suspected abnormal behavior.
  • the image is a "close-up" of the suspected abnormal behavior.
  • the acquisition of behavior data in the surveillance video may include multiple behavior executions.
  • the subject after the behavior data of a certain behavior execution subject is determined to be a suspected abnormal behavior, the area where the suspected abnormal behavior is located will be separately and clearly photographed to obtain an image of the current state.
  • the current status image is a complete image including the abnormal behavior and its execution subject.
  • the current state image is a complete image including abnormal behavior, for example, for suspected cases found in non-smoking places
  • the location information of the suspected smoking action will be obtained, and the camera PTZ and camera equipment will be adjusted according to the location information to take “close-up” shooting within a certain area of the location information to obtain the current state image .
  • the method further includes:
  • the coordinate information corresponding to the abnormal behavior is the coordinate information corresponding to the abnormal behavior.
  • the abnormal behavior alert information can be an alarm ringtone, a prompt box pops up on some preset interface, or the abnormal behavior information is sent to a designated terminal or server. In this embodiment, it is not Make a limit.
  • the coordinate information corresponding to the abnormal behavior may be the identification information in the unit of longitude and latitude, for example: the north latitude is 29.35 and the east longitude is 106.33; it may be the information in the unit of location, for example: No. XX, XX Road, XX District, XX City, XX province; It can also be the coordinate rule information set by the user, such as XX area, XX meeting room X row X block, XX ward XX bed, XX floor, XX box, etc.
  • determining whether the suspected abnormal behavior is an abnormal behavior according to the current state image includes:
  • the first convolutional neural network determines whether the suspected abnormal behavior is an abnormal behavior according to the current state image.
  • the corresponding relationship between the first convolutional neural network and the behavior category can be limited according to the needs of the user, which is not limited in this embodiment.
  • the suspected abnormal behavior can be classified as a classroom class
  • the first convolutional neural network corresponding to the behavior category classroom class of the suspected abnormal behavior can be set as the classroom depth
  • the convolutional neural classification network then sends and transmits the acquired current state image corresponding to the suspected abnormal behavior to the classroom deep convolutional neural classification network, and the classroom deep convolutional neural classification network determines the current state image after receiving it.
  • the first convolutional neural network includes a deep convolutional neural classification network.
  • it may include multiple behavior categories corresponding to different first convolutional neural networks, and multiple behavior categories may correspond to the same first convolutional neural network, and the number of first convolutional neural networks is This is not limited, users can set according to their needs.
  • the first convolutional neural network in the embodiment of the present invention is the first convolutional neural network obtained by performing deep learning of artificially annotated abnormal behavior samples of preset behavior categories on the pre-established convolutional neural network.
  • the first convolutional neural network can further determine whether the suspected abnormal behavior is an abnormal behavior through the acquired current state image.
  • acquiring the current state image of the area where the suspected abnormal behavior is located includes:
  • the shooting area of the current state image may be the current state image obtained by expanding the preset range to the surrounding or preset direction with the coordinates of the suspected abnormal behavior as the center.
  • the coordinate information of the suspected abnormal behavior is 1.2 meters high, 3 meters distance, and 1 meter wide.
  • the area targeted by the current state image can be 1-1.5 meters high, 3 meters away, and 0.8-1.3 meters wide. Range.
  • the current state image can be obtained by controlling the camera pan/tilt and then controlling the camera angle of the camera device, and/or by controlling the parameters of the camera device such as focal length, horizontal resolution, signal-to-noise ratio, etc. .
  • the coordinate position of the suspected abnormal behavior is exactly the same as the angle of the current camera device, and the current state image can be obtained only by adjusting the parameters of the camera device.
  • the camera equipment includes, but is not limited to, cameras and other equipment that can perform surveillance shooting.
  • the coordinate information can be used to control the camera pan/tilt to adjust the elevation angle, horizontal angle, and camera focal length, so that the camera can be aimed at the suspected abnormal behavior area to obtain targeted Current state image. For example: adjust the elevation and horizontal angle of the gimbal to the center point of the abnormal behavior area. Adjust the focus so that the camera can receive the image with the suspected abnormal behavior coordinate center as the image center as the current state image.
  • determining at least one suspected abnormal behavior from the behavior data includes:
  • the suspected abnormal behavior is taken as the target suspected abnormal behavior.
  • acquiring the current state image of the area where the suspected abnormal behavior is located includes:
  • the efficiency of abnormal behavior determination can be further improved and the resource occupation can be reduced.
  • the confidence calibration rules can be preset by the user according to certain logic, or can be calibrated based on the confidence rules specified by some calculation models.
  • general model algorithms such as logistic regression and decision tree may be used to infer the confidence of the suspected abnormal behavior.
  • the preset threshold can be a unified preset threshold for all scenarios, that is, for all behavior categories, or it can be set corresponding to different behavior categories, or the user can set the corresponding preset threshold at any time according to needs. Adjusted preset threshold. For example: For the suspected abnormal behavior of suspected possession of dangerous goods obtained from the behavior information obtained in the surveillance video at the station, the confidence level is 0.8, and the initial preset threshold value is 0.85. At this time, the suspected abnormal behavior will be excluded. It will not go to the next step for further analysis. However, the current security level requirements are increased, and the preset threshold is set to 0.79, then the suspected abnormal behavior will be listed as the target suspected abnormal behavior.
  • the suspected abnormal behavior with a confidence level above the preset threshold is obtained as the target suspected abnormal behavior, that is, when the confidence of the suspected abnormal behavior is equal to the preset threshold, it is also listed as the target Suspected abnormal behavior.
  • the setting of the target's suspected abnormal behavior can be changed according to the needs of the user.
  • the current state image of the area where the suspected abnormal behavior is obtained in the original method will be immediately changed to only obtain the suspected abnormal behavior of the target
  • the current status image of the area where it is determined to be suspected abnormal behavior but not the target suspected abnormal behavior will no longer be obtained to reduce the resource occupation of the system and increase the processing speed.
  • Obtaining the current state image of the area where the target is suspected of abnormal behavior includes:
  • the target's suspected abnormal behavior sequence is obtained by sorting the confidence of the suspected abnormal behavior corresponding to each target's suspected abnormal behavior according to the preset rule;
  • the current state image of the area where the suspected abnormal behavior of the target is located is obtained.
  • the confidence level of the target's suspected abnormal behavior is equal to the calibrated confidence level of the suspected abnormal behavior corresponding to the target's suspected abnormal behavior.
  • the calibrated confidence of the suspected abnormal behavior is 0.9
  • the preset threshold is 0.8. If the confidence of the suspected abnormal behavior is higher than the preset threshold, the suspected abnormal behavior is regarded as the target suspected abnormal behavior, and the target suspected abnormal behavior The confidence level is equal to the confidence level of the suspected abnormal behavior is equal to 0.9.
  • the predetermined rule in prioritizing the suspected abnormal behavior of each target according to the confidence level corresponding to the suspected abnormal behavior of each target according to the preset rule may be sorting according to the degree of confidence from high to low, It can also be based on the degree of confidence to sort the target suspected abnormal behaviors for each behavior category from high to low, and then sort them according to the priority of the behavior category. It should be noted that the suspected abnormal behaviors for targets that include multiple behavior categories The order of the confidence level sorting and the category priority sorting in the behavior sorting can be set according to user requirements, and this embodiment does not limit it. .
  • behavior category A the target's suspected abnormal behavior A has a confidence of 0.7, the target's suspected abnormal behavior B has a confidence of 0.75, and the target's suspected abnormal behavior C has a confidence of 0.79; behavior category B: the target's suspected abnormal behavior D
  • the confidence level is 0.7, the confidence level of the target's suspected abnormal behavior E is 0.75, and the confidence level of the target's suspected abnormal behavior F is 0.79.
  • the preset rule is that the priority of behavior category A is higher than that of B, and the higher the confidence, the higher the priority.
  • the result of prioritizing the suspected abnormal behavior of the above targets is: suspected abnormal behavior of the target C> suspected abnormal behavior of the target B> suspected abnormal behavior of the target A> suspected abnormal behavior of the target F> suspected abnormal behavior of the target E >The target is suspected of abnormal behavior D.
  • the preset rule may also be prioritized according to the degree of confidence from low to high to obtain the target suspected abnormal behavior sequence.
  • the current state image of the area corresponding to the coordinate information of the target suspected abnormal behavior can be obtained by separately controlling and adjusting at least one of the parameters of the camera pan-tilt and the camera device. It can be understood that, in some embodiments, at least one of the camera pan/tilt and camera equipment parameters is controlled and adjusted according to the target suspected abnormal behavior sequence of each target suspected abnormal behavior from the coordinate information of the target with the higher priority where the suspected abnormal behavior is located.
  • the current state image of the area including the coordinate information of the suspected abnormal behavior of the corresponding target.
  • the target suspected abnormal behavior A is ranked higher in the target suspected abnormal behavior sequence than the target suspected abnormal behavior B priority, first control and adjust at least one of the camera PTZ and camera equipment parameters to obtain the target suspected abnormal behavior The current state image of the area of the coordinate information where A is located, and then at least one of the camera pan/tilt and camera equipment parameters is controlled and adjusted to obtain the current state image of the area including the coordinate information of the target suspected abnormal behavior B.
  • the location information is compared with the current position of the camera, and the elevation and horizontal angle of the pan/tilt are adjusted to align with the center point of the abnormal behavior area. Adjust the focal length so that the camera can receive the image centered on the coordinate center of the suspected abnormal behavior to obtain the current state image.
  • the above steps can be repeated in the order of the target suspected abnormal behavior sequence of each target's suspected abnormal behaviors until all current state images of the target suspected abnormal behaviors are obtained.
  • the determination of at least one suspected abnormal behavior from the behavior data can be determined by the second convolutional neural network.
  • the second convolutional neural network can be directly arranged on the front-end camera. Perform calculations on the GPU loaded on the camera.
  • the second convolutional neural network is a small convolutional neural network.
  • the second convolutional neural network can determine suspected abnormal behaviors in all behavior categories.
  • the second convolutional neural network is obtained in the following manner, creating a full-category of abnormal behavior samples that have been manually labeled, and a preset full-category convolutional neural network learns the above-mentioned samples and performs confidence calibration
  • the convolutional neural network obtained after training, learning, and training is used as the second convolutional neural network.
  • the second convolutional neural network can calculate and analyze the acquired behavior data, filter out suspected abnormal behaviors, and calibrate the confidence of the suspected abnormal behaviors to obtain the location information of the suspected abnormal behaviors.
  • the behavior data in this embodiment may be human behavior data, animal behavior data, or some mechanical equipment and other things with actions.
  • behavioral data in the monitoring range of a target in a protected area including a giant panda climbing a tree and another koala in a daze under the tree.
  • a small convolutional neural network can be used to determine that the giant panda is climbing a tree as a suspected abnormal behavior A, and the koala is in a daze under the tree as a suspected abnormal behavior B.
  • a method for judging abnormal behavior of shooting a screen in a conference room includes:
  • S302 Determine at least two suspected screen shooting behaviors from the behavior data through the second convolutional neural network
  • S303 Perform confidence calibration on the suspected screen shooting behavior through the second convolutional neural network, and obtain the confidence of the suspected screen shooting behavior;
  • S305 Sort the suspected screen shooting behavior of each target according to the confidence of each target's suspected shooting screen behavior to obtain the target suspected shooting screen behavior sequence;
  • S307 Controlling and adjusting the parameters of the camera pan/tilt and/or the camera equipment to obtain the current state image of the area corresponding to the coordinate information according to the sequence of the target's suspected shooting screen behavior sequence;
  • S310 Obtain the first convolutional neural network corresponding to the behavior category of the suspected screen shooting behavior, and determine whether the suspected screen shooting behavior is the shooting screen behavior according to the current state image through the first convolutional neural network;
  • S312 Output coordinate information corresponding to the shooting behavior
  • step S311 there is no sequence limitation between step S311 and step S312 in the foregoing embodiment, and at least one step may also be selected for execution.
  • the suspected screen-shooting behavior of the target includes the suspected screen-shooting behavior where the confidence of the suspected screen-shooting behavior is higher than the preset threshold, and the confidence of the suspected screen-shooting behavior of the target is equal to the corresponding confidence of the suspected screen-shooting behavior. degree.
  • the behavior data in the surveillance video in the embodiment of the present invention can be obtained through a set of PTZ and shooting equipment, or through multiple sets of PTZ and shooting equipment.
  • the current state image corresponding to the suspected abnormal behavior can be obtained through the shooting device corresponding to the suspected abnormal behavior . It is also possible to obtain the current state image corresponding to the suspected abnormal behavior through other photographing devices that can obtain the area where the suspected abnormal behavior is located. For example, there are currently four shooting devices A, B, C, and D in the four directions of the classroom.
  • the behavior data of the first group of seats in the classroom can be obtained by the above-mentioned A shooting device, and the second group of classrooms can be obtained by the above-mentioned B shooting device.
  • the behavior data of the seat position, the behavior data of the third group of seat positions in the classroom is obtained through the above-mentioned C shooting device, and the behavior data of the fourth group of seat positions in the classroom is obtained through the above-mentioned D shooting device.
  • the second convolutional neural network is calculated and analyzed It is believed that the behavior data of the second group of seat positions in the classroom includes the suspected abnormal behavior of playing with a mobile phone in class.
  • the method preprocesses behavior data—determines the suspected abnormal behavior, and then specifically obtains the current state image for the area where the suspected abnormal behavior is located, the image captures the suspected abnormal behavior more clearly and accurately. Then, it is determined whether the suspected abnormal behavior is an abnormal behavior according to the current state image.
  • the abnormal behavior determination device 400 includes at least one video monitoring module 401 and a data processing module 402, wherein:
  • the video surveillance module 401 includes: a data acquisition module 4011 for acquiring behavioral data in surveillance videos; a data preliminary processing module 4012 for determining at least one suspected abnormal behavior from behavioral data; and an image acquiring module 4013 for acquiring suspected abnormalities The current state image of the area where the behavior is located,
  • the data processing module 402 is used to determine whether the suspected abnormal behavior is an abnormal behavior according to the current state image.
  • the abnormal behavior in this embodiment can be specific behaviors in certain scenarios, such as cheating in an examination room, taking pictures in a secret meeting room, lighting an open fire or smoking in a dangerous area, etc.; it can also be specific behaviors. Refers to the behavior, such as: running, waving, clapping, etc.; it can also be the facial expressions of some characters, such as frowning, pouting, and sticking out tongue.
  • the abnormal behavior can be determined by preset rules, or it can be determined by the user according to requirements.
  • the surveillance area of the surveillance video may be a school, a hospital, a square, a conference room, a park, a scenic spot, etc.
  • the surveillance area is not limited in this embodiment.
  • the behavior data in the surveillance video may be acquired in real time, or may be acquired at a certain time interval, for example, every 1 minute, the behavior data in the surveillance video is acquired. It may also be that the behavior data in the surveillance video is acquired within a preset time period. For example, if the monitoring area is a conference room, it can be set to acquire the behavior data in the surveillance video during the meeting time period, and stop acquiring it after the meeting.
  • the behavior data can include the position of a specific subject, the appearance of a specific object, motion data, light changes, color changes, and so on. For example, you can obtain a certain frame or a few frames or all frames in the surveillance video and analyze the position of each preset object as the behavior data. For example, analyze the position of the vehicle in a traffic intersection as the behavior data. When the position of a car exceeds the zebra crossing and there are pedestrians on the zebra crossing, the behavior data can be used to determine that the behavior of the vehicle crossing the zebra crossing is suspected to be abnormal; the brightness data of the image in the surveillance video can be obtained and analyzed as the behavior data, for example, In the kitchen surveillance video, the brightness data of each area in the surveillance screen is obtained.
  • the behavior in which the brightness of the non-stove area exceeds the preset range is a suspected abnormal behavior; for example, in a surveillance video of a square, images of multiple people moving are captured, and the moving state of each person can be used as behavior data. For example, in a surveillance video screen of a non-smoking place, the objects in the screen are identified, and each object is the behavior data. When a suspected smoke or a lighter appears in the object, it is determined that there is a suspected abnormal behavior in the screen.
  • the acquisition of specific behavior data can be set by those skilled in the art according to the characteristics of the abnormal behavior that needs to be determined.
  • the abnormal behavior determination device further includes a control data transmission module 501, and the control data transmission module 501 is used to normalize the current state image.
  • normalizing the current state image includes but is not limited to adjusting the size of the current state image to a preset size.
  • the current state image of the area where the abnormal behavior is acquired for the first time may be The size of the picture may be 480*270 pixels.
  • the size of the current state image can be adjusted to a unified preset size 240* 135 pixels.
  • normalizing the current state image also includes unifying the format of the current state image, for example, saving them as JPG format, mp4 format, etc.
  • performing normalization processing on the current state image further includes adjusting parameters such as brightness and contrast of the current state image.
  • the current state image is an image that includes a subject of suspected abnormal behavior.
  • the image is a "close-up" of the suspected abnormal behavior.
  • the acquisition of behavior data in the surveillance video may include multiple behavior executions.
  • the subject after the behavior data of a certain behavior execution subject is determined to be a suspected abnormal behavior, the area where the suspected abnormal behavior is located will be separately and clearly photographed to obtain an image of the current state.
  • the behavior data obtained in the surveillance video of the classroom includes the behavior data of 10 students in the classroom. Among them, the student Wang on the left side of the third row in the second group is standing. If the abnormal behavior data in this scene is standing behavior, then It was determined that Wang’s behavior was suspected to be abnormal, and only the left area of the third row of the second group was photographed, and the current state image of the left area of the third row of the second group was obtained.
  • the current status image is a complete image including the abnormal behavior and its execution subject. For example, in a meeting room of a secret meeting, a participant using a mobile phone to shoot a demo screen that is suspected of abnormal behavior will be used A certain frame or a few frames or a certain segment of the current surveillance video of the object photographed by the mobile phone is used as the current state image; in some embodiments, the current state image is a complete image including abnormal behavior, for example, for suspected cases found in non-smoking places The location information of smoking, and adjust the camera platform and camera equipment according to the location information to take “close-up” shooting within a certain range of the area where the location information is located, so as to obtain an image of the current state.
  • the data processing module is also used to:
  • the coordinate information corresponding to the abnormal behavior is the coordinate information corresponding to the abnormal behavior.
  • the abnormal behavior alert information can be an alarm ringtone, a prompt box pops up on some preset interface, or the abnormal behavior information is sent to a designated terminal or server. In this embodiment, it is not Make a limit.
  • the coordinate information corresponding to the abnormal behavior can be the identification information in the unit of latitude and longitude, for example: the north latitude is 29.35 and the east longitude is 106.33; it can be the information in the unit of location, for example: No. XX, XX Road, XX District, XX City, XX province; It can also be the coordinate rule information set by the user, such as XX area, XX meeting room X row X block, XX ward XX bed, XX floor, XX box, etc.
  • the data processing module further includes at least one first convolutional neural network; the preliminary data processing module is also used to obtain the behavior category of the suspected abnormal behavior;
  • the data processing module is also used to obtain a first convolutional neural network corresponding to the behavior category of the suspected abnormal behavior, and determine whether the suspected abnormal behavior is an abnormal behavior according to the current state image through the first convolutional neural network.
  • the first convolutional neural network includes a deep convolutional neural classification network
  • the classification of the first convolutional neural network may be limited according to the needs of users, which is not limited in this embodiment. For example, if the currently acquired suspected abnormal behavior is running in a classroom, the suspected abnormal behavior can be classified as a classroom class, and the deep convolutional neural classification network corresponding to the class of classroom class of the suspected abnormal behavior can be set as the depth of the classroom
  • the convolutional neural classification network then sends and transmits the acquired current state image corresponding to the suspected abnormal behavior to the classroom deep convolutional neural classification network, and the classroom deep convolutional neural classification network determines the current state image after receiving it.
  • it may include multiple behavior categories corresponding to different first convolutional neural networks, and multiple behavior categories may correspond to the same first convolutional neural network, and the number of first convolutional neural networks is This is not limited, users can set according to their needs.
  • the first convolutional neural network in the embodiment of the present invention is the first convolutional neural network obtained by performing deep learning of artificially annotated abnormal behavior samples of preset behavior categories on the pre-established convolutional neural network. Through the first convolutional neural network, it is possible to further determine whether the suspected abnormal behavior is an abnormal behavior according to the acquired current state image.
  • the data preliminary processing module 4012 is also used to obtain the confidence level of the suspected abnormal behavior
  • the abnormal behavior determination device 400 further includes a threshold determination module 601, and the threshold determination module 601 is further configured to use the suspected abnormal behavior as the target suspected abnormal behavior if the confidence level of the suspected abnormal behavior is greater than a preset threshold. behavior.
  • the image acquisition module 4013 is used to obtain the current state image of the area where the suspected abnormal behavior of the target is located.
  • the current state image can be obtained only for the suspected abnormal behavior of the target whose confidence level reaches a certain standard, and for the suspected abnormal behavior that does not meet the confidence level, subsequent judgments are not performed, which saves resource occupation and improves the judgment efficiency .
  • the shooting area of the current state image may be the current state image obtained by expanding the preset range to the surrounding or preset direction with the coordinates of the suspected abnormal behavior as the center.
  • the coordinate information of the suspected abnormal behavior is 1.2 meters high, 3 meters distance, and 1 meter wide.
  • the area targeted by the current state image can be 1-1.5 meters high, 3 meters away, and 0.8-1.3 meters wide. Range.
  • the current state image of the area including the coordinate information can be obtained by controlling the camera pan/tilt and then controlling the camera angle of the camera device, and/or by controlling the parameters of the camera device such as focal length, horizontal resolution, and information. Noise ratio and so on to obtain.
  • the coordinate position of the suspected abnormal behavior is exactly the same as the angle of the current camera device, and the current state image can be obtained only by adjusting the parameters of the camera device.
  • the camera equipment includes, but is not limited to, cameras and other equipment that can perform surveillance shooting.
  • the coordinate information can be used to control the camera pan/tilt to adjust the elevation angle, horizontal angle, and camera focal length, so that the camera can be aimed at the suspected abnormal behavior area to obtain targeted Current state image. For example: adjust the elevation and horizontal angle of the gimbal to the center point of the abnormal behavior area. Adjust the focus so that the camera can receive the image with the suspected abnormal behavior coordinate center as the image center as the current state image.
  • the abnormal behavior determination device further includes a sorting module 602.
  • the sorting module 602 is used to obtain the target suspected abnormal behavior sequence when the target suspected abnormal behavior exceeds one.
  • the sequence is obtained by sorting the confidence of the suspected abnormal behavior corresponding to each target's suspected abnormal behavior according to a preset rule; the image acquisition module 4013 obtains the current state image of the area where the target suspected abnormal behavior is located according to the sequence of the target suspected abnormal behavior sequence.
  • the confidence calibration rules can be preset by the user according to certain logic, or can be calibrated based on the confidence rules specified by some calculation models. Those skilled in the art can use available confidence calculation methods to calculate the confidence of the suspected abnormal behavior.
  • general model algorithms such as logistic regression and decision tree may be used to infer the confidence of the suspected abnormal behavior.
  • the preset threshold can be a unified preset threshold for all scenarios, that is, for all behavior categories, or it can be set corresponding to different behavior categories, or the user can set the corresponding preset threshold at any time according to needs. Adjusted preset threshold. For example: For the suspected abnormal behavior of suspected possession of dangerous goods obtained from the behavior information obtained in the surveillance video at the station, the confidence level is 0.8, and the initial preset threshold value is 0.85. At this time, the suspected abnormal behavior will be excluded. It will not go to the next step for further analysis. However, the current security level requirements are increased, and the preset threshold is set to 0.79, then the suspected abnormal behavior will be listed as the target suspected abnormal behavior.
  • the suspected abnormal behavior with a confidence level above the preset threshold is obtained as the target suspected abnormal behavior, that is, when the confidence of the suspected abnormal behavior is equal to the preset threshold, it is also listed as the target Suspected abnormal behavior.
  • the setting of the target's suspected abnormal behavior can be changed according to the needs of the user.
  • the current state image of the area where the suspected abnormal behavior is obtained in the original method will be immediately changed to only obtain the suspected abnormal behavior of the target
  • the current status image of the area where it is determined to be suspected abnormal behavior but not the target suspected abnormal behavior will no longer be obtained to reduce the resource occupation of the system and increase the processing speed.
  • the confidence level of the target's suspected abnormal behavior is equal to the calibrated confidence level of the suspected abnormal behavior corresponding to the target's suspected abnormal behavior.
  • the calibrated confidence of the suspected abnormal behavior is 0.9
  • the preset threshold is 0.8. If the confidence of the suspected abnormal behavior is higher than the preset threshold, the suspected abnormal behavior is regarded as the target suspected abnormal behavior, and the target suspected abnormal behavior The confidence level is equal to the confidence level of the suspected abnormal behavior is equal to 0.9.
  • the predetermined rule in prioritizing the suspected abnormal behavior of each target according to the confidence level corresponding to the suspected abnormal behavior of each target according to the preset rule may be sorting according to the degree of confidence from high to low, It can also be based on the degree of confidence to sort the target suspected abnormal behaviors for each behavior category from high to low, and then sort them according to the priority of the behavior category. It should be noted that the suspected abnormal behaviors for targets that include multiple behavior categories The order of the confidence level sorting and the category priority sorting in the behavior sorting can be set according to user requirements, and this embodiment does not limit it.
  • behavior category A the target's suspected abnormal behavior A has a confidence of 0.7, the target's suspected abnormal behavior B has a confidence of 0.75, and the target's suspected abnormal behavior C has a confidence of 0.79; behavior category B: the target's suspected abnormal behavior D
  • the confidence level is 0.7, the confidence level of the target's suspected abnormal behavior E is 0.75, and the confidence level of the target's suspected abnormal behavior F is 0.79.
  • the preset rule is that the priority of behavior category A is higher than that of B, and the higher the confidence, the higher the priority.
  • the result of prioritizing the suspected abnormal behavior of the above targets is: suspected abnormal behavior of the target C> suspected abnormal behavior of the target B> suspected abnormal behavior of the target A> suspected abnormal behavior of the target F> suspected abnormal behavior of the target E >The target is suspected of abnormal behavior D.
  • the preset rule may also be prioritized according to the degree of confidence from low to high to obtain the target suspected abnormal behavior sequence.
  • the current state image of the area corresponding to the coordinate information of the target suspected abnormal behavior can be obtained by separately controlling and adjusting at least one of the parameters of the camera pan-tilt and the camera device. It can be understood that, in some embodiments, at least one of the camera pan/tilt and camera equipment parameters is controlled and adjusted according to the target suspected abnormal behavior sequence of each target suspected abnormal behavior from the coordinate information of the target with the higher priority where the suspected abnormal behavior is located.
  • the current state image of the area including the coordinate information of the suspected abnormal behavior of the corresponding target.
  • the target suspected abnormal behavior A is ranked higher in the target suspected abnormal behavior sequence than the target suspected abnormal behavior B priority, first control and adjust at least one of the camera PTZ and camera equipment parameters to obtain the target suspected abnormal behavior The current state image of the area of the coordinate information where A is located, and then at least one of the camera pan/tilt and camera equipment parameters is controlled and adjusted to obtain the current state image of the area including the coordinate information of the target suspected abnormal behavior B.
  • the location information is compared with the current position of the camera, and the elevation and horizontal angle of the pan/tilt are adjusted to align with the center point of the abnormal behavior area. Adjust the focal length so that the camera can receive the image centered on the coordinate center of the suspected abnormal behavior to obtain the current state image.
  • the above steps can be repeated in the order of the target suspected abnormal behavior sequence of each target's suspected abnormal behaviors until all current state images of the target's suspected abnormal behaviors are obtained.
  • the data preliminary processing module includes a second convolutional neural network.
  • the second convolutional neural network includes a small convolutional neural network, which can be implemented by a small convolutional neural network arranged on the front-end camera. , Calculate directly on the GPU loaded by the camera.
  • the second convolutional neural network may be used to preliminarily determine the suspected abnormal behaviors of all behavior categories.
  • the second convolutional neural network is obtained in the following manner, creating a full-category of abnormal behavior samples that have been manually labeled, and a preset full-category convolutional neural network learns the above-mentioned samples and performs confidence calibration
  • the convolutional neural network obtained after training, learning, and training is used as the second convolutional neural network.
  • the second convolutional neural network can calculate and analyze the acquired behavior data, filter out suspected abnormal behaviors, and calibrate the confidence of the suspected abnormal behaviors to obtain the location information of the suspected abnormal behaviors.
  • the behavior data in this embodiment may be human behavior data, animal behavior data, or some mechanical equipment and other things with actions.
  • behavioral data in the monitoring range of a target in a protected area including a giant panda climbing a tree and another koala in a daze under the tree.
  • a small convolutional neural network can be used to determine that the giant panda is climbing a tree as a suspected abnormal behavior A, and the koala is in a daze under the tree as a suspected abnormal behavior B.
  • the abnormal behavior determination device provided by the present invention will be further described below in conjunction with a specific embodiment.
  • the abnormal behavior determination device 700 includes: at least one video monitoring module 701, a control data transmission module 702, a threshold value determination module 703, a sorting module 704, and a data processing module 705. Among them:
  • the video monitoring module 701 includes a data acquisition module 7011, a data preliminary processing module 7012, and an image acquisition module 7013.
  • One of the data acquisition module 7011 is mainly responsible for image acquisition and behavior data acquisition of the monitored area.
  • the behavior data includes but is not limited to images.
  • the data acquisition module 7011 transmits the image to the data preliminary processing module 7012.
  • the data preliminary processing module 7012 includes a small convolutional neural network at the front of the camera and a GPU. The received image is calculated through the small convolutional neural network and the GPU to output suspected abnormal behaviors.
  • the basic information of the personnel includes but is not limited to the personnel’s name, position, category, gender, age, job number and other information.
  • the small convolutional neural network has a high recall rate for abnormal behavior detection, and the difference of suspected abnormal behavior in a piece of behavior data is relatively high. For example, there are 5 suspected abnormal behaviors in a certain behavior data. This small convolutional neural network can find all 5 suspected abnormal behaviors.
  • the high recall rate is mainly because the small convolutional neural network used in the embodiment of the present invention is guaranteed by training and learning of abnormal behavior samples completed through a large number of manual annotations in the early stage.
  • the image acquisition module 7013 can complete the adjustment of the camera pan/tilt angle and camera focal length, and mainly receive coordinate information sent by the control module, and obtain clear image information for the area where the coordinate information is located by adjusting parameters such as the camera pan/tilt angle and focal length.
  • the small convolutional neural network is also used to obtain the degree of execution of suspected abnormal behaviors, the small convolutional neural network is used to calibrate the suspected abnormal behaviors, and the suspected abnormal behaviors including the confidence are transmitted to the threshold judgment module For further processing.
  • the abnormal behavior determination device 700 further includes a control data transmission module 702, a threshold determination module 703, and a sorting module 704.
  • the threshold determination module 703 receives the suspected abnormal behavior including the confidence level sent from the video surveillance module, and sorts the information
  • the module sorts the suspected abnormal behaviors according to their corresponding confidence levels, and sends the suspected abnormal behaviors larger than the preset threshold to the image acquisition module 7013.
  • the threshold here is a value between 0-1. A better value can be obtained through the test set of different scenarios. Generally, for particularly sensitive areas, a lower value can be selected to extract suspected areas with lower abnormal behavior confidence. , And send it to the image acquisition module 7013.
  • the image acquisition module 7013 receives the coordinate information of the suspected abnormal behavior sent by the threshold judgment module 703, and controls the camera pan-tilt to adjust the elevation angle, horizontal angle and camera focal length, so that the camera is aimed at the suspected abnormal behavior area to obtain targeted image data.
  • the control data transmission module receives the image for the suspected abnormal behavior area, then reduces the image to a size of 240*135 pixels, and then encodes and transmits it to the data processing module. It should be noted that the images can also be unified into PNG and other formats here.
  • the data processing module mainly includes high-performance GPU clusters and deep convolutional neural classification networks for abnormal behaviors.
  • This module receives the image sent by the control data transmission module, and uses the deep convolutional neural network to further confirm it. For the information confirming the abnormal behavior, the original image position is calculated back, and finally the alarm for confirming the abnormal behavior and the corresponding coordinate information are output.
  • the corresponding coordinate information output by the data processing module can be the coordinate information calculated by the data processing module, or the data processing module will receive the suspected abnormal behavior sent by the threshold judgment module included in the image.
  • the coordinate information is output directly.
  • the coordinate information corresponding to the suspected abnormal behavior may be calculated by the threshold judgment module, or calculated by a small convolutional neural network.
  • the abnormal behavior determination device may include multiple video monitoring modules, and the multiple video monitoring modules may correspond to the same or multiple data processing modules.
  • multiple video monitoring modules correspond to multiple threshold judgment modules, control data transmission modules, and sorting modules, and each threshold judgment module, control data transmission module, and sorting module correspond to the same data processing module.
  • each of the above modules is only an exemplary partition, in which the data processing module, the threshold judgment module, the control data transmission module, and the sorting module may belong to the same module.
  • the data processing module is located at the back end of the entire abnormal behavior determination device, and the video monitoring module and the control module are located at the front end of the abnormal behavior determination device.
  • the abnormal behavior determination device for a monitoring scene, such as a meeting room, a classroom, etc., only one data acquisition module, for example, a camera, needs to be arranged, which greatly reduces the complexity of installation. It greatly reduces the processing load of the data processing module and accelerates the speed of information processing: the data processing module does not need to process each frame of data acquired by the data acquisition module, but only processes some of the frames with the suspected abnormal behavior confidence higher than the preset threshold. For some of the processed frames, normalization is also carried out, which greatly reduces the amount of data, reduces the load of information transmission and data processing, and improves the accuracy of detection and judgment.
  • the accuracy of abnormal behavior detection is greatly improved: for such behavior data, it is divided into two steps to determine, including the first determination of the video monitoring module through the second convolutional neural network to select suspected abnormal behaviors, Then adjust the PTZ and camera parameters for the suspected abnormal behavior to obtain a clear current state image of the suspected abnormal behavior area, and control the data transmission module to normalize and transmit it to the data processing module.
  • the data processing module uses the targeted volume 1
  • the product neural network such as the deep convolutional neural classification network, detects and judges the current state image extracted in the first step again.
  • the current state image obtained in the second step is clearer and more targeted.
  • an abnormal behavior determination device which includes: at least one video monitoring module and a data processing module, where: the video monitoring module includes: a data acquisition module for acquiring behavior data in a surveillance video; a data preliminary processing module, At least one suspected abnormal behavior is determined from the behavior data; the image acquisition module obtains the current state image of the area where the suspected abnormal behavior is located; the data processing module is used to determine whether the suspected abnormal behavior is an abnormal behavior according to the current state image.
  • the video monitoring module includes: a data acquisition module for acquiring behavior data in a surveillance video; a data preliminary processing module, At least one suspected abnormal behavior is determined from the behavior data; the image acquisition module obtains the current state image of the area where the suspected abnormal behavior is located; the data processing module is used to determine whether the suspected abnormal behavior is an abnormal behavior according to the current state image.
  • This embodiment also provides an abnormal behavior determination terminal, as shown in FIG. 8, which includes a processor 81, a memory 83, and a communication bus 82, in which:
  • the communication bus 82 is used to implement connection and communication between the processor 81 and the memory 83;
  • the processor 81 is configured to execute one or more computer programs stored in the memory 83 to implement at least one step in the abnormal behavior determination method in the foregoing embodiments.
  • This embodiment also provides a computer-readable storage medium, which is included in any method or technology for storing information (such as computer-readable instructions, data structures, computer program modules, or other data). Volatile or non-volatile, removable or non-removable media.
  • Computer-readable storage media include but are not limited to RAM (Random Access Memory), ROM (Read-Only Memory, read-only memory), EEPROM (Electrically Erasable Programmable read only memory, charged Erasable Programmable Read-Only Memory) ), flash memory or other memory technology, CD-ROM (Compact Disc Read-Only Memory), digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, Or any other medium that can be used to store desired information and that can be accessed by a computer.
  • the computer-readable storage medium in this embodiment can be used to store one or more computer programs, and the stored one or more computer programs can be executed by a processor to implement at least one of the abnormal behavior determination methods in the foregoing embodiments step.
  • This embodiment also provides a computer program (or computer software).
  • the computer program can be distributed on a computer-readable medium and executed by a computable device to implement at least the abnormal behavior determination method in the foregoing embodiments.
  • One step; and in some cases, at least one step shown or described can be performed in an order different from that described in the foregoing embodiment.
  • This embodiment also provides a computer program product, including a computer readable device, and the computer readable device stores the computer program as shown above.
  • the computer-readable device in this embodiment may include the computer-readable storage medium as shown above.
  • the abnormal behavior determination method obtains behavior data in surveillance videos; determines at least one suspected abnormal behavior from the behavior data; and obtains information about the area where the suspected abnormal behavior is located.
  • Current state image determine whether the suspected abnormal behavior is abnormal based on the current state image. Since the method provided by the embodiment of the present invention determines the suspected abnormal behavior by preprocessing the behavior data, and then specifically obtains the current state image for the area where the suspected abnormal behavior is located, the image is more clear and accurate for the shooting of the suspected abnormal behavior, Then, it is determined whether the suspected abnormal behavior is abnormal behavior according to the current state image.
  • the accuracy of determining abnormal behavior is improved, resource occupation is reduced, and processing speed is improved.
  • communication media usually contain computer-readable instructions, data structures, computer program modules, or other data in a modulated data signal such as carrier waves or other transmission mechanisms, and may include any information delivery medium. Therefore, the present invention is not limited to any specific combination of hardware and software.

Abstract

一种异常行为判定方法、装置、终端及可读存储介质，该异常行为判定方法通过获取监控视频中的行为数据(S201)；从行为数据中确定至少一个疑似异常行为(S202)；获取疑似异常行为所在区域的当前状态图片(S203)；根据当前状态图片判定疑似异常行为是否是异常行为(S204)。

Description

一种异常行为判定方法、装置、终端及可读存储介质
相关申请的交叉引用
本申请基于申请号为201910854597.9、申请日为2019年9月10日的中国专利申请提出，并要求该中国专利申请的优先权，该中国专利申请的全部内容在此引入本申请作为参考。
技术领域
本发明实施例涉及但不限于视频监控技术领域，具体而言，涉及但不限于一种异常行为判定方法、装置、终端及可读存储介质
背景技术
随着人工智能技术在图像处理等方面的飞速发展，基于人工智能的视频监控技术在很多场合被广泛使用。在很多敏感区域例如会议室、考场、金融机构等地方，往往需要更多的智能摄像机监控设备来统计人员信息和检测异常行为。但是目前的智能监控设备从前端获取图像数据后，直接将原始图片传送给后端GPU集群通过卷积神经网络进行检测，其具体过程可参见图1，如图1所示，在目前的一些情形中，其判定过程为：
S101：智能监控设备从前端获取图像数据；
S102：后端GPU集群接收图像数据并通过卷积神经网络进行检测；
S103：输出结果。
利用该方法进行判定，有如下缺点：
1，对监控人员的异常行为判定不准确。由于监控摄像机分辨率不足；监控区域里人员较多且在图像中像素较小不够清晰；基于深度学习的异常行为检测模型单步检测效果不够好等原因，往往会造成检测信息错误，判定不准确。
2，资源占用较大，处理速度慢。为了获取更清晰可靠的数据，需要布置多个摄像头；前端多个摄像头获取的图像数据，全部不间断发送给后端高清图像数据，让后端进行信息分析和异常行为检测。这样不仅增加了现场布控的复杂度，而且还需要更多的空间资源和硬件资源，处理速度慢。
发明内容
本发明实施例提供的一种异常行为判定方法、装置、终端及可读存储介质，旨在至少在一定程度上解决在一些情形中对异常行为的判定不准确，资源占用较大，处理速度慢的技术问题。
有鉴于此，本发明实施例提供一种异常行为判定方法，包括：获取监控视频中的行为数据；从所述行为数据中确定至少一个疑似异常行为；获取所述疑似异常行为所在区域的当前状态图像；根据所述当前状态图像判定所述疑似异常行为是否是异常行为。
本发明实施例还提供了一种异常行为判定装置，包括：至少一个视频监控模块和数据处理模块，其中：所述视频监控模块包括：数据获取模块，用于获取监控视频中的行为数据；数据初步处理模块，用于从所述行为数据中确定至少一个疑似异常行为；图像获取模块，用于获取所述疑似异常行为所在区域的当前状态图像；所述数据处理模块，用于根据所述当前状态图像判定所述疑似异常行为是否是异常行为。
本发明实施例还提供了一种异常行为判定终端，包括处理器、存储器及通信总线；所述通信总线用于实现处理器和存储器之间的连接通信；所述处理器用于执行存储器中存储的一个或者多个计算机程序，以实现上述任一项所述的异常行为判定方法的步骤。
本发明实施例还提供了一种可读存储介质，所述可读存储介质存储有一个或者多个计算机程序，所述一个或者多个计算机程序可被一个或者多个处理器执行，以实现上述任一项所述的异常行为判定方法的步骤。
本发明其他特征和相应的有益效果在说明书的后面部分进行阐述说明，且应当理解，至少部分有益效果从本发明说明书中的记载变的显而易见。
附图说明
图1为本发明背景技术中的异常行为判定方法的流程示意图；
图2为本发明实施例一的一种异常行为判定方法的流程示意图；
图3为本发明实施例一的一种异常行为判定方法具体实施例的流程示意图；
图4为本发明实施例二提供的一种异常行为判定装置的结构图；
图5为本发明实施例二提供的另一种异常行为判定装置的结构图；
图6为本发明实施例二提供的另一种异常行为判定装置的结构图；
图7为本发明实施例二提供的另一种异常行为判定装置的结构图；
图8为本发明实施例三提供的终端的结构示意图。
具体实施方式
为了使本发明的目的、技术方案及优点更加清楚明白，下面通过具体实施方式结合附图对本发明实施例作进一步详细说明。应当理解，此处所描述的具体实施例仅仅用以解释本发明，并不用于限定本发明。
实施例一：
请参见图2，本实施例提供的一种异常行为判定方法包括：
S201：获取监控视频中的行为数据；
S202：从行为数据中确定至少一个疑似异常行为；
S203：获取疑似异常行为所在区域的当前状态图像；
S204：根据所当前状态图像判定疑似异常行为是否是异常行为。
需要说明的是，本实施例中的异常行为可以是在某些场景下特定的某些行为，例如：考场作弊、涉密会议室拍照、危险区域点明火或吸烟等；也可以是某些特指的行为，例如： 跑动、挥手、鼓掌等；还可以是一些人物的面部微表情，例如：皱眉、嘟嘴、吐舌头等。异常行为可以是通过预先设置规则所确定，也可以是用户根据需求进行确定。
在一些实施例中，监控视频的监控区域可以是学校、医院、广场、会议室、公园、景区等场所，其监控区域在本实施例中不做限定。
在一些实施例中，获取监控视频中的行为数据可以是实时获取，也可以是间隔一定时间进行获取，例如每隔1分钟，获取一次监控视频中的行为数据。还可以是在预设时间段内实时获取监控视频中的行为数据。例如监控区域为会议室，则可以设定，在开会时间段内实时获取实时监控视频中的行为数据，而在会议结束后，停止获取。
需要说明的是，行为数据可以包括特定主体的位置、特定事物的出现、运动数据、光线变化、色彩变化等。例如，可以通过获取监控视频中的某一帧或者某几帧或者全部帧的画面图像并分析其中的各预设事物的位置作为行为数据，例如分析交通路口中车辆的位置作为行为数据，当某一辆车的位置超出斑马线且斑马线上有行人时，则可以通过该行为数据确定该车辆超出斑马线的行为为疑似异常行为；通过获取并分析监控视频中画面图像的亮度数据作为行为数据，例如，在厨房的监控视频中，获取到监控画面中的各区域亮度数据，若某一非灶台区域的亮度超出预设范围，则认为该区域存在疑似着火的可能性，则可以通过该行为数据确定非灶台区域的亮度超出预设范围的行为为疑似异常行为；例如，在广场的监控视频中，拍摄到多个人在移动的图像，则可以将每个人移动的状态作为行为数据。例如，在禁烟场所的监控视频画面中，识别画面中存在的事物，各事物即为行为数据，当事物中出现疑似烟或者疑似打火机时，则确定该画面中存在疑似异常行为。具体的行为数据的获取可以是本领域技术人员根据所需要确定的异常行为的特性而设定的。
在一些实施例中，获取疑似异常行为所在区域的当前状态图像之后，根据当前状态图像判定疑似异常行为是否是异常行为之前还包括：对当前状态图像进行归一化处理。
需要说明的是，对当前状态图像进行归一化处理包括但不限于对当前状态图像的尺寸进行调整，将其调整为预设尺寸，例如获取到的异常行为所在区域的当前状态图像可能是图片，其尺寸大小可能是480*270像素大小，为使后续对该当前状态图像的处理更加快速，占用资源更加小，则可以将该当前状态图像的尺寸大小调整为统一的预设尺寸240*135像素。在一些实施例中，对当前状态图像进行归一化处理还包括对当前状态图像的格式进行统一，例如，将其均另存为JPG格式、mp4格式等。在一些实施例中，对当前状态图像进行归一化处理还包括对当前状态图像的亮度、对比度等参数进行调整。
在一些实施例中，当前状态图像是包括疑似异常行为主体的图像，该图像是推疑似异常行为的“特写”，在一些实施例中，由于获取监控视频中的行为数据可能包括多个行为执行主体，在某一行为执行主体的行为数据被判定为疑似异常行为后，将对该疑似异常行为所在的区域进行单独、加清晰的拍摄，以获取当前状态图像。在一些实施例中，当前状态图像是包括异常行为及其执行执行主体的完整图像，例如针对密会议的会议室中，有参会 人员使用手机拍摄演示屏幕这一疑似异常行为，将会获取使用手机进行拍摄的对象的当前监控视频的某一帧或某几帧或某段视频作为当前状态图像；在一些实施例中，当前状态图像是包括异常行为的完整图像，例如针对在禁烟场所发现疑似吸烟这一疑似异常行为后，将会获取疑似吸烟动作的位置信息，并根据该位置信息调整摄像云台和摄像设备对该位置信息所在区域一定范围内进行“特写”拍摄，以获得当前状态图像。
在一些实施例中，根据当前状态图像判定疑似异常行为是否是异常行为之后还包括：
若判定疑似异常行为是异常行为，输出以下信息中至少之一：
异常行为警报信息；
异常行为对应的坐标信息。
需要说明的是，异常行为警报信息可以是发出警报铃声，也可以是在某些预设界面弹出提示框，还可以是将异常行为信息发送至指定的终端或服务器等，在本实施例中不做限定。
需要说明的是，异常行为对应的坐标信息可以是以经纬度为单位的标识信息，例如：北纬29.35东经106.33；可以是以地点为单位的信息，例如：XX省XX市XX区XX路XX号；还可以是用户自行设定的坐标规则信息，例如XX区、XX会议室X排X座、XX病房XX床、XX楼层、XX包厢等。
在一些实施例中，根据当前状态图像判定疑似异常行为是否是异常行为包括：
获取疑似异常行为所在的行为类别；
获取与疑似异常行为所在的行为类别对应的第一卷积神经网络；
通过第一卷积神经网络根据当前状态图像判定疑似异常行为是否是异常行为。
需要说明的，第一卷积神经网络与行为类别的对应关系可以根据用户的需求进行限定，本实施例中不做限定。例如，当前获取的疑似异常行为为在教室中跑动，可以将该疑似异常行为分类为教室类，可以设定与疑似异常行为所在的行为类别教室类对应的第一卷积神经网络为教室深度卷积神经分类网络，进而将该疑似异常行为所对应获取的当前状态图像发送传输给该教室深度卷积神经分类网络，教室深度卷积神经分类网络接收到该当前状态图像后对其进行判定。在一些实施例中，第一卷积神经网络包括深度卷积神经分类网络。
在一些实施例中，可以包括多个行为类别分别一一对应不同的第一卷积神经网络，还可以多个行为类别对应同一个第一卷积神经网络，第一卷积神经网络的数量在此不做限定，用户可以根据需要设定。
本发明实施例中的第一卷积神经网络是通过对预先建立的卷积神经网络进行预设行为类别的经人工标注完成的异常行为样本深度学习完成后得到的第一卷积神经网络。该第一卷积神经网络可以通过获取到的当前状态图像进一步判定疑似异常行为是否是异常行为。
在一些实施例中，获取疑似异常行为所在区域的当前状态图像包括：
获取疑似异常行为所在的坐标信息；
获取该疑似异常行为所在的坐标信息所对应的区域的当前状态图像。
需要说明的是，此时当前状态图像的拍摄区域可以是以疑似异常行为所在的坐标为中心，向四周或预设的方向扩张预设范围后得到的当前状态图像。例如：获取到疑似异常行为所在的坐标信息为高1.2米，距离3米，宽1米，此时当前状态图像所针对的区域可以是高1-1.5米，距离3米，宽0.8-1.3米的范围。
在一些实施例中，当前状态图像可以通过控制调整摄像机云台进而控制摄像设备的摄像角度来获取，和/或，通过控制摄像设备的参数例如焦距、水平分辨率、信噪比等来来获取。有时疑似异常行为的坐标位置与当前摄像设备的角度刚好一致，则可以仅通过调整摄像设备的参数来获取当前状态图像。需要说明的是，摄像设备包括但不限于摄像机等可以进行监控拍摄的设备。
在一些实施例中，在获取到异常行为所在的坐标信息后，可以对照该坐标信息来控制摄像机云台调整仰角、水平角度和摄像机焦距，使摄像机对准疑似异常行为的区域来获取针对性的当前状态图像。例如：调整云台仰角和水平角度对准异常行为区域中心点。调整焦距让摄像机可以接收以疑似异常行为坐标中心点为图像中心的图像作为当前状态图像。
在一些实施例中，从行为数据中确定至少一个疑似异常行为包括：
获取疑似异常行为的置信度；
若疑似异常行为的置信度大于预设阈值，则将疑似异常行为作为目标疑似异常行为。
在一些实施例中，获取疑似异常行为所在区域的当前状态图像包括：
获取目标疑似异常行为所在区域的当前状态图像。
通过在疑似异常行为中根据置信度来筛选出目标疑似异常行为，并针对目标疑似异常行为才去获取其当前状态图像，可以进一步提升异常行为判定的效率，降低资源占用。
需要说明的是，置信度标定的规则可以是用户根据一定逻辑进行预设的，也可以是基于一些计算模型所规定的置信度规则进行标定。
在一些实施例中，可以通过逻辑回归、决策树等通用的模型算法推测疑似异常行为的置信度。
需要说明的是，预设阈值可以是针对全部场景也即针对全部行为类别均设置统一的预设阈值，也可以是根据不同的行为类别分别设置对应的预设阈值，还可以是用户根据需要随时调整的预设阈值。例如：针对在车站获取监控视频中的行为信息中获取到疑似持有危险物品这一疑似异常行为，其置信度为0.8，按照初始预设阈值为0.85，此时该疑似异常行为将被排除，不会进入下一步骤，对其进行进一步分析。但当前安保等级要求提升，预设阈值设置为0.79，则，该疑似异常行为将被列为目标疑似异常行为。
需要说明的是，在一些实施例中，获取置信度在预设阈值以上的疑似异常行为作为目标疑似异常行为，也即，疑似异常行为的置信度等于预设阈值时，也将其列为目标疑似异 常行为。目标疑似异常行为的设定可以根据用户的需要进行更换。
需要说明的是，当异常行为判定方法中存在从疑似异常行为中确定目标疑似异常行为的步骤时，原方法中的获取疑似异常行为所在区域的当前状态图像将随即更改换仅获取目标疑似异常行为所在区域的当前状态图像，对于那些被确定为是疑似异常行为，但不是目标疑似异常行为将不再获取其所在区域的当前状态图像，以降低系统的资源占用，提升处理速度。
在一些实施例中，当目标疑似异常行为的数量超过1个时，
获取目标疑似异常行为所在区域的当前状态图像包括：
获取目标疑似异常行为序列，目标疑似异常行为序列由各目标疑似异常行为所对应的疑似异常行为的置信度按照预设规则排序得到；
根据目标疑似异常行为序列的顺序获取目标疑似异常行为所在区域的当前状态图像。
需要说明的是，目标疑似异常行为的置信度等于该目标疑似异常行为所对应的疑似异常行为所被标定的置信度。例如疑似异常行为所标定的置信度为0.9，预设阈值为0.8，则该疑似异常行为的置信度高于预设阈值，则区该疑似异常行为作为目标疑似异常行为，该目标疑似异常行为的置信度等于疑似异常行为的置信度等于0.9。
在一些实施例中，根据各目标疑似异常行为所对应的置信度根据预设规则对各目标疑似异常行为进行优先级排序中的预设规则可以是按照置信度的大小从高到低进行排序，也可以是根据置信度的大小针对每个行为类别的目标疑似异常行为进行从高到低排序，再根据行为类别的优先级进行排序，需要说明的是，针对包括多个行为类别的目标疑似异常行为的排序中置信度大小排序与类别优先级排序的顺序可以根据用户需求自行设定，本实施例不做限定。。例如：当前有行为类别甲类：目标疑似异常行为A的置信度0.7，目标疑似异常行为B的置信度0.75，目标疑似异常行为C的置信度0.79；行为类别乙类：目标疑似异常行为D的置信度0.7，目标疑似异常行为E的置信度0.75，目标疑似异常行为F的置信度0.79。其中预设规则是行为类别甲类优先级高于乙类，置信度越高优先级越高。则按照此预设规则对上述各目标疑似异常行为进行优先级排序的结果就是：目标疑似异常行为C>目标疑似异常行为B>目标疑似异常行为A>目标疑似异常行为F>目标疑似异常行为E>目标疑似异常行为D。
需要说明的是，在一些实施例中，预设规则也可以是按照置信度由低到高来进行优先级排序，得到目标疑似异常行为序列。
需要说明的是，根据目标疑似异常行为序列可以通过分别控制调整摄像机云台和摄像设备的参数中至少之一来获取目标疑似异常行为所在的坐标信息所对应的区域的当前状态图像。可以理解，在一些实施例中，根据各目标疑似异常行为的目标疑似异常行为序列从优先级高的的目标疑似异常行为所在的坐标信息控制调整摄像机云台和摄像设备参数中至少之一来获取包括相应目标疑似异常行为所在的坐标信息的区域的当前状态图像。例 如，目标疑似异常行为A的在目标疑似异常行为序列中的排序高于目标疑似异常行为B的优先级，则先控制调整摄像机云台和摄像设备参数中至少之一来获取包括目标疑似异常行为A所在的坐标信息的区域的当前状态图像，再控制调整摄像机云台和摄像设备参数中至少之一来获取包括目标疑似异常行为B所在的坐标信息的区域的当前状态图像。
在一些实施例中，获取到目标疑似异常行为的位置信息后，将该位置信息于摄像机当前位置进行对比，调整调整云台仰角和水平角度对准异常行为区域中心点。调整焦距让摄像机可以接收以疑似异常行为坐标中心点为图像中心的图像来获取当前状态图像。对于当前存在多个目标疑似异常行为时，可以按照各目标疑似异常行为的目标疑似异常行为序列的顺序重复上述步骤，直到获取到所有的目标疑似异常行为的当前状态图像。
在一些实施例中，从行为数据中确定至少一个疑似异常行为可以通过第二卷积神经网络来进行确定，在一些实施例中，可以通过在前端摄像机上布置的第二卷积神经网络，直接在摄像机加载的GPU上进行计算。在一些实施例中，第二卷积神经网络为小型卷积神经网络。
需要说明的是，在一些实施例中，第二卷积神经网络可以确定全部行为类别的疑似异常行为。
在一些实施例中，第二卷积神经网络通过以下方式获取，建立人工标注完成的全类别的异常行为样本，预先设置的全类别的卷积神经网络对上述样本进行学习，并进行置信度标定训练，学习、训练完成后得到的卷积神经网络作为第二卷积神经网络。该第二卷积神经网络可以对获取到的行为数据进行计算分析，从中筛选出疑似异常行为，并标定该疑似异常行为的置信度，得到该疑似异常行为的位置信息。
需要说明的是，本实施例中的行为数据可以是人的行为数据，也可以是动物的行为数据，还可以是一些机械设备等具有动作的事物。例如，可以获取保护区中目标监控范围中的行为数据，其中包括一只大熊猫在爬树，另一只考拉在树下发呆。此时可以通过小型卷积神经网络确定大熊猫在爬树是疑似异常行为A，考拉在树下发呆为疑似异常行为B。又例如，工厂中某传送带正常工作是以5米每秒的速度在运转，但在监控视频中的行为数据中提取到该传送带的状态时静止不动的，则可以确定该传送带静止为疑似异常行为。
下面结合一种具体的实施例对上述实施例中的异常行为判定方法进行进一步的说明。参见图3，如图3所示，提供了一种会议室中拍摄屏幕这一异常行为判定方法，该方法包括：
S301：获取针对会议室的监控视频中的行为数据；
S302：通过第二卷积神经网络从行为数据中确定至少两个疑似拍摄屏幕行为；
S303：通过第二卷积神经网络对疑似拍摄屏幕行为进行置信度标定，获取到疑似拍摄屏幕行为的置信度；
S304：获取至少两个目标疑似拍摄屏幕行为；
S305：根据各目标疑似拍摄屏幕行为的置信度对各目标疑似拍摄屏幕行为进行排序，得到目标疑似拍摄屏幕行为序列；
S306：获取各目标疑似异常行为所在的坐标信息；
S307：控制调整摄像机云台和/或摄像设备的参数按照目标疑似拍摄屏幕行为序列的顺序来获取其坐标信息所对应的区域的当前状态图像；
S308：对当前状态图像进行归一化处理；
S309：获取目标疑似拍摄屏幕行为所在的行为类别；
S310：获取与疑似拍摄屏幕行为所在的行为类别对应的第一卷积神经网络，通过第一卷积神经网络根据当前状态图像判定疑似拍摄屏幕行为是否是拍摄屏幕行为；
S311：若是，则输出拍摄行为警报信息；
S312：输出拍摄行为对应的坐标信息；
S313：若否，则结束流程。
需要说明的是，上述实施例中步骤S311和步骤S312之间没有顺序限定，且也可以选择至少一个步骤执行。
需要说明的是，其中目标疑似拍摄屏幕行为包括疑似拍摄屏幕行为的置信度高于预设阈值的疑似拍摄屏幕行为，且目标疑似拍摄屏幕行为的置信度等于其所对应的疑似拍摄屏幕行为的置信度。
需要说明的是，行为数据可以是监控视频中的每一帧。确定疑似异常行为可以通过小型卷积神经网络来进行。
在一些实施例中，本发明实施例中获取监控视频中的行为数据可以是通过一套云台和拍摄设备进行获取，也可以是通过多套云台和拍摄设备进行获取，当通过多套云台和拍摄设备来获取多个行为数据时，当从多个行为数据中获取到疑似异常行为数据时，可以通过与疑似异常行为所对应的拍摄设备来获取该疑似异常行为所对应的当前状态图像，也可以通过能够获取到疑似异常行为所在区域的其他拍摄设备来获取该疑似异常行为所对应的当前状态图像。例如，当前在教室的四个方位存在四个拍摄设备A、B、C、D，通过上述A拍摄设备获取到教室第一组座位位置的行为数据，通过上述B拍摄设备获取到教室第二组座位位置的行为数据，通过上述C拍摄设备获取到教室第三组座位位置的行为数据，通过上述D拍摄设备获取到教室第四组座位位置的行为数据，第二卷积神经网络经过计算分析后认为教室第二组座位位置的行为数据中包括疑似上课玩手机这一疑似异常行为，此时，需要采集疑似上课玩手机这一疑似异常行为所在区域的当前状态图像以进一步确认是否是上课玩手机这一行为。对于该当前状态图像，可以通过先获取疑似上课玩手机这一行为更加精确的位置信息，假设为第二组第三排，此时，可以选择调整四个拍摄设备中至少一个拍摄设备的拍摄角度等参数来获取第二组第三排的“特写”，例如仅拍摄第二组第三排区域，以获取至少一张当前状态图像。
本发明实施例提供的异常行为判定方法，通过获取监控视频中的行为
数据；从行为数据中确定至少一个疑似异常行为；获取疑似异常行为所在区域的当前状态图像；接收当前状态图像并判定疑似异常行为是否是异常行为。由于本发明实施例提供的方法存在对行为数据的预处理—确定疑似异常行为，再有针对性的针对疑似异常行为所在的区域获取当前状态图像，该图像对疑似异常行为的拍摄更加清晰准确，再根据该当前状态图像判定疑似异常行为是否是异常行为。通过本方法的实施，可以提升对异常行为判定准确的准确性，降低了资源占用，提升了处理速度。
实施例二：
本实施例还提供了一种异常行为判定装置，如图4所示，异常行为判定装置400其包括至少一个视频监控模块401，和数据处理模块402，其中：
视频监控模块401包括：数据获取模块4011，用于获取监控视频中的行为数据；数据初步处理模块4012，用于从行为数据中确定至少一个疑似异常行为；图像获取模块4013，用于获取疑似异常行为所在区域的当前状态图像，
数据处理模块402，用于根据当前状态图像判定疑似异常行为是否是异常行为。
需要说明的是，本实施例中的异常行为可以是在某些场景下特定的某些行为，例如：考场作弊、涉密会议室拍照、危险区域点明火或吸烟等；也可以是某些特指的行为，例如：跑动、挥手、鼓掌等；还可以是一些人物的面部微表情，例如：皱眉、嘟嘴、吐舌头等。异常行为可以是通过预先设置规则所确定，也可以是用户根据需求进行确定。
在一些实施例中，监控视频的监控区域可以是学校、医院、广场、会议室、公园、景区等场所，其监控区域在本实施例中不做限定。
在一些实施例中，获取监控视频中的行为数据可以是实时获取，也可以是间隔一定时间进行获取，例如每隔1分钟，获取一次监控视频中的行为数据。还可以是在预设时间段内获取监控视频中的行为数据。例如监控区域为会议室，则可以设定，在开会时间段内获取监控视频中的行为数据，而在会议结束后，停止获取。
需要说明的是，行为数据可以包括特定主体的位置、特定事物的出现、运动数据、光线变化、色彩变化等。例如，可以通过获取监控视频中的某一帧或者某几帧或者全部帧的画面图像并分析其中的各预设事物的位置作为行为数据，例如分析交通路口中车辆的位置作为行为数据，当某一辆车的位置超出斑马线且斑马线上有行人时，则可以通过该行为数据确定该车辆超出斑马线的行为为疑似异常行为；通过获取并分析监控视频中画面图像的亮度数据作为行为数据，例如，在厨房的监控视频中，获取到监控画面中的各区域亮度数据，若某一非灶台区域的亮度超出预设范围，则认为该区域存在疑似着火的可能性，则可以通过该行为数据确定非灶台区域的亮度超出预设范围的行为为疑似异常行为；例如，在广场的监控视频中，拍摄到多个人在移动的图像，则可以将每个人移动的状态作为行为数据。例如，在禁烟场所的监控视频画面中，识别画面中存在的事物，各事物即为行为数据， 当事物中出现疑似烟或者疑似打火机时，则确定该画面中存在疑似异常行为。具体的行为数据的获取可以是本领域技术人员根据所需要确定的异常行为的特性而设定的。
在一些实施例中，如图5所示，异常行为判定装置还包括控制数据传输模块501，控制数据传输模块501用于对当前状态图像进行归一化处理。
需要说明的是，对当前状态图像进行归一化处理包括但不限于对当前状态图像的尺寸进行调整，将其调整为预设尺寸，例如初次获取到的异常行为所在区域的当前状态图像可能是图片，其尺寸大小可能是480*270像素大小，为使后续对该当前状态图像的处理更加快速，占用资源更加小，则可以将该当前状态图像的尺寸大小调整为统一的预设尺寸240*135像素。在一些实施例中，对当前状态图像进行归一化处理还包括对当前状态图像的格式进行统一，例如，将其均另存为JPG格式、mp4格式等。在一些实施例中，对当前状态图像进行归一化处理还包括对当前状态图像的亮度、对比度等参数进行调整。
在一些实施例中，当前状态图像是包括疑似异常行为主体的图像，该图像是推疑似异常行为的“特写”，在一些实施例中，由于获取监控视频中的行为数据可能包括多个行为执行主体，在某一行为执行主体的行为数据被判定为疑似异常行为后，将对该疑似异常行为所在的区域进行单独、加清晰的拍摄，以获取当前状态图像。例如，在教室的监控视频中获取的行为数据包括教室中10个学生的动作数据，其中第二组第三排左侧的王同学处于站立状态，假使该场景下异常行为数据为站立行为，则判定王同学的行为为疑似异常行为，并仅针对对第二组第三排左侧区域进行拍摄，获取第二组第三排左侧区域的当前状态图像。
在一些实施例中，当前状态图像是包括异常行为及其执行执行主体的完整图像，例如针对密会议的会议室中，有参会人员使用手机拍摄演示屏幕这一疑似异常行为，将会获取使用手机进行拍摄的对象的当前监控视频的某一帧或某几帧或某段视频作为当前状态图像；在一些实施例中，当前状态图像是包括异常行为的完整图像，例如针对在禁烟场所发现疑似吸烟的位置信息，并根据该位置信息调整摄像云台和摄像设备对该位置信息所在区域一定范围内进行“特写”拍摄，以获得当前状态图像。
在一些实施例中，数据处理模块还用于：
若判定疑似异常行为是异常行为，输出以下信息中至少之一：
异常行为警报信息；
异常行为对应的坐标信息。
需要说明的是，异常行为警报信息可以是发出警报铃声，也可以是在某些预设界面弹出提示框，还可以是将异常行为信息发送至指定的终端或服务器等，在本实施例中不做限定。
需要说明的是，异常行为对应的坐标信息可以是以经纬度为单位的标识信息，例如：北纬29.35东经106.33；可以是以地点为单位的信息，例如：XX省XX市XX区XX路XX号；还可以是用户自行设定的坐标规则信息，例如XX区、XX会议室X排X座、XX 病房XX床、XX楼层、XX包厢等。
在一些实施例中，数据处理模块还包括至少一个第一卷积神经网络；数据初步处理模块还用于获取疑似异常行为所在的行为类别；
数据处理模块还用于获取与疑似异常行为所在的行为类别对应的第一卷积神经网络，通过第一卷积神经网络根据当前状态图像判定疑似异常行为是否是异常行为。
需要说明的，第一卷积神经网络包括深度卷积神经分类网络，第一卷积神经网络的分类可以根据用户的需求进行限定，本实施例中不做限定。例如，当前获取的疑似异常行为为在教室中跑动，可以将该疑似异常行为分类为教室类，可以设定与疑似异常行为所在的行为类别教室类对应的深度卷积神经分类网络为教室深度卷积神经分类网络，进而将该疑似异常行为所对应获取的当前状态图像发送传输给该教室深度卷积神经分类网络，教室深度卷积神经分类网络接收到该当前状态图像后对其进行判定。
在一些实施例中，可以包括多个行为类别分别一一对应不同的第一卷积神经网络，还可以多个行为类别对应同一个第一卷积神经网络，第一卷积神经网络的数量在此不做限定，用户可以根据需要设定。
本发明实施例中的第一卷积神经网络是通过对预先建立的卷积神经网络进行预设行为类别的经人工标注完成的异常行为样本深度学习完成后得到的第一卷积神经网络。通过该第一卷积神经网络可以根据获取到的当前状态图像进一步判定疑似异常行为是否是异常行为。
在一些实施例中，数据初步处理模块4012，还用于获取疑似异常行为的置信度；
如图6所示，异常行为判定装置400还包括阈值判断模块601，阈值判断模块601还用于若所述疑似异常行为的置信度大于预设阈值，则将所述疑似异常行为作为目标疑似异常行为。
在一些实施例中，当阈值判断模块601确定了目标疑似异常行为后，图像获取模块4013，用于获取目标疑似异常行为所在区域的当前状态图像。
此时，可以仅针对置信度达到一定标准的目标疑似异常行为进行获取当前状态图像，而对于置信度未达标的疑似异常行为，则不再进行后续的判断，节约的资源占用，提升了判定效率。
需要说明的是，此时当前状态图像的拍摄区域可以是以疑似异常行为所在的坐标为中心，向四周或预设的方向扩张预设范围后得到的当前状态图像。例如：获取到疑似异常行为所在的坐标信息为高1.2米，距离3米，宽1米，此时当前状态图像所针对的区域可以是高1-1.5米，距离3米，宽0.8-1.3米的范围。
在一些实施例中，包括坐标信息的区域的当前状态图像可以通过控制调整摄像机云台进而控制摄像设备的摄像角度来获取，和/或，通过控制摄像设备的参数例如焦距、水平分辨率、信噪比等来来获取。有时疑似异常行为的坐标位置与当前摄像设备的角度刚好一致， 则可以仅通过调整摄像设备的参数来获取当前状态图像。需要说明的是，摄像设备包括但不限于摄像机等可以进行监控拍摄的设备。
在一些实施例中，在获取到异常行为所在的坐标信息后，可以对照该坐标信息来控制摄像机云台调整仰角、水平角度和摄像机焦距，使摄像机对准疑似异常行为的区域来获取针对性的当前状态图像。例如：调整云台仰角和水平角度对准异常行为区域中心点。调整焦距让摄像机可以接收以疑似异常行为坐标中心点为图像中心的图像作为当前状态图像。
在一些实施例中，如图6所示，异常行为判定装置还包括排序模块602，排序模块602用于当目标疑似异常行为的数量超过1个时，获取目标疑似异常行为序列，目标疑似异常行为序列由各目标疑似异常行为所对应的疑似异常行为的置信度按照预设规则排序得到；图像获取模块4013根据目标疑似异常行为序列的顺序获取目标疑似异常行为所在区域的当前状态图像。
需要说明的是，置信度标定的规则可以是用户根据一定逻辑进行预设的，也可以是基于一些计算模型所规定的置信度规则进行标定。本领域技术人员可以采用可获的置信度计算方法对该疑似异常行为的置信度进行计算。
在一些实施例中，可以通过逻辑回归、决策树等通用的模型算法推测疑似异常行为的置信度。
需要说明的是，预设阈值可以是针对全部场景也即针对全部行为类别均设置统一的预设阈值，也可以是根据不同的行为类别分别设置对应的预设阈值，还可以是用户根据需要随时调整的预设阈值。例如：针对在车站获取监控视频中的行为信息中获取到疑似持有危险物品这一疑似异常行为，其置信度为0.8，按照初始预设阈值为0.85，此时该疑似异常行为将被排除，不会进入下一步骤，对其进行进一步分析。但当前安保等级要求提升，预设阈值设置为0.79，则，该疑似异常行为将被列为目标疑似异常行为。
需要说明的是，在一些实施例中，获取置信度在预设阈值以上的疑似异常行为作为目标疑似异常行为，也即，疑似异常行为的置信度等于预设阈值时，也将其列为目标疑似异常行为。目标疑似异常行为的设定可以根据用户的需要进行更换。
需要说明的是，当异常行为判定方法中存在从疑似异常行为中确定目标疑似异常行为的步骤时，原方法中的获取疑似异常行为所在区域的当前状态图像将随即更改换仅获取目标疑似异常行为所在区域的当前状态图像，对于那些被确定为是疑似异常行为，但不是目标疑似异常行为将不再获取其所在区域的当前状态图像，以降低系统的资源占用，提升处理速度。
需要说明的是，目标疑似异常行为的置信度等于该目标疑似异常行为所对应的疑似异常行为所被标定的置信度。例如疑似异常行为所标定的置信度为0.9，预设阈值为0.8，则该疑似异常行为的置信度高于预设阈值，则区该疑似异常行为作为目标疑似异常行为，该目标疑似异常行为的置信度等于疑似异常行为的置信度等于0.9。
在一些实施例中，根据各目标疑似异常行为所对应的置信度根据预设规则对各目标疑似异常行为进行优先级排序中的预设规则可以是按照置信度的大小从高到低进行排序，也可以是根据置信度的大小针对每个行为类别的目标疑似异常行为进行从高到低排序，再根据行为类别的优先级进行排序，需要说明的是，针对包括多个行为类别的目标疑似异常行为的排序中置信度大小排序与类别优先级排序的顺序可以根据用户需求自行设定，本实施例不做限定。例如：当前有行为类别甲类：目标疑似异常行为A的置信度0.7，目标疑似异常行为B的置信度0.75，目标疑似异常行为C的置信度0.79；行为类别乙类：目标疑似异常行为D的置信度0.7，目标疑似异常行为E的置信度0.75，目标疑似异常行为F的置信度0.79。其中预设规则是行为类别甲类优先级高于乙类，置信度越高优先级越高。则按照此预设规则对上述各目标疑似异常行为进行优先级排序的结果就是：目标疑似异常行为C>目标疑似异常行为B>目标疑似异常行为A>目标疑似异常行为F>目标疑似异常行为E>目标疑似异常行为D。
需要说明的是，在一些实施例中，预设规则也可以是按照置信度由低到高来进行优先级排序，得到目标疑似异常行为序列。
需要说明的是，根据目标疑似异常行为序列可以通过分别控制调整摄像机云台和摄像设备的参数中至少之一来获取目标疑似异常行为所在的坐标信息所对应的区域的当前状态图像。可以理解，在一些实施例中，根据各目标疑似异常行为的目标疑似异常行为序列从优先级高的的目标疑似异常行为所在的坐标信息控制调整摄像机云台和摄像设备参数中至少之一来获取包括相应目标疑似异常行为所在的坐标信息的区域的当前状态图像。例如，目标疑似异常行为A的在目标疑似异常行为序列中的排序高于目标疑似异常行为B的优先级，则先控制调整摄像机云台和摄像设备参数中至少之一来获取包括目标疑似异常行为A所在的坐标信息的区域的当前状态图像，再控制调整摄像机云台和摄像设备参数中至少之一来获取包括目标疑似异常行为B所在的坐标信息的区域的当前状态图像。
在一些实施例中，获取到目标疑似异常行为的位置信息后，将该位置信息于摄像机当前位置进行对比，调整调整云台仰角和水平角度对准异常行为区域中心点。调整焦距让摄像机可以接收以疑似异常行为坐标中心点为图像中心的图像来获取当前状态图像。对于当前存在多个目标疑似异常行为时，可以按照各目标疑似异常行为目标疑似异常行为序列的顺序重复上述步骤，直到获取到所有的目标疑似异常行为的当前状态图像。
在一些实施例中，数据初步处理模块包括第二卷积神经网络，在一些实施例中，第二卷积神经网络包括小型卷积神经网络，可以通过在前端摄像机上布置的小型卷积神经网络，直接在摄像机加载的GPU上进行计算。
需要说明的是，在一些实施例中，通过第二卷积神经网络可以初步确定全部行为类别的疑似异常行为。
在一些实施例中，第二卷积神经网络通过以下方式获取，建立人工标注完成的全类别 的异常行为样本，预先设置的全类别的卷积神经网络对上述样本进行学习，并进行置信度标定训练，学习、训练完成后得到的卷积神经网络作为第二卷积神经网络。该第二卷积神经网络可以对获取到的行为数据进行计算分析，从中筛选出疑似异常行为，并标定该疑似异常行为的置信度，得到该疑似异常行为的位置信息。
需要说明的是，本实施例中的行为数据可以是人的行为数据，也可以是动物的行为数据，还可以是一些机械设备等具有动作的事物。例如，可以获取保护区中目标监控范围中的行为数据，其中包括一只大熊猫在爬树，另一只考拉在树下发呆。此时可以通过小型卷积神经网络确定大熊猫在爬树是疑似异常行为A，考拉在树下发呆为疑似异常行为B。又例如，工厂中某传送带正常工作是以5米每秒的速度在运转，但在监控视频中的行为数据中提取到该传送带的状态时静止不动的，则可以确定该传送带静止为疑似异常行为。
下面结合一个具体的实施例对本发明所提供的异常行为判定装置进行进一步的说明。
参见图7，如图7所示，异常行为判定装置700包括：至少一个视频监控模块701、控制数据传输模块702、阈值判断模块703、排序模块704以及数据处理模块705其中：
视频监控模块701包括数据获取模块7011、数据初步处理模块7012和图像获取模块7013，其中一个数据获取模块7011主要负责其所监控区域的图像获取及行为数据获取，其中，行为数据包括但不限于图像。数据获取模块7011将图像传输给数据初步处理模块7012，数据初步处理模块7012包括摄像机前端的小型卷积神经网络以及GPU，通过小型卷积神经网络及GPU将接收到的图像计算后输出疑似异常行为和人员基本信息，其中，人员基本信息包括但不限于人员姓名、职位、所属类别、性别、年龄、工号等信息。需要说明的是，小型卷积神经网络对于异常的行为检测有很高召回率，对于一份行为数据中的疑似异常行为的差全度较高，例如某行为数据中存在5个疑似异常行为，该小型卷积神经网络可以查找到全部5个疑似异常行为。高召回率主要是由于本发明实施例所使用的小型卷积神经网络是经前期通过大量人工标注完成的异常行为样本的训练学习所保证的。图像获取模块7013可通过完成对摄像机云台角度和摄像机焦距等的调整，主要接收控制模块发送的坐标信息，通过调整摄像机云台角度和焦距等参数来获取针对于坐标信息所在区域清晰的图像信息。需要说明的是，小型卷积神经网络还用于获取疑似异常行为的执行度，通过小型卷积神经网络对疑似异常行为进行置信度标定，并将包括置信度的疑似异常行为传输给阈值判断模块进行进一步处理。
异常行为判定装置700还包括，控制数据传输模块702、阈值判断模块703和排序模块704，其中，阈值判断模块703接收视频监控模块中发送的包括置信度的疑似异常行为，并对此信息通过排序模块对疑似异常行为根据其所对应的置信度完成排序，将比预设阈值大的疑似异常行为发送到图像获取模块7013。这里的阈值为0-1之间的值，可以通过不同场景的测试集来获取一个较优值，一般对于特别敏感的区域，可以选择较低的值来提取更低异常行为置信度的疑似区域，并发送给图像获取模块7013。图像获取模块7013接收阈 值判断模块703发送的疑似异常行为所在的坐标信息，控制摄像机云台调整仰角、水平角度和摄像机焦距，使摄像机对准疑似异常行为的区域来获取针对性的图像数据。控制数据传输模块接收针对于疑似异常行为区域的图像，然后将图像缩小为240*135像素大小，然后编码传输至数据处理模块。需要说明的是，此处还可以将图像统一为PNG等格式。
数据处理模块主要有高性能的GPU集群和针对异常行为的深度卷积神经分类网络。此模块接收控制数据传输模块发送的图像，用深度卷积神经网络对其进一步确认。对于确认异常行为的信息，推算回原始图像位置，最后输出确认异常行为警报和对应的坐标信息。
需要说明的是，其中数据处理模块所输出的对应的坐标信息可以是数据处理模块所计算得到的坐标信息，也可以是数据处理模块将接收到图像所包括的阈值判断模块所发送的疑似异常行为所在的坐标信息直接输出。
需要说明的是，疑似异常行为所对应的坐标信息可以是通过阈值判断模块进行计算得到的，也可以是通过小型卷积神经网络所计算得到的。
需要说明的是，异常行为判定装置，可以包括多个视频监控模块，多个视频监控模块可以对应同一个或者多个数据处理模块。在一些实施例中，多个视频监控模块对应多个阈值判断模块、控制数据传输模块和排序模块，各阈值判断模块、控制数据传输模块和排序模块对应同一个数据处理模块。
需要说明的是，上述各模块仅是做了一种示例性的分区，其中数据处理模块与阈值判断模块、控制数据传输模块和排序模块可以分属于同一模块。
在一些实施例中，数据处理模块位于整个异常行为判定装置的后端，而视频监控模块和控制模块位于异常行为判定装置的前端。
通过本实施例所提供的异常行为判定装置的使用，对于一个监控场景，例如会议室、教室等仅需要布置一个数据获取模块，例如，布置一个摄像头，极大降低了安装的复杂度。极大减轻了数据处理模块处理的负荷，加速了信息处理速度：数据处理模块不需要处理每一帧数据获取模块所获取的数据，仅仅处理疑似异常行为置信度高于预设阈值的部分帧。对于处理的部分帧，也经过了归一化，极大减少了数据量，降低了信息传输和数据处理的负荷，提高了检测判断准确度。另外极大提高了对于异常行为检测的准确性：对于这样的行为数据分为两步判断，包括第一步视频监控模块的通过第二卷积神经网络进行的初次判定后，选取疑似异常行为，再针对疑似异常行为调整云台和摄像头参数获取疑似异常行为区域清晰的当前状态图像，控制数据传输模块归一化后传输给数据处理模块，第二步数据处理模块利用有针对性的第一卷积神经网络例如深度卷积神经分类网络对第一步提取的当前状态图像再次检测判定。第二步获得的当前状态图像更清晰且有针对性。经过第二步深度卷积神经网络的再判断，明显提高了准确度。
本实施例公开了一种异常行为判定装置，包括：至少一个视频监控模块和数据处理模块，其中：视频监控模块包括：数据获取模块，用于获取监控视频中的行为数据；数据初 步处理模块，从行为数据中确定至少一个疑似异常行为；图像获取模块，获取疑似异常行为所在区域的当前状态图像；数据处理模块，用于根据当前状态图像判定疑似异常行为是否是异常行为。通过本实施例的实施不仅可以有效利用计算资源，还可以提升对异常行为判定准确的准确性，降低了资源占用，提升了处理速度。
实施例三：
本实施例还提供了一种异常行为判定终端，参见图8所示，其包括处理器81、存储器83及通信总线82，其中：
通信总线82用于实现处理器81和存储器83之间的连接通信；
处理器81用于执行存储器83中存储的一个或者多个计算机程序，以实现上述各实施例中的异常行为判定方法中的至少一个步骤。
实施例四：
本实施例还提供了一种计算机可读存储介质，该计算机可读存储介质包括在用于存储信息(诸如计算机可读指令、数据结构、计算机程序模块或其他数据)的任何方法或技术中实施的易失性或非易失性、可移除或不可移除的介质。计算机可读存储介质包括但不限于RAM(Random Access Memory，随机存取存储器)，ROM(Read-Only Memory，只读存储器)，EEPROM(Electrically Erasable Programmable read only memory，带电可擦可编程只读存储器)、闪存或其他存储器技术、CD-ROM(Compact Disc Read-Only Memory，光盘只读存储器)，数字多功能盘(DVD)或其他光盘存储、磁盒、磁带、磁盘存储或其他磁存储装置、或者可以用于存储期望的信息并且可以被计算机访问的任何其他的介质。
本实施例中的计算机可读存储介质可用于存储一个或者多个计算机程序，其存储的一个或者多个计算机程序可被处理器执行，以实现上述各实施例中的异常行为判定方法的至少一个步骤。
本实施例还提供了一种计算机程序(或称计算机软件)，该计算机程序可以分布在计算机可读介质上，由可计算装置来执行，以实现上述各实施例中的异常行为判定方法的至少一个步骤；并且在某些情况下，可以采用不同于上述实施例所描述的顺序执行所示出或描述的至少一个步骤。
应当理解的是，在某些情况下，可以采用不同于上述实施例所描述的顺序执行所示出或描述的至少一个步骤。
本实施例还提供了一种计算机程序产品，包括计算机可读装置，该计算机可读装置上存储有如上所示的计算机程序。本实施例中该计算机可读装置可包括如上所示的计算机可读存储介质。
本发明提供的异常行为判定方法、装置、终端及可读存储介质，该异常行为判定方法通过获取监控视频中的行为数据；从行为数据中确定至少一个疑似异常行为；获取疑似异常行为所在区域的当前状态图像；根据当前状态图像判定疑似异常行为是否是异常行为。 由于本发明实施例提供的方法通过对行为数据的预处理来确定疑似异常行为，再有针对性的针对疑似异常行为所在的区域获取当前状态图像，该图像对疑似异常行为的拍摄更加清晰准确，再根据该当前状态图像判定疑似异常行为是否是异常行为。通过本发明的实施，提升对异常行为判定准确的准确性，降低了资源占用，提升了处理速度。
可见，本领域的技术人员应该明白，上文中所公开方法中的全部或某些步骤、系统、装置中的功能模块/单元可以被实施为软件(可以用计算装置可执行的计算机程序代码来实现)、固件、硬件及其适当的组合。在硬件实施方式中，在以上描述中提及的功能模块/单元之间的划分不一定对应于物理组件的划分；例如，一个物理组件可以具有多个功能，或者一个功能或步骤可以由若干物理组件合作执行。某些物理组件或所有物理组件可以被实施为由处理器，如中央处理器、数字信号处理器或微处理器执行的软件，或者被实施为硬件，或者被实施为集成电路，如专用集成电路。
此外，本领域普通技术人员公知的是，通信介质通常包含计算机可读指令、数据结构、计算机程序模块或者诸如载波或其他传输机制之类的调制数据信号中的其他数据，并且可包括任何信息递送介质。所以，本发明不限制于任何特定的硬件和软件结合。

Claims (11)

  1. 一种异常行为判定方法，包括：
    获取监控视频中的行为数据；
    从所述行为数据中确定至少一个疑似异常行为；
    获取所述疑似异常行为所在区域的当前状态图像；
    根据所述当前状态图像判定所述疑似异常行为是否是异常行为。
  2. 如权利要求1所述的异常行为判定方法，其中，所述从所述行为数据中确定至少一个疑似异常行为包括：
    获取所述疑似异常行为的置信度；
    若所述疑似异常行为的置信度大于预设阈值，则将所述疑似异常行为作为目标疑似异常行为。
  3. 如权利要求2所述的异常行为判定方法，其中，所述获取所述疑似异常行为所在区域的当前状态图像包括：
    获取所述目标疑似异常行为所在区域的当前状态图像。
  4. 如权利要求3所述的异常行为判定方法，其中，
    当所述目标疑似异常行为的数量超过1个时，
    所述获取目标疑似异常行为所在区域的当前状态图像包括：
    获取所述目标疑似异常行为序列，所述目标疑似异常行为序列由各所述目标疑似异常行为所对应的疑似异常行为的置信度按照预设规则排序得到；
    根据所述目标疑似异常行为序列的顺序获取所述目标疑似异常行为所在区域的当前状态图像。
  5. 如权利要求1-4任一项所述的异常行为判定方法，其中，所述获取所述疑似异常行为所在区域的当前状态图像之后，根据所述当前状态图像判定所述疑似异常行为是否是异常行为之前还包括：
    对所述当前状态图像进行归一化处理。
  6. 如权利要求1-4任一项所述的异常行为判定方法，其中，所述根据所述当前状态图像判定所述疑似异常行为是否是异常行为还包括：
    若判定所述疑似异常行为是异常行为，输出以下信息中至少之一：
    异常行为警报信息；
    所述异常行为对应的坐标信息。
  7. 如权利要求1-4任一项所述的异常行为判定方法，其中，所述根据所述当前状态图像判定所述疑似异常行为是否是异常行为包括：
    获取所述疑似异常行为所在的行为类别；
    获取与所述疑似异常行为所在的行为类别对应的第一卷积神经网络；
    通过所述第一卷积神经网络根据所述当前状态图像判定所述疑似异常行为是否是异常行为。
  8. 如权利要求1-4任一项所述的异常行为判定方法，其中，所述从所述行为数据中确定至少一个疑似异常行为包括：
    通过第二卷积神经网络从所述行为数据中确定至少一个疑似异常行为。
  9. 一种异常行为判定装置，其中，所述异常行为判定装置包括：至少一个视频监控模块和数据处理模块，其中：
    所述视频监控模块包括：
    数据获取模块，用于获取监控视频中的行为数据；
    数据初步处理模块，用于从所述行为数据中确定至少一个疑似异常行为；
    图像获取模块，用于获取所述疑似异常行为所在区域的当前状态图像；
    所述数据处理模块，用于根据所述当前状态图像判定所述疑似异常行为是否是异常行为。
  10. 一种异常行为判定终端，包括：处理器、存储器及通信总线；其中，
    所述通信总线用于实现处理器和存储器之间的连接通信；
    所述处理器用于执行存储器中存储的一个或者多个计算机程序，以实现如权利要求1至8中任一项所述的异常行为判定方法的步骤。
  11. 一种可读存储介质，存储有一个或者多个计算机程序，其中，所述一个或者多个计算机程序可被一个或者多个处理器执行，以实现如权利要求1至8中任一项所述的异常行为判定方法的步骤。
PCT/CN2020/104520 2019-09-10 2020-07-24 一种异常行为判定方法、装置、终端及可读存储介质 WO2021047306A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910854597.9 2019-09-10
CN201910854597.9A CN112560547A (zh) 2019-09-10 2019-09-10 一种异常行为判定方法、装置、终端及可读存储介质

Publications (1)

Publication Number Publication Date
WO2021047306A1 true WO2021047306A1 (zh) 2021-03-18

Family

ID=74866908

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/104520 WO2021047306A1 (zh) 2019-09-10 2020-07-24 一种异常行为判定方法、装置、终端及可读存储介质

Country Status (2)

Country Link
CN (1) CN112560547A (zh)
WO (1) WO2021047306A1 (zh)

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113301309A (zh) * 2021-05-25 2021-08-24 上海松鼠课堂人工智能科技有限公司 通过视频监控的学生考试作弊行为监测方法与系统
CN113347381A (zh) * 2021-05-24 2021-09-03 随锐科技集团股份有限公司 预测不雅举止轨迹的方法及系统
CN113450001A (zh) * 2021-07-02 2021-09-28 中标慧安信息技术股份有限公司 用于监测熟食制作实施情况的方法和系统
CN113507577A (zh) * 2021-07-07 2021-10-15 杭州海康威视系统技术有限公司 目标对象检测方法、装置、设备及存储介质
CN113635305A (zh) * 2021-08-17 2021-11-12 乐聚(深圳)机器人技术有限公司 机器人运动保护方法、装置、控制器及存储介质
CN115100871A (zh) * 2022-06-20 2022-09-23 钟孟玲 一种行人交通违规识别方法及系统
WO2023185429A1 (zh) * 2022-03-31 2023-10-05 东方电气集团东方电机有限公司 定子铁心压紧状态故障检测方法、装置、发电机组监测系统及计算机可读存储介质
CN117421689A (zh) * 2023-12-18 2024-01-19 杭州湘亭科技有限公司 一种基于管道机器人的铀放射性污染测量传输系统
CN117649642A (zh) * 2024-01-29 2024-03-05 深圳市瀚晖威视科技有限公司 基于多视频摄像头的异常行为分析方法及系统

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113673319A (zh) * 2021-07-12 2021-11-19 浙江大华技术股份有限公司 异常姿态检测方法、装置、电子装置和存储介质
CN114040115A (zh) * 2021-11-29 2022-02-11 海南哦课教育科技有限公司 目标对象异常动作的捕获方法、装置、介质和电子设备

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070086621A1 (en) * 2004-10-13 2007-04-19 Manoj Aggarwal Flexible layer tracking with weak online appearance model
CN103325080A (zh) * 2013-06-21 2013-09-25 电子科技大学 一种基于物联网技术的敬老院智能看护系统及方法
CN109448365A (zh) * 2018-10-16 2019-03-08 北京航空航天大学 一种跨尺度空基平台区域道路交通系统综合监视方法
CN110119712A (zh) * 2019-05-14 2019-08-13 上海阿刻忒科技有限公司 一种基于深度学习的家畜行为智能监控方法及其系统

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070086621A1 (en) * 2004-10-13 2007-04-19 Manoj Aggarwal Flexible layer tracking with weak online appearance model
CN103325080A (zh) * 2013-06-21 2013-09-25 电子科技大学 一种基于物联网技术的敬老院智能看护系统及方法
CN109448365A (zh) * 2018-10-16 2019-03-08 北京航空航天大学 一种跨尺度空基平台区域道路交通系统综合监视方法
CN110119712A (zh) * 2019-05-14 2019-08-13 上海阿刻忒科技有限公司 一种基于深度学习的家畜行为智能监控方法及其系统

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113347381A (zh) * 2021-05-24 2021-09-03 随锐科技集团股份有限公司 预测不雅举止轨迹的方法及系统
CN113301309A (zh) * 2021-05-25 2021-08-24 上海松鼠课堂人工智能科技有限公司 通过视频监控的学生考试作弊行为监测方法与系统
CN113450001A (zh) * 2021-07-02 2021-09-28 中标慧安信息技术股份有限公司 用于监测熟食制作实施情况的方法和系统
CN113507577A (zh) * 2021-07-07 2021-10-15 杭州海康威视系统技术有限公司 目标对象检测方法、装置、设备及存储介质
CN113635305A (zh) * 2021-08-17 2021-11-12 乐聚(深圳)机器人技术有限公司 机器人运动保护方法、装置、控制器及存储介质
CN113635305B (zh) * 2021-08-17 2023-06-23 乐聚(深圳)机器人技术有限公司 机器人运动保护方法、装置、控制器及存储介质
WO2023185429A1 (zh) * 2022-03-31 2023-10-05 东方电气集团东方电机有限公司 定子铁心压紧状态故障检测方法、装置、发电机组监测系统及计算机可读存储介质
CN115100871A (zh) * 2022-06-20 2022-09-23 钟孟玲 一种行人交通违规识别方法及系统
CN117421689A (zh) * 2023-12-18 2024-01-19 杭州湘亭科技有限公司 一种基于管道机器人的铀放射性污染测量传输系统
CN117421689B (zh) * 2023-12-18 2024-03-12 杭州湘亭科技有限公司 一种基于管道机器人的铀放射性污染测量传输系统
CN117649642A (zh) * 2024-01-29 2024-03-05 深圳市瀚晖威视科技有限公司 基于多视频摄像头的异常行为分析方法及系统
CN117649642B (zh) * 2024-01-29 2024-04-05 深圳市瀚晖威视科技有限公司 基于多视频摄像头的异常行为分析方法及系统

Also Published As

Publication number Publication date
CN112560547A (zh) 2021-03-26

Similar Documents

Publication Publication Date Title
WO2021047306A1 (zh) 一种异常行为判定方法、装置、终端及可读存储介质
US10372970B2 (en) Automatic scene calibration method for video analytics
US10366509B2 (en) Setting different background model sensitivities by user defined regions and background filters
WO2020078229A1 (zh) 目标对象的识别方法和装置、存储介质、电子装置
US11615620B2 (en) Systems and methods of enforcing distancing rules
CN104813339A (zh) 用于检测视频中的对象的方法、设备和系统
WO2020000912A1 (zh) 一种行为检测方法、装置、电子设备和存储介质
CN108389359B (zh) 一种基于深度学习的城市火灾报警方法
KR102195706B1 (ko) 침입 탐지방법 및 그 장치
CN109815787B (zh) 目标识别方法、装置、存储介质及电子设备
KR20170074076A (ko) 능동형 교통 신호 제어 방법 및 그 시스템
CN112733690A (zh) 一种高空抛物检测方法、装置及电子设备
US10909388B2 (en) Population density determination from multi-camera sourced imagery
US20220414997A1 (en) Methods and systems for providing a tutorial for graphic manipulation of objects including real-time scanning in an augmented reality
WO2022205632A1 (zh) 目标检测方法、装置、设备及存储介质
CN114724330A (zh) 一种自适应模式切换的多路视频火灾实时报警系统的实现方法
CN108319892A (zh) 一种基于遗传算法的车辆安全预警方法及系统
US11887374B2 (en) Systems and methods for 2D detections and tracking
KR102511287B1 (ko) 영상 기반 자세 예측 및 행동 검출 방법 및 장치
CN107578595A (zh) 流动体分析装置
US20210357654A1 (en) Systems and methods of identifying persons-of-interest
CN114040094A (zh) 一种基于云台相机的预置位调节方法和设备
CN110796580B (zh) 智能交通系统管理方法及相关产品
CN110445966B (zh) 一种全景相机视频拍摄方法、装置、电子设备及存储介质
CN109460077B (zh) 一种自动跟踪方法、自动跟踪设备及自动跟踪系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20863492

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20863492

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 25.08.2022)

122 Ep: pct application non-entry in european phase

Ref document number: 20863492

Country of ref document: EP

Kind code of ref document: A1