CN113111808B - Abnormal behavior detection method and system based on machine vision - Google Patents

Abnormal behavior detection method and system based on machine vision Download PDF

Info

Publication number
CN113111808B
CN113111808B CN202110423830.5A CN202110423830A CN113111808B CN 113111808 B CN113111808 B CN 113111808B CN 202110423830 A CN202110423830 A CN 202110423830A CN 113111808 B CN113111808 B CN 113111808B
Authority
CN
China
Prior art keywords
limb
key points
boundary
frame
abnormal behavior
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110423830.5A
Other languages
Chinese (zh)
Other versions
CN113111808A (en
Inventor
王德强
焦广超
郑来波
王鸣天
李晓
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shandong University
Original Assignee
Shandong University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shandong University filed Critical Shandong University
Priority to CN202110423830.5A priority Critical patent/CN113111808B/en
Publication of CN113111808A publication Critical patent/CN113111808A/en
Application granted granted Critical
Publication of CN113111808B publication Critical patent/CN113111808B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/103—Static body considered as a whole, e.g. static pedestrian or occupant recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/243—Classification techniques relating to the number of classes
    • G06F18/2433—Single-class perspective, e.g. one-against-all classification; Novelty detection; Outlier detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20048—Transform domain processing
    • G06T2207/20056—Discrete and fast Fourier transform, [DFT, FFT]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30232—Surveillance

Abstract

The invention discloses an abnormal behavior detection method and system based on machine vision, which comprises the following steps: extracting limb key points in each frame of video image; obtaining the movement speed of the key point of the limb according to the movement distance of the key point of the same limb in the two adjacent images and the time difference between the two images; obtaining the motion frequency between two limb key points through fast Fourier transform according to the distance sequence of the two limb key points in each frame of image in unit time; judging whether the boundary is out of range or not according to the limb key point coordinates and the demarcated boundary of the movable area; and judging whether abnormal behaviors exist or not according to the movement speed and the movement frequency of the key points of the limbs and the fact whether the key points cross the boundary or not. Abnormal behaviors are detected in real time, early warning is carried out, and the intelligent level is improved.

Description

Abnormal behavior detection method and system based on machine vision
Technical Field
The invention relates to the technical field of intelligent monitoring, in particular to an abnormal behavior detection method and system based on machine vision.
Background
The statements in this section merely provide background information related to the present disclosure and may not necessarily constitute prior art.
With the progress of science and technology, a remote telephone or remote video mode appears in a traditional meeting mode, and in the video process, when video personnel have situations such as mood fluctuation, aggressive behavior, sudden diseases and the like, if the video personnel are not checked or concerned in time, various risks and hidden dangers easily exist; although the monitoring video plays a role in assisting management and control to a certain extent, evidence recording and backtracking evidence obtaining efficiency is low when risk hazards occur.
Disclosure of Invention
In order to solve the problems, the invention provides an abnormal behavior detection method and system based on machine vision, which comprises the steps of extracting coordinate information of key points of limbs in a video image by using a deep neural network model, calculating the motion speed of the key points according to the coordinates of the key points, calculating the motion frequency of the key points by using fast Fourier transform, judging whether a specified boundary is crossed or not according to the coordinates of the key points, obtaining an abnormal behavior detection result according to the motion speed and the motion frequency of the key points and whether the specified boundary is crossed or not, and early warning according to the detection result.
In order to achieve the purpose, the invention adopts the following technical scheme:
in a first aspect, the present invention provides a method for detecting abnormal behavior based on machine vision, including:
extracting limb key points in each frame of video image;
obtaining the movement speed of the key point of the limb according to the movement distance of the key point of the same limb in the two adjacent images and the time difference between the two images;
obtaining the motion frequency between two limb key points through fast Fourier transform according to the distance sequence of the two limb key points in each frame of image in unit time;
judging whether the boundary is out of range or not according to the limb key point coordinates and the demarcated boundary of the movable area;
and judging whether abnormal behaviors exist or not according to the movement speed and the movement frequency of the key points of the limbs and the fact whether the key points cross the boundary or not.
As an alternative embodiment, the process of calculating the movement speed of the limb key point includes: respectively calculating the movement speeds of the left wrist and the right wrist by using the movement distance of the left wrist and the right wrist and the time difference between two adjacent frames of images, wherein the movement speeds of the left wrist and the right wrist are calculated by adopting the following formulas:
Figure RE-GDA0003080067290000021
wherein, Vi,nRepresents the motion speed of the left wrist (or right wrist) of the ith frame, (x)i,n，yi,n) Coordinate values (x) representing the left wrist (or right wrist) of the ith framei-1,n，yi-1,n) Coordinate values (x) representing the left wrist (or right wrist) of the i-1 th framei,6，yi,6) Coordinate value (x) indicating the left shoulder of the ith framei,7，yi,7) Coordinate value, t, representing the right shoulder of frame iTRepresenting the time difference between the previous and the next two frames of images.
As an alternative embodiment, the distance between two limb key points in each frame of image is calculated by the following formula:
Figure RE-GDA0003080067290000022
wherein L isi,p,qRepresenting Euclidean distance of two limb key points of p and q of the ith frame, (x)i,p,yi,p) Coordinate value (x) representing the p-th limb key pointi,q,yi,q) Coordinate values representing the qth limb key point;
as an alternative embodiment, the calculation process of the motion frequency between two limb key points comprises: sequentially dividing the distance L between two key points p and q in all the N image frames in unit timei,p,qForming a time series Q of length Np,q＝(Li,p,q,Li+1,p,q,...,Li+N-1,p,q) For time series Qp,qAnd performing N-point fast Fourier transform, removing direct current components in the transform result, and taking the frequency corresponding to the component with the maximum amplitude in the residual components as the motion frequency.
As an alternative embodiment, the process of determining whether there is an abnormal behavior includes: presetting a movement speed threshold and a limb overspeed frequency threshold, comparing the movement speeds of the left wrist and the right wrist with the movement speed threshold, recording that the limb is overspeed once if the movement speed exceeds the movement speed threshold, and judging that abnormal behaviors exist if the limb overspeed frequency exceeds the threshold in unit time.
As an alternative embodiment, the process of determining whether there is an abnormal behavior includes: presetting a motion frequency threshold, comparing the motion frequency among the key points of the limbs in unit time with the motion frequency threshold, and if the motion frequency among at least one group of the key points of the limbs exceeds the motion frequency threshold, judging that abnormal behaviors exist.
As an alternative embodiment, the process of determining whether there is an abnormal behavior includes: presetting a boundary of a movable area, comparing the coordinate value of the nose with the boundary of the movable area, and judging that abnormal behaviors exist if the coordinate value of the nose crosses the boundary of the movable area.
In a second aspect, the present invention provides a system for detecting abnormal behavior based on machine vision, including:
the key point extraction module is configured to extract limb key points in each frame of video image;
the motion speed calculation module is configured to obtain the motion speed of the key point of the limb according to the moving distance of the key point of the same limb in the two adjacent frames of images and the time difference between the two frames of images;
the motion frequency calculation module is configured to obtain the motion frequency between two limb key points through fast Fourier transform according to the distance sequence of the two limb key points in each frame of image in unit time;
the boundary crossing judging module is configured to judge whether boundary crossing exists according to the limb key point coordinates and the boundary of the demarcated movable area;
and the abnormal behavior judging module is configured to judge whether abnormal behaviors exist according to the movement speed and the movement frequency of the key points of the limb and whether the key points cross the boundary.
In a third aspect, the present invention provides an electronic device comprising a memory and a processor, and computer instructions stored on the memory and executed on the processor, wherein when the computer instructions are executed by the processor, the method of the first aspect is performed.
In a fourth aspect, the present invention provides a computer readable storage medium for storing computer instructions which, when executed by a processor, perform the method of the first aspect.
Compared with the prior art, the invention has the beneficial effects that:
the invention provides the abnormal behavior detection method and system based on the machine vision by combining the meeting video, detects the abnormal behavior of people in real time and carries out early warning, thereby being beneficial to reducing the input and working intensity of manpower and improving the intelligent level.
According to the abnormal behavior detection method based on the video, coordinate information of human body key points in a video image is extracted by using a deep neural network model, the movement speed of the key points is calculated according to the key point coordinates, the movement frequency of the key points is calculated by using fast Fourier transform, whether the key point coordinate points cross the boundary of a defined movable area or not is judged, and whether abnormal behavior occurs or not and potential risk early warning is carried out according to the key point movement speed, the movement frequency and the comprehensive judgment of whether the boundary is out of range or not.
The method utilizes the combination of deep learning and signal processing technology to detect the abnormal behavior, and has high accuracy and high operation speed.
The invention utilizes the neural network model to obtain the coordinates of the key points of the human body, and the model can be replaced with a model with higher operation speed and higher accuracy at any time according to the latest research progress, has strong flexibility and is beneficial to later-stage upgrading.
The method fully utilizes the coordinate information of the key points to judge the speed state of the key points, and is an important characteristic for detecting abnormal behaviors.
The method provided by the invention utilizes fast Fourier transform in a signal processing technology to analyze and obtain the operating frequency of the key point, and combines deep learning and signal processing to provide a new method for detecting abnormal behaviors.
The invention comprehensively judges whether the movement speed of the key point, the movement frequency of the key point and the coordinates of the key point cross the boundary of the defined area, and the system gives an early warning when abnormal behaviors such as repeated table shooting, head beating by hands, chest beating by hands and the like occur.
Compared with a method for detecting abnormal behaviors by using single behavior characteristics, the method provided by the invention has the advantages that the correctness is improved, and the application value is good.
Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, are included to provide a further understanding of the invention, and are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the invention and together with the description serve to explain the invention and not to limit the invention.
Fig. 1 is a flowchart of a method for detecting abnormal behavior based on machine vision according to embodiment 1 of the present invention.
The specific implementation mode is as follows:
the invention is further described with reference to the following figures and examples.
It is to be understood that the following detailed description is exemplary and is intended to provide further explanation of the invention as claimed. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of exemplary embodiments according to the invention. As used herein, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise, and it should be understood that the terms "comprises" and "comprising", and any variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
The embodiments and features of the embodiments of the present invention may be combined with each other without conflict.
Example 1
As shown in fig. 1, the present embodiment provides a method for detecting abnormal behavior based on machine vision, including:
s1: extracting the coordinates of the limb key points in each frame of video image;
s2: obtaining the movement speed of the key point of the limb according to the movement distance of the key point of the same limb in the two adjacent images and the time difference between the two images;
s3: obtaining the motion frequency between two limb key points through fast Fourier transform according to the distance sequence of the two limb key points in each frame of image in unit time;
s4: judging whether the boundary is out of range or not according to the limb key point coordinates and the demarcated boundary of the movable area;
s5: and judging whether abnormal behaviors exist or not according to the movement speed and the movement frequency of the key points of the limbs and the fact whether the key points cross the boundary or not.
In step S1, the video image is obtained by: and video images are acquired in real time through the camera.
In step S1, the method further includes performing a normalization preprocessing on each frame of the acquired video image, and cutting the size of the video image into an input size required by the neural network;
preferably, the normalization preprocessing includes subtracting the mean values from the three RGB channels of the video image, and then dividing the mean values by the variance to complete the normalization, so as to accelerate the convergence speed of the model;
preferably, the pixel values of the video image after normalization are adjusted to a prescribed size; in this embodiment, the Simple Baselines model is used to detect the key points of the limbs, so the pixel value of the video image needs to be adjusted to 192 × 256.
In step S1, inputting the processed video image into a limb key point neural network model to obtain coordinate values of limb key points; in the embodiment, a Simple Baselines model is adopted to obtain coordinate values of 11 limb key points of the upper half of the body, including a nose, two eyes, two ears, two shoulders, two elbows and two wrists;
in step S2, according to the coordinates of the key points of the limbs in the two adjacent frames of images, specifically, by using the moving distance of the left wrist and the right wrist and the time difference between the two frames of images, the movement speeds of the left wrist and the right wrist are respectively calculated;
specifically, the euclidean distance between shoulders is used as a scale for measuring the moving distance, and the motion speed calculation formula is as follows:
Figure RE-GDA0003080067290000071
wherein, Vi,nRepresents the motion speed of the left wrist (or right wrist) of the ith frame, (x)i,n，yi,n) Coordinate values (x) representing the left wrist (or right wrist) of the ith framei-1,n，yi-1,n) Coordinate values (x) representing the left wrist (or right wrist) of the i-1 th framei,6，yi,6) Coordinate value (x) indicating the left shoulder of the ith framei,7，yi,7) Coordinate value, t, representing the right shoulder of frame iTRepresenting the time difference between the previous and the next two frames of images.
In step S3, the distances between two limb key points in each frame of image, specifically, the distances between four sets of limb key points, namely, the left wrist and the left ear, the left wrist and the right shoulder, the right wrist and the right ear, and the right wrist and the left shoulder, are respectively combined into a time sequence in unit time according to the image frame sequence, and the motion frequencies between the four sets of limb key points are respectively calculated through fast fourier transform;
in step S3, the distance calculation formula between the four sets of limb key points is:
Figure RE-GDA0003080067290000081
wherein L isi,p,qRepresenting Euclidean distance of two limb key points of p and q of the ith frame, (x)i,p,yi,p) Coordinate value (x) representing the p-th limb key pointi,q,yi,q) Representing the sit of the qth limb keypointMarking a value;
in step S3, the unit time refers to a time slice of an integer number of seconds;
in step S3, the process of forming time series by the distances between the four calculated limb key points according to the image frame sequence includes the specific steps of forming the distances L between the key points p and q in unit timei,p,qThe time series of compositions may be represented as Qp,q＝(Li,p,q,Li+1,p,q,...,Li+N-1,p,q) Wherein N represents the number of image frames per unit time;
in step S3, the process of calculating the motion frequency between the four sets of key points of the limb by fast fourier transform includes, specifically, the distance Q between the key points p and Qp,qAnd performing N-point fast Fourier transform, removing direct current components in the transform result, and taking the frequency corresponding to the component with the maximum amplitude (absolute value) in the residual components as the motion frequency.
In step S4, it is determined whether the boundary is out of bounds according to the coordinates of the nose in each frame of image and the boundary of the defined movable region;
the movable region is a preset rectangle in a video image, and four vertexes have definite coordinate values;
the out-of-bounds is that the coordinates of the nose are not within the rectangle corresponding to the moveable region.
In step S5, determining whether there is an abnormal behavior and giving an alarm according to the movement speed of the left and right wrists, the movement frequency between the four groups of key points, and whether the boundary is crossed; specifically, the method comprises the following steps: setting a movement speed threshold and a limb overspeed frequency threshold, comparing the movement speeds of the left wrist and the right wrist with the movement speed threshold, if the movement speeds of the left wrist and the right wrist exceed the movement speed threshold, recording that the limb is overspeed once, and if the limb overspeed frequency exceeds the threshold in unit time, judging that the behavior is abnormal; setting a motion frequency threshold, comparing the motion frequency among the four groups of key points in unit time with the motion frequency threshold, and if the motion frequency threshold is exceeded, judging that the behavior is abnormal; comparing the nose coordinate with the boundary of the movable area, and judging the abnormal behavior if the nose coordinate exceeds the boundary of the movable area once; and sending alarm information if abnormal behaviors occur according to the judgment result.
The abnormal behavior detection method based on the video is applied to the management of the personnel behaviors, and more accurate personnel behavior information can be obtained compared with the traditional manual method.
Example 2
The embodiment provides an abnormal behavior detection system based on machine vision, which comprises:
the key point extraction module is configured to extract limb key points in each frame of video image;
the motion speed calculation module is configured to obtain the motion speed of the key point of the limb according to the moving distance of the key point of the same limb in the two adjacent frames of images and the time difference between the two frames of images;
the motion frequency calculation module is configured to obtain the motion frequency between two limb key points through fast Fourier transform according to the distance sequence of the two limb key points in each frame of image in unit time;
the boundary crossing judging module is configured to judge whether boundary crossing exists according to the limb key point coordinates and the boundary of the demarcated movable area;
and the abnormal behavior judging module is configured to judge whether abnormal behaviors exist according to the movement speed and the movement frequency of the key points of the limb and whether the key points cross the boundary.
It should be noted that the modules correspond to the steps described in embodiment 1, and the modules are the same as the corresponding steps in the implementation examples and application scenarios, but are not limited to the disclosure in embodiment 1. It should be noted that the modules described above as part of a system may be implemented in a computer system such as a set of computer-executable instructions.
In further embodiments, there is also provided:
an electronic device comprising a memory and a processor and computer instructions stored on the memory and executed on the processor, the computer instructions when executed by the processor performing the method of embodiment 1. For brevity, no further description is provided herein.
It should be understood that in this embodiment, the processor may be a central processing unit CPU, and the processor may also be other general purpose processors, digital signal processors DSP, application specific integrated circuits ASIC, off-the-shelf programmable gate arrays FPGA or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and so on. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory may include both read-only memory and random access memory, and may provide instructions and data to the processor, and a portion of the memory may also include non-volatile random access memory. For example, the memory may also store device type information.
A computer readable storage medium storing computer instructions which, when executed by a processor, perform the method described in embodiment 1.
The method in embodiment 1 may be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules in the processor. The software modules may be located in ram, flash, rom, prom, or eprom, registers, among other storage media as is well known in the art. The storage medium is located in a memory, and a processor reads information in the memory and completes the steps of the method in combination with hardware of the processor. To avoid repetition, it is not described in detail here.
Those of ordinary skill in the art will appreciate that the various illustrative elements, i.e., algorithm steps, described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
The above is only a preferred embodiment of the present invention, and is not intended to limit the present invention, and various modifications and changes will occur to those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Although the embodiments of the present invention have been described with reference to the accompanying drawings, it is not intended to limit the scope of the present invention, and it should be understood by those skilled in the art that various modifications and variations can be made without inventive efforts by those skilled in the art based on the technical solution of the present invention.

Claims (10)

1. A method for detecting abnormal behaviors based on machine vision is characterized by comprising the following steps:
extracting limb key points in each frame of video image;
obtaining the movement speed of the key point of the limb according to the movement distance of the key point of the same limb in the two adjacent images and the time difference between the two images;
obtaining the motion frequency between two limb key points through fast Fourier transform according to the distance sequence of the two limb key points in each frame of image in unit time;
judging whether the boundary is out of range or not according to the limb key point coordinates and the demarcated boundary of the movable area;
and judging whether abnormal behaviors exist or not according to the movement speed and the movement frequency of the key points of the limbs and the fact whether the key points cross the boundary or not.
2. The abnormal behavior detection method based on machine vision as claimed in claim 1, wherein the calculation process of the movement speed of the limb key point comprises: respectively calculating the movement speeds of the left wrist and the right wrist by using the movement distance of the left wrist and the right wrist and the time difference between two adjacent frames of images, wherein the movement speeds of the left wrist and the right wrist are calculated by adopting the following formulas:
Figure FDA0003496600020000011
wherein, Vi,nRepresents the motion speed of the left or right wrist of the ith frame, (x)i,n，yi,n) Coordinate values representing the left or right wrist of the ith frame, (x)i-1,n，yi-1,n) Coordinate values representing the left or right wrist of the i-1 th frame, (x)i,6，yi,6) Coordinate value (x) indicating the left shoulder of the ith framei,7，yi,7) Coordinate value, t, representing the right shoulder of frame iTRepresenting the time difference between the previous and the next two frames of images.
3. The abnormal behavior detection method based on machine vision as claimed in claim 1, characterized in that the distance between two key points of the limbs in each frame of image is calculated by the following formula:
Figure FDA0003496600020000012
wherein L isi,p,qRepresenting Euclidean distance of two limb key points of p and q of the ith frame, (x)i,p,yi,p) Coordinate value (x) representing the p-th limb key pointi,q,yi,q) Coordinate values representing the qth limb keypoint.
4. The abnormal behavior detection method based on machine vision as claimed in claim 1, wherein the calculation process of the motion frequency between two limb key points comprises: sequentially dividing the distance L between two key points p and q in all the N image frames in unit timei,p,qForming a time series Q of length Np,q＝(Li,p,q,Li+1,p,q,...,Li+N-1,p,q) Wherein L isi,p,qRepresenting Euclidean distance of two critical points of the p th and Q th limbs of the ith frame to a time sequence Qp,qAnd performing N-point fast Fourier transform, removing direct current components in the transform result, and taking the frequency corresponding to the component with the maximum amplitude in the residual components as the motion frequency.
5. The abnormal behavior detection method based on machine vision as claimed in claim 1, wherein the process of determining whether there is abnormal behavior comprises: presetting a movement speed threshold and a limb overspeed frequency threshold, comparing the movement speeds of the left wrist and the right wrist with the movement speed threshold, recording that the limb is overspeed once if the movement speed exceeds the movement speed threshold, and judging that abnormal behaviors exist if the limb overspeed frequency exceeds the threshold in unit time.
6. The abnormal behavior detection method based on machine vision as claimed in claim 1, wherein the process of determining whether there is abnormal behavior comprises: presetting a motion frequency threshold, comparing the motion frequency among the key points of the limbs in unit time with the motion frequency threshold, and if the motion frequency among at least one group of the key points of the limbs exceeds the motion frequency threshold, judging that abnormal behaviors exist.
7. The abnormal behavior detection method based on machine vision as claimed in claim 1, wherein the process of determining whether there is abnormal behavior comprises: presetting a boundary of a movable area, comparing the coordinate value of the nose with the boundary of the movable area, and judging that abnormal behaviors exist if the coordinate value of the nose crosses the boundary of the movable area.
8. A machine vision based abnormal behavior detection system, comprising:
the key point extraction module is configured to extract limb key points in each frame of video image;
the motion speed calculation module is configured to obtain the motion speed of the key point of the limb according to the moving distance of the key point of the same limb in the two adjacent frames of images and the time difference between the two frames of images;
the motion frequency calculation module is configured to obtain the motion frequency between two limb key points through fast Fourier transform according to the distance sequence of the two limb key points in each frame of image in unit time;
the boundary crossing judging module is configured to judge whether boundary crossing exists according to the limb key point coordinates and the boundary of the demarcated movable area;
and the abnormal behavior judging module is configured to judge whether abnormal behaviors exist according to the movement speed and the movement frequency of the key points of the limb and whether the key points cross the boundary.
9. An electronic device comprising a memory and a processor and computer instructions stored on the memory and executed on the processor, the computer instructions when executed by the processor performing the method of any of claims 1-7.
10. A computer-readable storage medium storing computer instructions which, when executed by a processor, perform the method of any one of claims 1 to 7.
CN202110423830.5A 2021-04-20 2021-04-20 Abnormal behavior detection method and system based on machine vision Active CN113111808B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110423830.5A CN113111808B (en) 2021-04-20 2021-04-20 Abnormal behavior detection method and system based on machine vision

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110423830.5A CN113111808B (en) 2021-04-20 2021-04-20 Abnormal behavior detection method and system based on machine vision

Publications (2)

Publication Number Publication Date
CN113111808A CN113111808A (en) 2021-07-13
CN113111808B true CN113111808B (en) 2022-03-29

Family

ID=76718857

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110423830.5A Active CN113111808B (en) 2021-04-20 2021-04-20 Abnormal behavior detection method and system based on machine vision

Country Status (1)

Country Link
CN (1) CN113111808B (en)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105518710A (en) * 2015-04-30 2016-04-20 北京旷视科技有限公司 Video detecting method, video detecting system and computer program product
CN108564599A (en) * 2018-04-08 2018-09-21 广东省智能制造研究所 A kind of human motion speed estimation method
CN109325476A (en) * 2018-11-20 2019-02-12 齐鲁工业大学 A kind of human body exception attitude detection system and method based on 3D vision
CN109508736A (en) * 2018-10-30 2019-03-22 航天信息股份有限公司 A kind of prison abnormal conditions monitoring method and monitoring system based on deep learning
CN110147736A (en) * 2019-04-25 2019-08-20 沈阳航空航天大学 A kind of repetition anomaly detection method based on posture
CN110738095A (en) * 2019-08-23 2020-01-31 珠海格力电器股份有限公司 behavior analysis method and device
CN112528763A (en) * 2020-11-24 2021-03-19 浙江大华汽车技术有限公司 Target detection method, electronic device and computer storage medium

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
AU2005278003B8 (en) * 2004-06-21 2011-06-23 Google Llc Single image based multi-biometric system and method
CN105912986B (en) * 2016-04-01 2019-06-07 北京旷视科技有限公司 A kind of biopsy method and system

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105518710A (en) * 2015-04-30 2016-04-20 北京旷视科技有限公司 Video detecting method, video detecting system and computer program product
CN108564599A (en) * 2018-04-08 2018-09-21 广东省智能制造研究所 A kind of human motion speed estimation method
CN109508736A (en) * 2018-10-30 2019-03-22 航天信息股份有限公司 A kind of prison abnormal conditions monitoring method and monitoring system based on deep learning
CN109325476A (en) * 2018-11-20 2019-02-12 齐鲁工业大学 A kind of human body exception attitude detection system and method based on 3D vision
CN110147736A (en) * 2019-04-25 2019-08-20 沈阳航空航天大学 A kind of repetition anomaly detection method based on posture
CN110738095A (en) * 2019-08-23 2020-01-31 珠海格力电器股份有限公司 behavior analysis method and device
CN112528763A (en) * 2020-11-24 2021-03-19 浙江大华汽车技术有限公司 Target detection method, electronic device and computer storage medium

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
"Smart Monitoring Cameras Driven Intelligent Processing to Big Surveillance Video Data";Zhenfeng Shao 等;《 IEEE Transactions on Big Data》;20180501;第4卷(第1期);第105-116页 *
"基于三维骨架序列的人体行为识别研究";姜新波;《中国优秀博硕士学位论文全文数据库(博士)信息科技辑》;20160415(第4期);全文 *

Also Published As

Publication number Publication date
CN113111808A (en) 2021-07-13

Similar Documents

Publication Publication Date Title
WO2020199480A1 (en) Body movement recognition method and device
EP3909504A2 (en) Monitoring the performance of physical exercises
Sawant et al. Real time sign language recognition using pca
WO2020215552A1 (en) Multi-target tracking method, apparatus, computer device, and storage medium
Mozaffari et al. Gender classification using single frontal image per person: combination of appearance and geometric based features
CN110263603B (en) Face recognition method and device based on central loss and residual error visual simulation network
CN111753747B (en) Violent motion detection method based on monocular camera and three-dimensional attitude estimation
CN107194361A (en) Two-dimentional pose detection method and device
CN114937232B (en) Wearing detection method, system and equipment for medical waste treatment personnel protective appliance
WO2022228252A1 (en) Human behavior detection method and apparatus, electronic device and storage medium
CN110414400A (en) A kind of construction site safety cap wearing automatic testing method and system
CN111709296A (en) Scene identification method and device, electronic equipment and readable storage medium
CN114998830A (en) Wearing detection method and system for safety helmet of transformer substation personnel
CN112633221A (en) Face direction detection method and related device
CN114550027A (en) Vision-based motion video fine analysis method and device
CN110969110A (en) Face tracking method and system based on deep learning
CN108921010B (en) Pupil detection method and detection device
CN113111808B (en) Abnormal behavior detection method and system based on machine vision
CN113706837B (en) Engine abnormal state detection method and device
CN112699950B (en) Medical image classification method, image classification network processing method, device and equipment
CN112733677B (en) People flow rate statistical system and method
CN111444555B (en) Temperature measurement information display method and device and terminal equipment
Suzuki et al. Automatic fault detection in race walking from a smartphone camera via fine-tuning pose estimation
US11527090B2 (en) Information processing apparatus, control method, and non-transitory storage medium
CN113536842A (en) Electric power operator safety dressing identification method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant