CN111241874A - Behavior monitoring method and device and computer readable storage medium - Google Patents

Abstract

The embodiment of the present application discloses a behavior monitoring method. The behavior monitoring method includes: determining a target key point of a target object in a real-time video stream collected; determining the behavior information of the target object according to the target key point; The behavior information determines the behavior state of the target object; the embodiment of the present application further discloses a behavior monitoring device and a computer-readable storage medium.

Description

Behavior monitoring method, device and computer-readable storage medium

technical field

The embodiments of the present application relate to the technical field of computer vision, and relate to, but are not limited to, a behavior monitoring method, an apparatus, and a computer-readable storage medium.

Background technique

The technologies for monitoring user behavior in the related art include: vehicle monitoring, contour detection, Gaussian background modeling, and somatosensory Kinect sensor technology.

When the vehicle monitoring technology monitors the user's behavior, the user needs to wear the vehicle, which is inconvenient to use and may bring danger to special groups; when the contour detection technology monitors the user's behavior, all the fitting coordinates are missed, and the tracking Lost situation; when Gaussian background modeling technology monitors user behavior, it needs to collect user behavior images in advance; when Kinect sensor technology monitors user behavior, on the one hand, it needs to record behavior videos in advance, on the other hand, it needs to purchase Kinect sensor.

SUMMARY OF THE INVENTION

Embodiments of the present application provide a behavior monitoring method, device, and computer-readable storage medium.

The technical solutions of the embodiments of the present application are implemented as follows:

The present application provides a method for behavior monitoring, the method includes: determining a target key point of a target object in a real-time video stream collected; determining behavior information of the target object according to the target key point; according to the behavior The information determines the behavioral state of the target object.

In the above solution, determining the target key points of the target object in the collected real-time video stream includes: identifying the target object in the video stream and the key points of the target object; The target key point is determined from the key points.

In the above solution, the determining the target key points of the target object in the collected real-time video stream includes: identifying the target object in the video stream and the key points of the target object; partition to obtain at least one partition, and the partition contains at least one key point; and a key point of one partition in the partition is selected as the target key point.

In the above solution, the determining the behavior information of the target object according to the target key points includes: determining the coordinate information of the target key points; behavioral information.

In the above solution, the method further includes: acquiring sample data of different behavior states of the object in different application scenarios; determining behavior information of the object according to the behavior states of the sample data; determining the behavior information according to the behavior information. The target key point of the object is described; the corresponding relationship between the application scene, behavior state, behavior information and the target key point is determined; the deep network model is established according to the corresponding relationship.

In the above solution, the method further includes: setting an application scene; correspondingly, the determining the behavior state of the target object according to the behavior information includes: determining the target object according to the behavior information and the application scene behavioral state.

In the above solution, the method further includes: acquiring a user identifier corresponding to the behavior state; and using the user identifier as a target address, outputting the behavior state of the target object.

The present application also provides a behavior monitoring device, the device includes: a first determination module, a second determination module, and a third determination module; wherein the first determination module is used to determine the real-time video stream collected. The target key point of the target object; the second determination module is used to determine the behavior information of the target object according to the target key point; the third determination module is used to determine the target according to the behavior information. The behavioral state of the object.

The present application also provides an apparatus for behavior monitoring, including a processor and a memory for storing a computer program that can be run on the processor; wherein, when the processor is used to run the computer program, the execution is applied to a terminal device. The steps of the method of behavioral monitoring described in the above scheme.

The present application also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, implements the steps of the method for behavior monitoring described in the above solution applied to a terminal device.

The behavior monitoring method, device, and computer-readable storage medium provided by the embodiments of the present application determine the target key points of the target object in the collected real-time video stream; determine the behavior information of the target object according to the target key points; The behavior information determines the behavior state of the target object; in this way, the user's behavior can be recognized by determining and monitoring the key points of the human body by using the collected real-time video stream, thereby improving the user experience.

Description of drawings

FIG. 1 is a schematic diagram 1 of the implementation flow of the behavior monitoring method according to the embodiment of the present application;

2 is a schematic diagram of a human body key point according to an embodiment of the application;

FIG. 3 is a second implementation flowchart of the behavior monitoring method according to an embodiment of the present application;

4 is a schematic diagram of single-person key point monitoring according to an embodiment of the application;

5 is a schematic diagram of multi-person key point monitoring according to an embodiment of the application;

6 is a first structural schematic diagram of a behavior monitoring device according to an embodiment of the present application;

FIG. 7 is a second schematic structural diagram of a behavior monitoring apparatus according to an embodiment of the present application.

Detailed ways

The present application will be described in further detail below with reference to the accompanying drawings and specific embodiments.

FIG. 1 is a schematic flowchart of the implementation of the behavior monitoring method in the embodiment of the present application. As shown in FIG. 1 , the method includes the following steps:

Step 101: Determine the target key point of the target object in the real-time video stream collected;

Among them, the target key points represent different parts of the target object, such as: neck, left wrist and so on. Determine the target object and the target key points of the target object in the acquired real-time video stream.

In practical applications, the camera can be used to collect video streams, process the collected video streams in real time, determine the target object in the video stream, locate the target object, and determine the target key points of the target object.

When determining the target object, the contour information of the object in the video stream can be determined, and the contour information of each object is compared with the preset contour information; the object whose contour information is consistent with the preset contour information is determined as the target object.

In one embodiment, the determining the target key points of the target object in the collected real-time video stream includes: identifying the target object in the video stream and the key points of the target object; The target keypoint is determined among the keypoints.

After using the camera to collect images, process the collected real-time video stream, and determine the target object, all key points of the target object are obtained; through the deep network model, the target key corresponding to the application scene is selected from all the key points of the target object. point. The application scenarios may include: elderly monitoring scenarios, patient monitoring scenarios and other application scenarios, and the deep network model selects corresponding target key points from key points according to different application scenarios.

For example: when the target object in the video stream is the elderly, the key points are the points corresponding to 18 parts, such as: key point 1, key point 2... key point 18, among which, different key points represent different parts. When the application scenario is elderly guardianship, according to the corresponding relationship between the elderly guardianship scene and the target key points in the deep network model, the target key points in the elderly guardianship scene are determined as six key points: key point 1, key point 3, Keypoint 7, Keypoint 11, Keypoint 14, Keypoint 16.

In one embodiment, the determining the target key points of the target object in the captured real-time video stream includes: identifying the target object in the video stream and the key points of the target object; identifying the key points of the target object Partitioning is performed to obtain at least one partition, and the partition includes at least one key point; and a key point of one partition in the partition is selected as the target key point.

After the camera is used for image acquisition, the collected real-time video stream is processed, and after the target object is determined, all key points of the target object are obtained; all key points of the target object are partitioned to obtain multiple partitions, each partition contains multiple A key point is selected, and multiple key points in one partition of the above partition are selected as target key points.

For example, the key points of the target object determined by the camera include points corresponding to 18 parts, such as: nose, neck, right shoulder, right elbow, right wrist, left shoulder, left elbow, left wrist, right hip, right knee, right ankle, left Hip, left knee, left eye, left eye, right eye, left ear and right ear. All keypoints are partitioned to form two partitions: face keypoint partition and leg keypoint partition. The key points of the face key point partition can include: the points corresponding to the nose, left eye, right eye, left ear and right ear; the key points of the leg key point partition can include: right hip, right knee, right ankle, left hip , the corresponding points of the left knee and left eye. When the facial behavior of the target object needs to be determined, the key points of the facial key point partition can be selected as the target key points; when the leg behavior of the target object needs to be determined, the key points of the leg key point partition can be selected as the target key points.

Step 102: Determine the behavior information of the target object according to the target key point;

Wherein, the behavior information may include: slope, angle and other information representing the behavior state of the target object.

After selecting the target key points from the key points of the target object, according to the information of the target key points, the deep network model is used to determine the behavior information of the target object.

In an embodiment, the determining the behavior information of the target object according to the target key points includes: determining the coordinate information of the target key points; according to the coordinate information, using a deep network model to determine the target object behavioral information.

According to the target key point, the obtained real-time video stream is used to determine the coordinate information of the target key point; the coordinate information of the target key point is input into the deep network model, and the deep network model brings the coordinate information of the target key point into the simulation of the linear regression model. Solve the equation and determine the behavior information of the target object.

The fitting equation is shown in formula (1):

为回归系数。Among them, y _i and _xi are the coordinate information of the target key point, e _i is the error between the determined coordinate information of the target key point and the actual coordinate information,

is the regression coefficient.

进行求解，将

作为目标对象的行为信息。Here, the coordinates of the target key points of the target object are brought into x _i , y _i of the above fitting equation, and according to the least squares method

to solve,

Information about the behavior of the target object.

Step 103: Determine the behavior state of the target object according to the behavior information.

The behavior state of the target object is determined according to the acquired behavior information, where the behavior state may include user behaviors such as leaning, falling, standing, and the like.

For example, the acquired behavior information of the target object is the slope, and if the angle corresponding to the slope is equal to 45 degrees, it can be determined that the behavior state of the target object is inclined.

In one embodiment, the method further includes: acquiring sample data of different behavior states of an object in different application scenarios; determining behavior information of the object according to the behavior states of the sample data; determining, according to the behavior information, target key points of the object; determine the corresponding relationship between application scenarios, behavior states, behavior information and target key points; establish the deep network model according to the corresponding relationship.

Before determining the target key points of the target object in the acquired video stream, the method further includes: modeling based on the behavior of the object. Wherein, the behavior information may include: slope, angle, and other information representing the behavior state of the target object; the behavior state may include user behaviors such as leaning, falling, standing, and the like.

In practical applications, sample data of different behavioral states of the object in different application scenarios are obtained, the behavioral state of the sample data is analyzed, the behavioral information of the object is determined, and the target key point of the object is determined by using a linear regression model according to the behavioral information. In this way, the corresponding relationship between application scenarios, behavioral states, behavioral information and target key points can be obtained, and the deep network model can be determined. The application scenarios may include: elderly monitoring scenarios, patient monitoring scenarios and other monitoring scenarios.

For example: obtain the sample data of the body tilt state of the elderly in the elderly monitoring scene, analyze the body tilt state of the elderly, determine the slope of the body in the tilt state of the elderly, that is, the behavior information, use a linear regression model to establish a fitting equation, and use the slope to establish a fitting equation. Bring into the fitting equation to solve, and obtain the coordinates of the target key points, so as to establish the corresponding relationship between the application scene, behavior state, behavior information and the target key points, and establish a deep network model according to the above corresponding relationship.

In an embodiment, the method further includes: setting an application scenario; correspondingly, the determining the behavior state of the target object according to the behavior information includes: determining the target according to the behavior information and the application scenario. The behavioral state of the object.

The application scene is set, and when the behavior state of the target object is determined according to the behavior information, the behavior state of the target object can also be jointly determined according to the behavior information and the application scene.

For example: the set application scenario is the scenario of patient monitoring, the acquired behavior information of the patient is the slope, and the angle corresponding to the slope is less than 45 degrees. In the above scenario of patient monitoring, when the angle corresponding to the slope is less than 45 degrees, it means that the patient The behavioral state is fall.

In an embodiment, the method further includes: acquiring a user identifier corresponding to the behavior state; and using the user identifier as a target address, outputting the behavior state of the target object.

Wherein, the user identification may include: a mobile phone number, a micro-signal code, and the like. Preset the user ID, after determining the behavior state of the target object, obtain the user ID corresponding to the behavior state of the target object, for example: when the behavior state of the target object is tilted, the corresponding user ID is the user ID of the family; the target object When the behavior status is fall, the corresponding user ID is the rescue number.

Taking the user ID as the target address, the behavior status of the target object is sent to the user ID related to the target object, or a call between the user ID and the target object is initiated according to the obtained user ID.

For example: when it is determined that the behavior state of the target object is falling, according to the user identification, a call between the user identification and the target object is initiated, or the user identification is used as the target address, and the behavioral state of the target object is sent to the user identification related to the target object. .

While sending the behavior state of the target object to the user ID, it can also give suggestions for dealing with the behavior state.

In the embodiment of the present application, the target key points of the target object in the collected real-time video stream are determined; the behavior information of the target object is determined according to the target key points; the behavior state of the target object is determined according to the behavior information ; In this way, the collected real-time video stream can be used to identify and monitor the key points of the human body to identify the user's behavior, and to monitor the behavior of a certain part of the target object in detail; when the abnormal behavior of the user is monitored, it can be targeted for the abnormal behavior. Behaviors are warned and suggestions are given, which further improves the user experience.

In this embodiment, taking a user as a patient as an example, the behavior monitoring method provided by this embodiment of the present application is described through a scenario. The main key points in this embodiment correspond to the target key points of the above-mentioned embodiments.

In terms of abnormal patient behavior monitoring, related technologies mainly include vehicle monitoring, contour detection, Gaussian background modeling, and Kinect sensor technology.

Vehicle monitoring technology needs to wear upper body vehicles, finger vehicles, hand single-node vehicles and other related products in limb recognition to monitor abnormal behavior. At the same time, it may also bring danger to some special groups.

When monitoring abnormal behavior through contour detection technology, it is necessary to perform contour detection on the collected monitoring images, and use the frame difference method to calculate the change in the number of pixels in the contour area of the elderly to determine the state of the elderly. Moreover, in the monitoring process, all possible fitting coordinates are all missed, and the tracking is lost.

When monitoring abnormal behavior through Gaussian background modeling technology, it is necessary to extract a video based on motion sequence technology and Radio Frequency Identification (RFID) tag technology, use the extracted video for Gaussian background modeling, and then extract the foreground The contour and matrix feature information of the target are compared, and the feature information of the foreground is compared with the abnormal action feature library of the person to determine whether there is abnormal behavior of the person in the foreground. Using RFID tag technology also needs to wear RFID tags for all elderly people who need to be monitored. The method based on template matching needs to collect behavioral images of each supervised person in advance, and extract features to form feature templates.

The bone information is obtained through the Kinect sensor technology, and the angular rotation and movement of the bone pair form matrix information, which is compared with the recorded dangerous action information to judge the gap. On the one hand, this technology requires the recording of dangerous action information in advance, which is not very flexible in mechanism, and requires the purchase of Kinect sensors, which increases hardware costs; on the other hand, Kinect can only track 20 key points in the body, and the head only uses A dot indicates that use is restricted in complex contexts. There are many inconveniences in using the above method in the actual application process, which reduces the use experience.

There are many problems in the application of related technologies to the monitoring of abnormal behavior of patients in the field of health care. Smart cameras cannot be used to monitor patients' limbs and behavior effectively, and to warn and notify family members and medical staff immediately when danger occurs. . The technical disadvantages of the related art are as follows:

1) Some existing technologies rely on wearing upper body carriers, finger carriers, hand single-node carriers and other related products to perform key point detection and then recognize actions. The hardware cost is high, and the solution is limited in actual scenarios. Sex is great. At the same time, considering that in the process of remote care, patients are not suitable to wear related vehicles due to their own reasons, and related vehicles may cause certain damage to themselves.

2) Some methods need to record abnormal behavior information in advance as a reference, which cannot achieve real-time identification. This cannot be detected in real time for some abnormal behaviors of many patients, which is likely to cause serious consequences and will not be rescued in time.

However, the embodiment of the present application does not require the patient to wear a vehicle and record dangerous actions in advance as a reference, and applies an algorithm capable of recognizing limb behavior to the camera monitoring device. By monitoring the key points of the patient's human body, the abnormal behavior of the patient can be automatically monitored and identified. , when the patient is in danger due to abnormal behavior, it will issue an alarm and contact rescue in time. Moreover, the embodiment of the present application adopts a new human body detection algorithm, which can detect 130 human body key points in real time, including 2×21 basic key points of the hand; 70 basic key points of the face, which are presented in the form of 3D color stick figures. , which integrates multiple functions such as real-time multi-person 2D pose estimation, dynamic 3D reconstruction and hand key point detection and other computer vision projects, and can well correspond to complex backgrounds, making it more competitive than traditional monitoring applications. As shown in Figure 2, the human body key points are represented in the form of 3D colored stick figures.

The embodiment of the present application adopts the novel body language recognition technology to monitor the abnormal behavior of the patient, which can make up for the deficiency that the traditional image processing technology only recognizes some postures of the patient, such as lying, sitting, and standing. In the traditional monitoring of abnormal patient behavior, it is difficult to capture the patient's facial expressions. In the new monitoring application, key points such as eyes and mouth are depicted, and body language and expressions can be recognized at the same time. The application can be applied to the auxiliary treatment and nursing of elderly care, auxiliary rehabilitation treatment, auxiliary treatment of mental diseases and other mental diseases.

In the embodiment of the present application, the incoming video source from the camera is read in real time, and the abnormal behavior of the patient can be judged through the identification of key points of the limb and the judgment of abnormal behavior parameters in a specific scene, and the intelligent voice communication with the patient can be carried out in time. When the abnormal behavior does not disappear within a short period of time, it can immediately issue a warning, and notify the patient's family members and medical staff in a timely manner through phone calls and text messages. Close to real people's communication and provide more personalized solutions.

The schematic flowchart of the embodiment of the present application is shown in FIG. 3 , including:

Step 301: obtain a real-time video stream;

Use existing high-definition webcams to obtain real-time video streams.

Step 302: Determine the main key points;

Using the deep network model, human body positioning is performed on the obtained video stream, key points are identified, and the main key points are selected from the key points. For example: the identified key points are a total of 18 points, namely: 0 nose, 1 neck, 2 right shoulder, 3 right elbow, 4 right wrist, 5 left shoulder, 6 left elbow, 7 left wrist, 8 right hip, 9 right knee , 10 right ankle, 11 left hip, 12 left knee, 13 left eye, 14 left eye, 15 right eye, 16 left ear and 17 right ear, the coordinates of each key point are a _i (x _i , y _i ), from Select the main key point from the key points. If the selected main key points are: 1 neck, 4 right wrist, 7 left wrist, 8 right hip, 10 right ankle and 11 left hip, it can be expressed as a _i (x _i , y _i ), (i=1, 4, 7 , 8, 10, 11), where neither the x-coordinate nor the y-coordinate is 0.

Step 303: Detect main key points;

Detect the main key points and obtain the coordinate information of the main key points;

Step 304: Fit the main key points to obtain the slope;

Perform linear regression on the main key points according to the linear regression model, and use the least squares method to obtain the slope of the regression line;

的计算如公式(2)，

的计算如公式(3)，其中

为计算获得的斜率。The linear regression model is shown in formula (1), and the least squares method is used to solve formula (1) to obtain the regression coefficient

is calculated as formula (2),

is calculated as formula (3), where

is the slope obtained for the calculation.

为回归系数。Among them, y _i and _xi are the coordinate information of the target key point, e _i is the error between the determined coordinate information of the target key point and the actual coordinate information,

is the regression coefficient.

Step 305: Determine whether the behavior is abnormal according to the determined slope;

Judging whether the behavior of the tested person is abnormal according to the slope; if the behavior is abnormal, go to step 306; if the behavior is normal, go back to step 303;

小于1时，说明被检测者的倾斜角度小于45°，则认为被检测者存在危险行为，有可能晕倒或摔倒，此时判断检测者行为异常。For example: when the slope

When it is less than 1, it means that the inclination angle of the detected person is less than 45°, and it is considered that the detected person has a dangerous behavior, and may faint or fall.

Step 306: conduct a voice inquiry according to the abnormal behavior;

Within the set time, if the voice message replied by the tested person is received, return to step 303;

If the positive feedback voice information of the tested person is received and recognized, the timing will be started and the monitoring will continue. If the abnormal behavior has been eliminated within a continuous period of time and the monitored person’s behavior is normal, no danger warning will be given this time. The system continues to monitor patient behavior.

Further, if the abnormal behavior is not eliminated within the set time after receiving and recognizing the positive feedback voice information of the tested person, immediately notify the patient's family members and medical staff, and take rescue as soon as possible after confirmation. .

Within the set time, if no voice message replied by the detected person is received, go to step 307 .

Step 307: Acquire a preset user ID, and output the abnormal behavior of the target object to the terminal corresponding to the user ID.

If no positive feedback from the monitored person is received through the voice recognition technology, the monitored person's family members and medical staff and other relevant personnel will be notified immediately through text messages and phone calls. rescue measures.

As shown in FIG. 4 , it is a schematic diagram of single-person key point monitoring according to the embodiment of the present application; the key points under the state of human fall are: 1 neck, 4 right wrist, 7 left wrist, 8 right hip, 10 right ankle, 11 left hip; As shown in FIG. 5 , which is a schematic diagram of multi-person key point monitoring according to an embodiment of the present application, different key points are determined according to different monitoring needs of different people.

The use characteristics of the embodiments of the present application include:

(1) Human body key points can be applied to different scenarios;

In the new abnormal patient behavior monitoring application, after the camera captures the 2D image, the key point detector will identify and mark the characteristic parts of the body, help the body tracking algorithm to understand the performance of each posture at different angles, and display the 3D color stick figure. form presented. Applying these key points of the human body to different dangerous scenarios through relevant rules is more flexible in practical applications.

(2) Tracking key points is more detailed;

Compared with traditional methods, which can only track 20 key points, the tracking capabilities of the new abnormal patient behavior monitoring application are much more detailed. For the same action, the traditional method perceives that a person is raising his hand, but the new patient abnormal behavior monitoring application based on the new generation recognition system can observe that the person is actually pointing his finger at something.

(3) Added facial tracking to recognize expressions;

In terms of face tracking, the entire head is just a point in the traditional method, but in the new application of abnormal patient behavior monitoring, the eyebrows, eyes, nose, mouth, etc. can be depicted by dozens of key points, and body language and expressions can be recognized.

(4) Build database modeling based on abnormal behavior of patients;

A database is established based on the observed patient behavior, and big data modeling technology is used to establish a model for the specific behavior of a certain disease, and to identify the deep representative meaning of the behavior. The model can be used to directly identify the behavior, evaluate the risk level, and give the recommended measures at the time of early warning.

The embodiment of this application does not require the patient to wear a vehicle, a label, or collect abnormal behaviors of the monitored person in advance, and does not require the hardware performance of the camera, nor the purchase of Kinect sensors and other equipment. can be monitored together. It is more universally applicable, improving the experience of patients and their family members, medical staff, etc.; and it can be monitored in real time, without delay when an emergency occurs due to abnormal patient behavior, and is suitable for multi-scenario applications in complex backgrounds.

On the basis of identifying the coordinates of each key point of the patient's limb, the embodiment of the present application further discriminates the abnormal behavior of the patient by defining the abnormal behavior of different scenarios, such as the elderly, mentally ill patients, etc.; The embodiment applies the new real-time recognition method of human body movement including key points of the human body to the fields of elderly care, assisted rehabilitation and other fields in the field of health care, combined with related technologies such as intelligent speech recognition, which has positive and novel industry application significance .

This embodiment provides a behavior monitoring device. As shown in FIG. 6 , the behavior monitoring device 60 includes: a first determination module 601, a second determination module 602, and a third determination module 603; wherein,

The first determination module 601 is used to determine the target key point of the target object in the real-time video stream collected;

A second determining module 602, configured to determine the behavior information of the target object according to the target key point;

The third determining module 603 is configured to determine the behavior state of the target object according to the behavior information.

In one embodiment, the first determination module 601 is configured to: identify the target object in the video stream and the key points of the target object; determine the key points in the key points according to the application scenario through a deep network model. target key points.

In one embodiment, the first determination module 601 is configured to: identify the target object in the video stream and the key points of the target object; partition the key points of the target object to obtain at least one The partition contains at least one key point; a key point of one of the partitions is selected as the target key point.

In one embodiment, the second determination module 602 is configured to: determine the coordinate information of the target key point; and use a deep network model to determine the behavior information of the target object according to the coordinate information.

In one embodiment, the behavior monitoring device 60 further includes: a modeling module 604, configured to acquire sample data of different behavior states of an object in different application scenarios; and determine behavior information of the object according to the behavior states of the sample data ; According to the behavior information, determine the target key point of the object; determine the corresponding relationship between the application scene, behavior state, behavior information and the target key point; establish the deep network model according to the corresponding relationship.

In an embodiment, the behavior monitoring device 60 further includes: a setting module 605, configured to set an application scenario; correspondingly, a third determination module 603, configured to: determine the target object according to the behavior information and the application scenario behavioral state.

In an embodiment, the behavior monitoring apparatus 60 further includes: a notification module 606, configured to acquire a user identifier corresponding to the behavior state; and output the behavior state of the target object by using the user identifier as a target address.

It should be noted that: when the behavior monitoring device provided in the above embodiment performs behavior monitoring, only the division of the above program modules is used as an example for illustration. In practical applications, the above processing can be allocated to different program modules according to needs. That is, the internal structure of the device is divided into different program modules to complete all or part of the processing described above. In addition, the behavior monitoring apparatus and the behavior monitoring method embodiments provided by the above embodiments belong to the same concept, and the specific implementation process thereof is detailed in the method embodiments, which will not be repeated here.

Based on the foregoing embodiments, an embodiment of the present application provides a behavior monitoring apparatus. As shown in FIG. 7 , the apparatus includes a processor 702 and a memory 701 for storing a computer program that can be executed on the processor 702; wherein, When the processor 702 is configured to run the computer program, to achieve:

Determine the target key point of the target object in the captured real-time video stream;

Determine the behavior information of the target object according to the target key point;

The behavior state of the target object is determined according to the behavior information.

The methods disclosed in the above embodiments of the present application may be applied to the processor 702 or implemented by the processor 702 . The processor 702 may be an integrated circuit chip with signal processing capability. In the implementation process, each step of the above-mentioned method may be completed by an integrated logic circuit of hardware in the processor 702 or an instruction in the form of software. The above-mentioned processor 702 may be a general-purpose processor, a DSP, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and the like. The processor 702 may implement or execute the methods, steps, and logical block diagrams disclosed in the embodiments of this application. A general purpose processor may be a microprocessor or any conventional processor or the like. The steps of the methods disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium, and the storage medium is located in the memory 701, and the processor 702 reads the information in the memory 701, and completes the steps of the foregoing method in combination with its hardware.

It can be understood that the memory (memory 701 ) in this embodiment of the present application may be a volatile memory or a non-volatile memory, and may also include both volatile and non-volatile memory. Among them, the non-volatile memory may be a read-only memory (ROM, Read Only Memory), a programmable read-only memory (PROM, Programmable Read-Only Memory), an erasable programmable read-only memory (EPROM, Erasable Programmable Read-Only Memory) Memory), Electrically Erasable Programmable Read-Only Memory (EEPROM, Electrically Erasable Programmable Read-Only Memory), Magnetic Random Access Memory (FRAM, ferromagnetic random access memory), Flash Memory (Flash Memory), Magnetic Surface Memory, Optical Disc , or compact disc read-only memory (CD-ROM, Compact Disc Read-Only Memory); the magnetic surface memory can be a magnetic disk memory or a tape memory. The volatile memory may be random access memory (RAM, Random Access Memory), which is used as an external cache memory. By way of example and not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory Memory (DRAM, Dynamic Random Access Memory), Synchronous Dynamic Random Access Memory (SDRAM, Synchronous Dynamic Random Access Memory), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM, Double Data Rate Synchronous Dynamic Random Access Memory), Enhanced Type Synchronous Dynamic Random Access Memory (ESDRAM, Enhanced Synchronous Dynamic Random Access Memory), Synchronous Link Dynamic Random Access Memory (SLDRAM, SyncLink Dynamic Random Access Memory), Direct Memory Bus Random Access Memory (DRRAM, Direct Rambus Random Access Memory). The memories described in the embodiments of the present application are intended to include, but not be limited to, these and any other suitable types of memories.

It should be pointed out here that the descriptions of the above terminal embodiments are similar to the descriptions of the above methods, and have the same beneficial effects as the method embodiments, so they will not be repeated. For technical details that are not disclosed in the terminal embodiments of the present application, those skilled in the art should refer to the description of the method embodiments of the present application to understand, and to save space, details are not repeated here.

In an exemplary embodiment, an embodiment of the present application further provides a computer-readable storage medium, for example, including a memory 701 storing a computer program, and the computer program can be processed by the processor 702 to complete the steps of the foregoing method. The computer-readable storage medium may be memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disk, or CD-ROM.

Embodiments of the present application also provide a computer-readable storage medium, on which a computer program is stored, and the computer program is implemented when processed by a processor:

Determine the target key point of the target object in the captured real-time video stream;

Determine the behavior information of the target object according to the target key point;

The behavior state of the target object is determined according to the behavior information.

It should be pointed out here that the descriptions of the above computer medium embodiments are similar to the descriptions of the above methods, and have the same beneficial effects as the method embodiments, so they are not repeated. For technical details that are not disclosed in the terminal embodiments of the present application, those skilled in the art should refer to the descriptions of the method embodiments of the present application to understand, and to save space, they will not be repeated here.

The above descriptions are only preferred embodiments of the present application, and are not intended to limit the protection scope of the present application.

Claims (10)

1. a behavior monitoring method, is characterized in that, described method comprises: Determine the target key point of the target object in the captured real-time video stream; Determine the behavior information of the target object according to the target key point; The behavior state of the target object is determined according to the behavior information. 2. The method according to claim 1, wherein the target key point of the target object in the real-time video stream that is determined to be collected comprises: Identifying a target object in the video stream and key points of the target object; Through the deep network model, the target key point is determined among the key points according to the application scenario. 3. The method according to claim 1, wherein the target key point of the target object in the real-time video stream that is determined to be collected comprises: Identifying a target object in the video stream and key points of the target object; Partition the key points of the target object to obtain at least one partition, and the partition includes at least one key point; A key point of one of the partitions is selected as the target key point. 4. The method according to claim 1, wherein the determining the behavior information of the target object according to the target key point comprises: Determine the coordinate information of the target key point; According to the coordinate information, the behavior information of the target object is determined by using a deep network model. 5. The method according to claim 2 or 4, wherein the method further comprises: Obtain sample data of different behavior states of objects in different application scenarios; Determine the behavior information of the object according to the behavior state of the sample data; According to the behavior information, determine the target key point of the object; Determine the correspondence between application scenarios, behavioral states, behavioral information and target key points; The deep network model is established according to the corresponding relationship. 6. The method of claim 1, wherein the method further comprises: Set application scenarios; Correspondingly, the determining the behavior state of the target object according to the behavior information includes: The behavior state of the target object is determined according to the behavior information and the application scenario. 7. The method of claim 1, wherein the method further comprises: Obtain the user ID corresponding to the behavior state; Taking the user ID as the target address, outputting the behavior status of the target object. 8. A behavior monitoring device, characterized in that the device comprises: a first determination module, a second determination module and a third determination module; wherein, The first determining module is used to determine the target key point of the target object in the collected real-time video stream; The second determining module is configured to determine the behavior information of the target object according to the target key point; The third determining module is configured to determine the behavior state of the target object according to the behavior information. 9. A behavior monitoring device, characterized in that it comprises a processor and a memory for storing a computer program that can be run on the processor; wherein, when the processor is used to run the computer program, it executes claims 1 to 1. 7 the steps of any one of the methods. 10. A computer-readable storage medium on which a computer program is stored, characterized in that, when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images