CN112115746A - Human body action recognition device and method and electronic equipment - Google Patents
Human body action recognition device and method and electronic equipment Download PDFInfo
- Publication number
- CN112115746A CN112115746A CN201910541727.3A CN201910541727A CN112115746A CN 112115746 A CN112115746 A CN 112115746A CN 201910541727 A CN201910541727 A CN 201910541727A CN 112115746 A CN112115746 A CN 112115746A
- Authority
- CN
- China
- Prior art keywords
- human body
- motion
- detecting
- recognition result
- detected
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 23
- 238000001514 detection method Methods 0.000 claims abstract description 77
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 19
- 238000012360 testing method Methods 0.000 claims description 4
- 210000001364 upper extremity Anatomy 0.000 claims description 3
- 238000004364 calculation method Methods 0.000 claims description 2
- 238000012545 processing Methods 0.000 abstract description 11
- 238000010586 diagram Methods 0.000 description 19
- 230000006870 function Effects 0.000 description 6
- 210000000707 wrist Anatomy 0.000 description 5
- 238000013135 deep learning Methods 0.000 description 4
- 238000012986 modification Methods 0.000 description 4
- 230000004048 modification Effects 0.000 description 4
- 210000003423 ankle Anatomy 0.000 description 3
- 210000003127 knee Anatomy 0.000 description 3
- 238000012549 training Methods 0.000 description 3
- 230000004075 alteration Effects 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 244000309466 calf Species 0.000 description 2
- 210000005069 ears Anatomy 0.000 description 2
- 210000003128 head Anatomy 0.000 description 2
- 210000002414 leg Anatomy 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 210000000689 upper leg Anatomy 0.000 description 2
- 230000006399 behavior Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 230000004886 head movement Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012806 monitoring device Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/23—Recognition of whole body movements, e.g. for sport training
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Biology (AREA)
- General Health & Medical Sciences (AREA)
- Psychiatry (AREA)
- Social Psychology (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the invention provides a human body action recognition device and method and electronic equipment. The method comprises the steps of firstly detecting a boundary frame of a human body in an input image, and selectively detecting the motion of the human body based on key points of the human body and/or detecting the motion of the human body based on a convolutional neural network in the detected boundary frame, so that the processing speed is high and the recognition accuracy is high through a hierarchical detection mode, and different detection modes can be selected according to different conditions by combining the two detection modes, so that various scenes and requirements can be flexibly met.
Description
Technical Field
The invention relates to the technical field of information.
Background
In recent years, with the help of deep learning, research in the field of computer vision has been greatly advanced. Deep learning refers to an algorithm set for solving various problems such as images and texts by applying various machine learning algorithms on a hierarchical neural network. The core of deep learning is feature learning, and aims to obtain hierarchical feature information through a hierarchical neural network, so that the important problem that features need to be designed manually in the past is solved.
Safety monitoring is one of important applications of deep learning, and human body action and behavior recognition are important components of safety monitoring.
It should be noted that the above background description is only for the sake of clarity and complete description of the technical solutions of the present invention and for the understanding of those skilled in the art. Such solutions are not considered to be known to the person skilled in the art merely because they have been set forth in the background section of the invention.
Disclosure of Invention
However, since the motion of the human body is relatively complex and the applied scenes are also variable, the conventional motion recognition method has a low processing speed and low recognition accuracy, and cannot flexibly cope with various different scenes and requirements.
The embodiment of the invention provides a human body action recognition device and method and electronic equipment.
According to a first aspect of embodiments of the present invention, there is provided a human body motion recognition apparatus, the apparatus including: an object detection unit for detecting a bounding box of a human body in an input image; a first detection unit, configured to calculate, in a detected human body boundary frame, a feature of the human body based on the key point of the human body, and detect a motion of the human body according to the feature of the human body, to obtain a first recognition result; a second detection unit for detecting a motion of the human body based on a convolutional neural network in the detected boundary frame of the human body to obtain a second recognition result; and a selection unit for selecting at least one of the first detection unit and the second detection unit to detect the motion of the human body to obtain at least one of the first recognition result and the second recognition result.
According to a second aspect of embodiments of the present invention, there is provided an electronic device comprising the apparatus according to the first aspect of embodiments of the present invention.
According to a third aspect of the embodiments of the present invention, there is provided a human body motion recognition method, the method including: detecting a bounding box of a human body in an input image; selecting and performing at least one of the following tests: calculating the characteristics of the human body based on the key points of the human body in the detected boundary frame of the human body, and detecting the action of the human body according to the characteristics of the human body to obtain a first recognition result; and detecting the motion of the human body based on the convolutional neural network in the detected boundary frame of the human body to obtain a second recognition result.
The invention has the beneficial effects that: the method comprises the steps of firstly detecting a boundary frame of a human body in an input image, and selectively detecting the motion of the human body based on key points of the human body and/or detecting the motion of the human body based on a convolutional neural network in the detected boundary frame, so that the processing speed is high and the recognition accuracy is high through a hierarchical detection mode, and different detection modes can be selected according to different conditions by combining the two detection modes, so that various scenes and requirements can be flexibly met.
Specific embodiments of the present invention are disclosed in detail with reference to the following description and drawings, indicating the manner in which the principles of the invention may be employed. It should be understood that the embodiments of the invention are not so limited in scope. The embodiments of the invention include many variations, modifications and equivalents within the spirit and scope of the appended claims.
Features that are described and/or illustrated with respect to one embodiment may be used in the same way or in a similar way in one or more other embodiments, in combination with or instead of the features of the other embodiments.
It should be emphasized that the term "comprises/comprising" when used herein, is taken to specify the presence of stated features, integers, steps or components but does not preclude the presence or addition of one or more other features, integers, steps or components.
Drawings
The accompanying drawings, which are included to provide a further understanding of the embodiments of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention. It is obvious that the drawings in the following description are only some embodiments of the invention, and that for a person skilled in the art, other drawings can be derived from them without inventive effort. In the drawings:
fig. 1 is a schematic view of a human body motion recognition apparatus according to embodiment 1 of the present invention;
fig. 2 is a schematic diagram of the first detecting unit 102 according to embodiment 1 of the present invention;
FIG. 3 is a schematic diagram of the detection result of the key points of the human body in embodiment 1 of the present invention;
FIG. 4 is a schematic diagram of obtaining human body features based on key points according to embodiment 1 of the present invention;
fig. 5 is a schematic diagram of human motion recognition by the human motion recognition apparatus 100 according to embodiment 1 of the present invention;
fig. 6 is a schematic view of an electronic device according to embodiment 2 of the present invention;
fig. 7 is a schematic block diagram of a system configuration of an electronic apparatus according to embodiment 2 of the present invention;
fig. 8 is a schematic diagram of a human body motion recognition method according to embodiment 3 of the present invention.
Detailed Description
The foregoing and other features of the invention will become apparent from the following description taken in conjunction with the accompanying drawings. In the description and drawings, particular embodiments of the invention have been disclosed in detail as being indicative of some of the embodiments in which the principles of the invention may be employed, it being understood that the invention is not limited to the embodiments described, but, on the contrary, is intended to cover all modifications, variations, and equivalents falling within the scope of the appended claims.
Example 1
The embodiment of the invention provides a human body action recognition device. Fig. 1 is a schematic view of a human body motion recognition apparatus according to embodiment 1 of the present invention.
As shown in fig. 1, the human motion recognition apparatus 100 includes:
an object detection unit 101 for detecting a bounding box of a human body in an input image;
a first detection unit 102 configured to calculate a feature of a human body based on a key point of the human body in the detected human body boundary frame, and detect a motion of the human body from the feature of the human body to obtain a first recognition result;
a second detection unit 103 for detecting a motion of the human body based on the convolutional neural network in the detected boundary frame of the human body, and obtaining a second recognition result; and
a selecting unit 104 for selecting at least one of the first detecting unit 102 and the second detecting unit 103 to detect the motion of the human body to obtain at least one of the first recognition result and the second recognition result.
It can be seen from the above embodiments that, firstly, a boundary frame of a human body is detected in an input image, and a motion of the human body is detected based on key points of the human body and/or a motion of the human body is detected based on a convolutional neural network selectively in the detected boundary frame, so that a processing speed is high and an identification accuracy is high through a hierarchical detection method, and different detection methods can be selected according to different situations by combining the two detection methods, so that various scenes and requirements can be flexibly met.
In this embodiment, the input image may be an image obtained in real time or obtained in advance. For example, the input images are video images captured by the monitoring device, and each input image corresponds to one frame of the video image.
In the present embodiment, the object detection unit 101 is configured to detect a bounding box of a human body in an input image. The object detection unit 101 can perform detection based on various object detection methods, for example, fast R-CNN, FPN, Yolo network, and the like.
In this embodiment, different networks may be used for detection according to different requirements, for example, a Yolo network may be used when the requirement on the processing speed is high, and a fast R-CNN network may be used when the requirement on the identification accuracy is high.
By the object detection unit 101, when at least one human body exists in the input image, a bounding box of the at least one human body is detected. After detecting the boundary frame of the human body, the selection unit 104 selects at least one of the first detection unit 102 and the second detection unit 103 to detect the motion of the human body to obtain at least one of the first recognition result and the second recognition result.
In this embodiment, the selection unit 104 may select at least one of the first detection unit 102 and the second detection unit 103 to detect the motion of the human body according to actual needs or application scenarios to obtain at least one of the first recognition result and the second recognition result. When the object detection unit 101 detects a plurality of boundary frames of a plurality of human bodies in the input image, the first detection unit 102 and/or the second detection unit 103 detects the plurality of boundary frames one by one according to the selection result of the selection unit 104.
For example, for a case where only a simple motion needs to be detected, for example, a simple trunk motion of walking, standing, sitting, etc., in this case, the selection unit 104 selects the second detection unit 103 to detect the motion of the human body, that is, outputs the second recognition result.
In the present embodiment, second detection section 103 detects the motion of the human body based on the Convolutional Neural Network (CNN) in the boundary box of the detected human body, and obtains a second recognition result.
In this embodiment, detection of torso motion may be implemented using popular CNN networks, for example, using AlexNet networks.
In this embodiment, when training the CNN network, a training data set may be established, where the training data set includes images of human bodies whose actions are labeled "walk", "stand", "sit", "run", "squat", and "lie", and these images may be obtained from an open data set.
For another example, in the case where it is necessary to simultaneously detect a simple motion and a more complex motion, for example, a more complex partial motion such as raising the head and raising the hand in addition to a simple trunk motion such as walking, standing, sitting, etc., the selection unit 104 selects the first detection unit 102 to detect a motion of the human body, that is, outputs the first recognition result. Alternatively, the first detection unit 102 and the second detection unit 103 may be selected to perform detection at the same time, that is, the first recognition result and the second recognition result may be output.
In the present embodiment, the first detection unit 102 calculates a feature of the human body based on the key point of the human body in the detected boundary frame of the human body, and detects the motion of the human body from the feature of the human body, thereby obtaining a first recognition result.
Fig. 2 is a schematic diagram of the first detecting unit 102 according to embodiment 1 of the present invention. As shown in fig. 2, the first detection unit 102 includes:
a first detection module 201, configured to detect a key point of a human body in a detected human body bounding box;
a calculating module 202 for calculating features of the human body from the detected key points of the human body; and
and the second detection module 203 is configured to detect the motion of the human body based on the classifier and/or a preset rule according to the calculated characteristics of the human body, so as to obtain a first recognition result.
In this embodiment, the first detection module 201 may detect key-points (keys) of the human body based on various methods, for example, the first detection module 201 detects key-points (keys) of the human body based on a Cascaded Pyramid Network (CPN). Alternatively, the detection may be performed by a method such as Open-dose or Alpha-dose.
In the present embodiment, the key points of the human body may include a plurality of points respectively representing positions where a plurality of parts of the human body are located, for example, points respectively representing two ears, two eyes, a nose, two shoulders, two elbows, two wrists, two hips, two knees, and two ankles of the human body.
Fig. 3 is a schematic diagram of a detection result of a key point of a human body in embodiment 1 of the present invention. As shown in fig. 3, in the bounding box of one human body, key points representing respective parts of the human body are detected by the CPN and position information of the key points can be output.
In this embodiment, the calculating module 202 calculates the feature of the human body according to the key points of the human body detected by the first detecting module 201, for example, the feature of the human body may include: two-dimensional coordinates of a plurality of points respectively representing positions of a plurality of parts of the human body; and at least one angle between the connecting lines of the plurality of points.
In this embodiment, the features of the human body to be calculated may be determined according to actual needs.
Fig. 4 is a schematic diagram of obtaining features of a human body based on key points in embodiment 1 of the present invention. As shown in fig. 4, the key points for calculating the features include the points where the following human body parts are located: nose, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle, and right ankle. The calculated characteristics of the human body include two-dimensional coordinates of the points, for example, X and Y coordinates of the points, and in addition, the characteristics of the human body may further include a 1 st angle between the left leg and the torso, a 2 nd angle between the right leg and the torso, a 3 rd angle between the left calf and the left thigh, and a 4 th angle between the right calf and the right thigh.
After the calculation module 202 calculates the features of the human body, the second detection module 203 detects the motion of the human body based on the classifier and/or the preset rule according to the calculated features of the human body, and obtains a first recognition result.
In this embodiment, the second detecting module 203 may detect the trunk motion of the human body based on the classifier according to the calculated features of the human body, and detect the head motion and the upper limb motion of the human body based on a preset rule.
In this embodiment, the second detection module 203 may detect the torso motion of the human body based on various classifiers, for example, the second detection module 203 may detect based on a Multi-Layer Perceptron (MLP) classifier. And the detection is carried out according to the calculated characteristics and based on an MLP classifier, so that better detection performance can be obtained.
In this embodiment, the second detecting module 203 may further detect head movements and upper limb movements of the human body based on preset rules, for example, head-up movements, head-down movements, hand-up movements, and the like. Preset rules can be set for different actions according to actual needs, for example, when the heights of the two ears are higher than the heights of the two eyes, the user is judged to look down; when the height of the wrist is higher than that of the elbow, the wrist is judged to be lifted.
Fig. 5 is a schematic diagram of human motion recognition performed by the human motion recognition apparatus 100 according to embodiment 1 of the present invention. As shown in fig. 5, an input image including a plurality of human bodies is input to the object detection unit 101, the object detection unit 101 detects a bounding box of each human body in the input image and outputs the bounding box to the first detection unit 102 and the second detection unit 103, respectively, the first detection unit 102 and the second detection unit 103 perform detection according to a selection result of the selection unit 104, and output at least one of a first recognition result and a second recognition result.
It can be seen from the above embodiments that, firstly, a boundary frame of a human body is detected in an input image, and a motion of the human body is detected based on key points of the human body and/or a motion of the human body is detected based on a convolutional neural network selectively in the detected boundary frame, so that a processing speed is high and an identification accuracy is high through a hierarchical detection method, and different detection methods can be selected according to different situations by combining the two detection methods, so that various scenes and requirements can be flexibly met.
Example 2
An embodiment of the present invention further provides an electronic device, and fig. 6 is a schematic diagram of the electronic device in embodiment 2 of the present invention. As shown in fig. 6, the electronic device 600 includes a human motion recognition device 601, and the structure and function of the human motion recognition device 601 are the same as those described in embodiment 1, and are not described again here.
Fig. 7 is a schematic block diagram of a system configuration of an electronic apparatus according to embodiment 2 of the present invention. As shown in fig. 7, the electronic device 700 may include a central processor 701 and a memory 702; the memory 702 is coupled to the central processor 701. The figure is exemplary; other types of structures may also be used in addition to or in place of the structure to implement telecommunications or other functions.
As shown in fig. 7, the electronic device 700 may further include: an input unit 703, a display 704, and a power source 705.
In one embodiment, the functions of the human motion recognition device described in embodiment 1 may be integrated into the central processor 701. Wherein, the central processor 701 may be configured to: detecting a bounding box of a human body in an input image; selecting and performing at least one of the following tests: calculating the characteristics of the human body based on the key points of the human body in the detected boundary frame of the human body, and detecting the action of the human body according to the characteristics of the human body to obtain a first recognition result; and detecting the motion of the human body based on the convolutional neural network in the detected boundary frame of the human body to obtain a second recognition result.
For example, in a detected human body bounding box, calculating features of the human body based on key points of the human body, and detecting motion of the human body according to the features of the human body to obtain a first recognition result, the method includes: detecting key points of the human body in a detected boundary frame of the human body; calculating the characteristics of the human body according to the detected key points of the human body; and detecting the action of the human body based on a classifier and/or a preset rule according to the calculated characteristics of the human body to obtain a first recognition result.
In another embodiment, the human motion recognition device described in embodiment 1 may be configured separately from the central processor 701, for example, the human motion recognition device may be configured as a chip connected to the central processor 701, and the function of the human motion recognition device is realized by the control of the central processor 701.
It is not necessary that the electronic device 700 in this embodiment include all of the components shown in fig. 7.
As shown in fig. 7, a central processing unit 701, also sometimes referred to as a controller or operational control, may include a microprocessor or other processor device and/or logic device, the central processing unit 701 receiving inputs and controlling the operation of the various components of the electronic device 700.
The memory 702, for example, may be one or more of a buffer, a flash memory, a hard drive, a removable media, a volatile memory, a non-volatile memory, or other suitable device. And the central processor 701 can execute the program stored in the memory 702 to realize information storage or processing, or the like. The functions of other parts are similar to the prior art and are not described in detail here. The various components of electronic device 700 may be implemented in dedicated hardware, firmware, software, or combinations thereof, without departing from the scope of the invention.
It can be seen from the above embodiments that, firstly, a boundary frame of a human body is detected in an input image, and a motion of the human body is detected based on key points of the human body and/or a motion of the human body is detected based on a convolutional neural network selectively in the detected boundary frame, so that a processing speed is high and an identification accuracy is high through a hierarchical detection method, and different detection methods can be selected according to different situations by combining the two detection methods, so that various scenes and requirements can be flexibly met.
Example 3
The embodiment of the invention also provides a human body action recognition method, which corresponds to the human body action recognition device in the embodiment 1. Fig. 8 is a schematic diagram of a human body motion recognition method according to embodiment 3 of the present invention. As shown in fig. 8, the method includes:
step 801: detecting a bounding box of a human body in an input image; and
step 802: selecting and performing at least one of the following tests: calculating the characteristics of the human body based on the key points of the human body in the detected boundary frame of the human body, and detecting the action of the human body according to the characteristics of the human body to obtain a first recognition result; and detecting the motion of the human body based on the convolutional neural network in the detected boundary frame of the human body to obtain a second recognition result.
In this embodiment, the specific implementation method of the above steps is the same as that described in embodiment 1, and is not repeated here.
It can be seen from the above embodiments that, firstly, a boundary frame of a human body is detected in an input image, and a motion of the human body is detected based on key points of the human body and/or a motion of the human body is detected based on a convolutional neural network selectively in the detected boundary frame, so that a processing speed is high and an identification accuracy is high through a hierarchical detection method, and different detection methods can be selected according to different situations by combining the two detection methods, so that various scenes and requirements can be flexibly met.
An embodiment of the present invention further provides a computer-readable program, where when the program is executed in a human body motion recognition apparatus or an electronic device, the program causes a computer to execute the human body motion recognition method described in embodiment 3 in the human body motion recognition apparatus or the electronic device.
An embodiment of the present invention further provides a storage medium storing a computer-readable program, where the computer-readable program enables a computer to execute the human motion recognition method described in embodiment 3 in a human motion recognition apparatus or an electronic device.
The human body motion recognition apparatus or the method for performing human body motion recognition in an electronic device described in connection with the embodiments of the present invention may be directly embodied as hardware, a software module executed by a processor, or a combination of the two. For example, one or more of the functional block diagrams and/or one or more combinations of the functional block diagrams illustrated in fig. 1 may correspond to individual software modules of a computer program flow or may correspond to individual hardware modules. These software modules may correspond to the steps shown in fig. 8, respectively. These hardware modules may be implemented, for example, by solidifying these software modules using a Field Programmable Gate Array (FPGA).
A software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. A storage medium may be coupled to the processor such that the processor can read information from, and write information to, the storage medium; or the storage medium may be integral to the processor. The processor and the storage medium may reside in an ASIC. The software module may be stored in the memory of the mobile terminal or in a memory card that is insertable into the mobile terminal. For example, if the electronic device employs a relatively large capacity MEGA-SIM card or a large capacity flash memory device, the software module may be stored in the MEGA-SIM card or the large capacity flash memory device.
One or more of the functional block diagrams and/or one or more combinations of the functional block diagrams described with respect to fig. 1 may be implemented as a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any suitable combination thereof designed to perform the functions described herein. One or more of the functional block diagrams and/or one or more combinations of the functional block diagrams described with respect to fig. 1 may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP communication, or any other such configuration.
While the invention has been described with reference to specific embodiments, it will be apparent to those skilled in the art that these descriptions are illustrative and not intended to limit the scope of the invention. Various modifications and alterations of this invention will become apparent to those skilled in the art based upon the spirit and principles of this invention, and such modifications and alterations are also within the scope of this invention.
Claims (10)
1. A human motion recognition device, the device comprising:
an object detection unit for detecting a bounding box of a human body in an input image;
a first detection unit, configured to calculate, in a detected human body boundary frame, a feature of the human body based on the key point of the human body, and detect a motion of the human body according to the feature of the human body, to obtain a first recognition result;
a second detection unit for detecting a motion of the human body based on a convolutional neural network in the detected boundary frame of the human body to obtain a second recognition result; and
a selection unit for selecting at least one of the first detection unit and the second detection unit to detect the motion of the human body to obtain at least one of the first recognition result and the second recognition result.
2. The apparatus of claim 1, wherein the first detection unit comprises:
a first detection module for detecting a key point of the human body in a detected human body boundary box;
the calculation module is used for calculating the characteristics of the human body according to the detected key points of the human body; and
and the second detection module is used for detecting the action of the human body based on a classifier and/or a preset rule according to the calculated characteristics of the human body to obtain a first recognition result.
3. The apparatus of claim 2, wherein,
the key points of the human body include a plurality of points respectively representing positions of a plurality of parts of the human body.
4. The apparatus of claim 2, wherein,
the features of the human body include:
two-dimensional coordinates of a plurality of points respectively representing positions of a plurality of parts of the human body; and
at least one angle between the connecting lines of the plurality of points.
5. The apparatus of claim 2, wherein,
the first detection module detects key points of the human body based on a Cascaded Pyramid Network (CPN).
6. The apparatus of claim 2, wherein,
the classifier is a multi-level perceptron (MLP) classifier.
7. The apparatus of claim 1, wherein,
the second detection module detects the trunk action of the human body based on the classifier according to the calculated characteristics of the human body, and detects the head action and the upper limb action of the human body based on a preset rule.
8. An electronic device comprising the apparatus of any one of claims 1-7.
9. A human motion recognition method, the method comprising:
detecting a bounding box of a human body in an input image;
selecting and performing at least one of the following tests: calculating the characteristics of the human body based on the key points of the human body in the detected boundary frame of the human body, and detecting the action of the human body according to the characteristics of the human body to obtain a first recognition result; and detecting the motion of the human body based on the convolutional neural network in the detected boundary frame of the human body to obtain a second recognition result.
10. The method according to claim 9, wherein calculating features of the human body based on key points of the human body in the detected bounding box of the human body, and detecting motion of the human body according to the features of the human body to obtain a first recognition result comprises:
detecting key points of the human body in a detected boundary frame of the human body;
calculating the characteristics of the human body according to the detected key points of the human body; and
and detecting the action of the human body based on a classifier and/or a preset rule according to the calculated characteristics of the human body to obtain a first recognition result.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910541727.3A CN112115746B (en) | 2019-06-21 | 2019-06-21 | Human body action recognition device and method and electronic equipment |
| JP2020086788A JP7419964B2 (en) | 2019-06-21 | 2020-05-18 | Human motion recognition device and method, electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910541727.3A CN112115746B (en) | 2019-06-21 | 2019-06-21 | Human body action recognition device and method and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112115746A true CN112115746A (en) | 2020-12-22 |
| CN112115746B CN112115746B (en) | 2024-06-18 |
Family
ID=73796387
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910541727.3A Active CN112115746B (en) | 2019-06-21 | 2019-06-21 | Human body action recognition device and method and electronic equipment |
Country Status (2)
| Country | Link |
|---|---|
| JP (1) | JP7419964B2 (en) |
| CN (1) | CN112115746B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105590099A (en) * | 2015-12-22 | 2016-05-18 | 中国石油大学(华东) | Multi-user behavior identification method based on improved convolutional neural network |
| CN106778796A (en) * | 2016-10-20 | 2017-05-31 | 江苏大学 | Human motion recognition method and system based on hybrid cooperative model training |
| CN106909887A (en) * | 2017-01-19 | 2017-06-30 | 南京邮电大学盐城大数据研究院有限公司 | A kind of action identification method based on CNN and SVM |
| CN107832708A (en) * | 2017-11-09 | 2018-03-23 | 云丁网络技术(北京)有限公司 | A kind of human motion recognition method and device |
| CN108197575A (en) * | 2018-01-05 | 2018-06-22 | 中国电子科技集团公司电子科学研究院 | A kind of abnormal behaviour recognition methods detected based on target detection and bone point and device |
| CN108960192A (en) * | 2018-07-23 | 2018-12-07 | 北京旷视科技有限公司 | Action identification method and its neural network generation method, device and electronic equipment |
| CN109101901A (en) * | 2018-07-23 | 2018-12-28 | 北京旷视科技有限公司 | Human action identification and its neural network generation method, device and electronic equipment |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170316578A1 (en) | 2016-04-29 | 2017-11-02 | Ecole Polytechnique Federale De Lausanne (Epfl) | Method, System and Device for Direct Prediction of 3D Body Poses from Motion Compensated Sequence |
| CN108229305B (en) | 2017-11-21 | 2021-06-04 | 北京市商汤科技开发有限公司 | Method and device for determining outer frame of target object and electronic equipment |
| CN109241829B (en) | 2018-07-25 | 2020-12-04 | 中国科学院自动化研究所 | Behavior identification method and device based on space-time attention convolutional neural network |
| CN109145867B (en) | 2018-09-07 | 2021-08-10 | 北京旷视科技有限公司 | Human body posture estimation method, device, system, electronic equipment and storage medium |
| CN109214346B (en) | 2018-09-18 | 2022-03-29 | 中山大学 | Picture human body action recognition method based on hierarchical information transmission |
-
2019
- 2019-06-21 CN CN201910541727.3A patent/CN112115746B/en active Active
-
2020
- 2020-05-18 JP JP2020086788A patent/JP7419964B2/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105590099A (en) * | 2015-12-22 | 2016-05-18 | 中国石油大学(华东) | Multi-user behavior identification method based on improved convolutional neural network |
| CN106778796A (en) * | 2016-10-20 | 2017-05-31 | 江苏大学 | Human motion recognition method and system based on hybrid cooperative model training |
| CN106909887A (en) * | 2017-01-19 | 2017-06-30 | 南京邮电大学盐城大数据研究院有限公司 | A kind of action identification method based on CNN and SVM |
| CN107832708A (en) * | 2017-11-09 | 2018-03-23 | 云丁网络技术(北京)有限公司 | A kind of human motion recognition method and device |
| CN108197575A (en) * | 2018-01-05 | 2018-06-22 | 中国电子科技集团公司电子科学研究院 | A kind of abnormal behaviour recognition methods detected based on target detection and bone point and device |
| CN108960192A (en) * | 2018-07-23 | 2018-12-07 | 北京旷视科技有限公司 | Action identification method and its neural network generation method, device and electronic equipment |
| CN109101901A (en) * | 2018-07-23 | 2018-12-28 | 北京旷视科技有限公司 | Human action identification and its neural network generation method, device and electronic equipment |
Non-Patent Citations (2)
| Title |
|---|
| 司阳;肖秦琨;李兴;: "基于自动编码器和神经网络的人体运动识别", 国外电子测量技术, no. 01 * |
| 宫法明;李然;马玉辉;: "复杂场景下的运动目标识别算法", 计算机系统应用, no. 08 * |
Also Published As
| Publication number | Publication date |
|---|---|
| JP2021002332A (en) | 2021-01-07 |
| JP7419964B2 (en) | 2024-01-23 |
| CN112115746B (en) | 2024-06-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109448090B (en) | Image processing method, device, electronic equipment and storage medium | |
| US11423699B2 (en) | Action recognition method and apparatus and electronic equipment | |
| CN111476097A (en) | Human body posture assessment method and device, computer equipment and storage medium | |
| CN110321795B (en) | User gesture recognition method and device, computer device and computer storage medium | |
| Chaudhari et al. | Yog-guru: Real-time yoga pose correction system using deep learning methods | |
| CN110688929B (en) | Human skeleton joint point positioning method and device | |
| CN109176512A (en) | A kind of method, robot and the control device of motion sensing control robot | |
| US11940774B2 (en) | Action imitation method and robot and computer readable storage medium using the same | |
| CN109960962B (en) | Image recognition method and device, electronic equipment and readable storage medium | |
| CN108304819B (en) | Gesture recognition system and method, and storage medium | |
| CN110874865A (en) | Three-dimensional skeleton generation method and computer equipment | |
| CN113658211B (en) | User gesture evaluation method and device and processing equipment | |
| US20150379333A1 (en) | Three-Dimensional Motion Analysis System | |
| KR20210006722A (en) | Apparatus, method and computer program for determining whether safety equipment is worn | |
| CN110390303A (en) | Tumble alarm method, electronic device, and computer-readable storage medium | |
| US20220362630A1 (en) | Method, device, and non-transitory computer-readable recording medium for estimating information on golf swing | |
| CN111222379A (en) | Hand detection method and device | |
| CN111368787A (en) | Video processing method and device, equipment and computer readable storage medium | |
| CN113033526A (en) | Computer-implemented method, electronic device and computer program product | |
| CN112115746B (en) | Human body action recognition device and method and electronic equipment | |
| CN109531578B (en) | Humanoid mechanical arm somatosensory control method and device | |
| CN115100745B (en) | Swin transducer model-based motion real-time counting method and system | |
| US11527090B2 (en) | Information processing apparatus, control method, and non-transitory storage medium | |
| CN112417205A (en) | Target retrieval device and method and electronic equipment | |
| CN113724176B (en) | Multi-camera motion capture seamless connection method, device, terminal and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |