WO2022237481A1 - 举手识别方法、装置、电子设备及存储介质 - Google Patents

举手识别方法、装置、电子设备及存储介质 Download PDF

Info

Publication number
WO2022237481A1
WO2022237481A1 PCT/CN2022/088032 CN2022088032W WO2022237481A1 WO 2022237481 A1 WO2022237481 A1 WO 2022237481A1 CN 2022088032 W CN2022088032 W CN 2022088032W WO 2022237481 A1 WO2022237481 A1 WO 2022237481A1
Authority
WO
WIPO (PCT)
Prior art keywords
hand
human body
score
key points
key point
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2022/088032
Other languages
English (en)
French (fr)
Inventor
卢子鹏
王健
孙昊
丁二锐
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Baidu Netcom Science and Technology Co Ltd
Original Assignee
Beijing Baidu Netcom Science and Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Baidu Netcom Science and Technology Co Ltd filed Critical Beijing Baidu Netcom Science and Technology Co Ltd
Publication of WO2022237481A1 publication Critical patent/WO2022237481A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/107—Static hand or arm

Definitions

  • the present disclosure relates to the field of image processing, and in particular to the fields of artificial intelligence such as computer vision and deep learning.
  • the collected images to be recognized are directly input into the training detection model, and image recognition is directly performed on the images to be recognized.
  • the acquired image to be recognized is a distant image, in which the human hand is usually small and not clear enough, and the characteristics of the human body's hand-raising movement are relatively blurred, which makes the hand-raising recognition accurate.
  • the rate is lower. Therefore, how to realize high-accuracy hand-raising recognition in the distant view image with insufficient clarity has become a problem to be solved.
  • the present disclosure proposes a method, device, electronic equipment, storage medium, and computer program product for hand-raising recognition.
  • a method for raising hands including: extracting key points of the upper body of the human body from the collected human body images, and extracting first image features based on the key points of the upper body, and inputting them into the human body action recognition model , the output is recognized as the first score of the human body performing the hand-raising action; determine the key points of the hand from the key points of the upper body; based on the key points of the hand, extract the second image feature of the human hand, and input it into the human hand classification model, Outputting a second score that is identified as the palm being in an open state; performing recognition of the state of raising the hand according to the first score and the second score.
  • a hand-raising recognition device including: an extraction module for extracting key points of the upper body of the human body from the collected human body images, and extracting first image features based on the key points of the upper body, Input in the human body action recognition model, the output is recognized as the first score of the human body performing the hand-raising action; the extraction module is also used to determine the key points of the hand from the key points of the upper body; the extraction module is also used to Based on the key points of the hand, the second image feature of the hand is extracted, input into the hand classification model, and the output is recognized as the second score of the palm being in an open state; the identification module is used for according to the first score and the The second score is for the recognition of the state of raising hands.
  • an electronic device including: at least one processor; and a memory connected in communication with the at least one processor; wherein, the memory stores information that can be used by the at least one processor Executable instructions, the instructions are executed by the at least one processor, so that the at least one processor can execute the method for recognizing raised hands proposed in the first aspect above.
  • a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute the method for recognizing raised hands proposed in the first aspect above.
  • a computer program product including a computer program, and when the computer program is executed by a processor, the method for recognizing raised hands proposed in the above-mentioned first aspect is implemented.
  • FIG. 1 is a schematic flowchart of a method for raising hands recognition according to an embodiment of the present disclosure
  • FIG. 2 is a schematic flowchart of a method for raising hands recognition according to another embodiment of the present disclosure
  • FIG. 3 is a schematic flowchart of a method for raising hands recognition according to another embodiment of the present disclosure
  • FIG. 4 is a schematic flowchart of a method for raising hands recognition according to another embodiment of the present disclosure.
  • FIG. 5 is a schematic flowchart of a method for raising hands recognition according to another embodiment of the present disclosure.
  • FIG. 6 is a schematic flowchart of a method for raising a hand recognition according to another embodiment of the present disclosure.
  • FIG. 7 is a schematic flowchart of a method for raising a hand recognition according to another embodiment of the present disclosure.
  • Fig. 8 is a schematic structural diagram of a hand-raising recognition device according to an embodiment of the present disclosure.
  • FIG. 9 is a schematic structural diagram of a hand-raising recognition device according to another embodiment of the present disclosure.
  • Fig. 10 is a schematic block diagram of an electronic device according to an embodiment of the present disclosure.
  • Image processing is a technology that uses a computer to analyze images to achieve the desired results. Also known as image processing. Image processing generally refers to digital image processing. A digital image refers to a large two-dimensional array obtained by shooting with industrial cameras, video cameras, scanners and other equipment. The elements of this array are called pixels, and their values are called grayscale values. Image processing technology generally includes three parts: image compression, enhancement and restoration, matching, description and recognition.
  • Deep Learning is a new research direction in the field of Machine Learning (ML for short). It is introduced into machine learning to make it closer to the original goal-artificial intelligence. Deep learning is to learn the internal law and representation level of sample data. The information obtained in the learning process is of great help to the interpretation of data such as text, images and sounds. Its ultimate goal is to enable machines to have the ability to analyze and learn like humans, and to be able to recognize data such as text, images, and sounds. Deep learning is a complex machine learning algorithm that has achieved results in speech and image recognition that far exceed previous related technologies.
  • Computer Vision is a science that studies how to make machines "see”. To put it further, it refers to the use of cameras and computers instead of human eyes to identify, track and measure targets, and further make graphics. Processing, so that the computer processing becomes an image that is more suitable for human observation or sent to the instrument for detection.
  • computer vision studies related theories and technologies, trying to build artificial intelligence systems that can obtain 'information' from images or multidimensional data.
  • the information referred to here refers to information that can be used to help make a "decision” as defined by Shannon. Because perception can be thought of as extracting information from sensory signals, computer vision can also be thought of as the science of how to make artificial systems "perceive" from images or multidimensional data.
  • Artificial Intelligence is a subject that studies certain thinking processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.) technology.
  • Artificial intelligence hardware technology generally includes computer vision technology, speech recognition technology, natural language processing technology and its learning/deep learning, big data processing technology, knowledge map technology and other major aspects.
  • Gesture recognition Gesture recognition in computer science is a topic that uses mathematical algorithms to recognize human gestures. Gesture recognition can come from the movement of various parts of the human body, but generally refers to the movement of the face and hands. Users can use simple gestures to control or interact with the device, allowing the computer to understand human behavior. Its core technologies are gesture segmentation, gesture analysis and gesture recognition. Gesture recognition can be seen as a way for computers to understand human body language, creating a richer bridge between machines and humans than raw text user interfaces or even graphical user interfaces.
  • Smart education that is, education informatization, refers to the process of comprehensively and deeply using modern information technology in the field of education (education management, education teaching and education research) to promote education reform and development. Its technical features are digitization, networking, intelligence, and multimedia, and its basic features are openness, sharing, interaction, collaboration, and ubiquity. Promote the modernization of education with educational informatization, and change the traditional model with information technology.
  • Fig. 1 is a schematic flowchart of a method for raising hands recognition according to an embodiment of the present disclosure. As shown in Fig. 1, the method includes:
  • each bone joint can be regarded as a key point, connecting the bone joint and the bone joint Human structures such as bones and skeletal muscles can be regarded as the connection between key points, so that the recognition of human actions can be realized based on the relative position changes between the extracted key points.
  • hand-raising recognition can be used in gesture recognition and smart education scenarios, such as class scenarios, voting resolution scenarios, and so on.
  • the image acquisition device can be deployed in the scene, relying on the image acquisition device to obtain the image of the human body to be recognized, and then realize the acquisition of the key points of the upper limbs of the human body in the image.
  • CNN Convolutional Neural Networks
  • the extracted key points can be connected based on the basic structure of the human body, and then the image features of the upper body of the collected human body can be generated, which is the first image feature, where , the first image feature may be a topology image feature generated based on the key points of the upper body of the human body.
  • the extracted key points of the upper body of the human body can include the head, nose, neck, left eye, left ear, right eye, right ear, left shoulder, left elbow, left wrist, right shoulder, right elbow, right wrist, chest , left hip, right hip, etc.
  • FIG. 2 is a schematic diagram of the distribution of key points of the human body. It is set to extract the key points of the human body in FIG. 2 for the human body in the collected image, wherein key point 1 is the left shoulder joint of the upper body of the human body, Key point 2 is the elbow joint on the left arm of the human body, and key point 3 is the left wrist joint. Connecting key points 1, 2 and 3 based on the bone structure of the human body is key point 1 and key point 3. Point 2 is connected, key point 2 is connected to key point 3, and there is no direct connection between key point 1 and key point 3. Further, key point 1, key point 2, and key point 3 may be generated to generate image features of the left arm of the human body in the captured image.
  • a human action recognition model can be set up, which can realize the recognition of the human body's hand-raising action based on the first image features, such as the Spatial Temporal Graph Convolutional Networks for Skeleton Based Action Reco (gcn), which is based on the human body
  • the skeleton diagram sequence is formulated, and each key point corresponds to a joint of the human body.
  • an initial structure can be carried out on the human action recognition model, and then the training of the human action recognition model is carried out according to the topological structure characteristics of the input human body key points, and the topological structure characteristics of the training sample human body key points are input to the human action recognition model Among them, the human body posture information carried in the topological structure features of key points of the human body is obtained through its feature extraction layer, such as the arm lift range, etc., and then the recognition of the actions performed by the human body in the topological structure features of key points of the human body is realized. Through multiple rounds of iterations, a mature and accurate human action recognition model is obtained.
  • the acquired first image feature is input into a mature human action recognition model, and the output result of the model is used to judge whether the human body in the captured image to which the first image feature belongs has performed a hand-raising action.
  • the output result of the human action recognition model is a score
  • the output result obtained after inputting the first image feature is the first score in the embodiment of the present disclosure.
  • the key points of the upper body extracted based on the collected images of the human body include all joints of the upper body of the human body and other parts and/or tissue structures that can reflect the movement of the human body.
  • the key points of the human hand position can be determined according to the relative positions between the extracted key points.
  • the key points of the human hand may include the key points corresponding to the elbow joint, the key point corresponding to the wrist joint, the key point corresponding to the finger joint, etc., and the key points corresponding to the elbow joint may be used as In the embodiment of the present disclosure, the boundary between the key points of the human hand and the key points corresponding to other tissue structure parts.
  • the key points of the human hand In general, among the key points of the human hand, there is a direct connection between the key points corresponding to the elbow joint and the key points corresponding to the shoulder joint. Therefore, the key points of the human hand can be regarded as a whole, through which The relative position between the overall key point and the key points corresponding to other tissue structures of the upper body of the human body is used to determine the key points of the hand of the human body from all the key points of the upper body.
  • the movement of raising the arm can be raising the hand or other movements.
  • the human body can lift the arm, The palm will also be open. Based on this setting, in the upper limb movement of raising the human arm, further screening needs to be carried out to avoid noise interference when the human body performs the hand raising movement and the palm is not open, thereby improving the accuracy of human hand raising recognition.
  • the key points of the hand are connected with the basic structure based on the human body between the key points.
  • the key point corresponding to the elbow joint needs to be connected with the key point corresponding to the wrist joint along the direction from the elbow to the wrist
  • the key point corresponding to the wrist joint needs to be connected with the key point corresponding to the nearest knuckle joint along the direction from wrist to finger .
  • an image feature of the hand of the human body is generated, which is the second image feature.
  • the second image feature is input into the preset recognition model of the state of the human hand, the output result of the model is obtained, and based on the result, it is judged whether the hand of the human body is in an open state.
  • the output result of the hand state recognition model of the human body is a score
  • what is obtained after inputting the second image feature is the second score of whether the palm of the human body in the collected image to which the second image feature belongs is in an open state.
  • the first score has a numerical interval, and the magnitude of the numerical value in this interval may represent the probability of the human body performing the hand-raising action.
  • the first score A For example, if the interval to which the first score A belongs is set to [0, 1], then the first score A ⁇ [0, 1].
  • A 0.23, that is, the probability of the human body performing the hand-raising action is 0.23%, it can be understood that the human body corresponding to the first score of the above value has performed other actions that are not the hand-raising action.
  • the numerical value in the range can represent the probability of opening the human hand.
  • the second score B For example, if the interval to which the second score B belongs is set to be [0, 1], then the second score B ⁇ [0, 1].
  • the human body based on the first score, it is judged whether the upper limbs of the human body perform the hand-raising action, and based on the second score, it is judged whether the hand of the human body is in an open state.
  • the human body is performing the hand-raising action while the hand is in an open state, then It can be determined that the human body has performed a hand-raising motion.
  • the hand-raising recognition method proposed in this disclosure extracts the key points of the upper body of the human body and generates the first image feature, inputs the first image feature into the human action recognition model to detect whether the hand-raising action is performed, and obtains the corresponding first score.
  • the hand-raising recognition method proposes the hand-raising recognition method.
  • the state detection of whether the human palm is opened is realized. Further, the fusion of the human body posture and human hand The double recognition of the state makes it possible to realize the recognition of the human body's hand-raising action in the relatively blurred and small distant view image, which effectively improves the accuracy of the human body's hand-raising recognition.
  • FIG. 3 is a schematic flowchart of a method for raising hands recognition in another embodiment of the present disclosure. As shown in FIG. 3 , the method includes:
  • a human body detection model which can generate human body detection frames based on the collected human body images to be recognized.
  • the human body detection model can be a lightweight detection model, wherein the backbone network (backbone) of the model can be a convolutional neural network model (ShuffleNet V2), and the detection framework can be an object detection algorithm (You Only Look Once V3, YOLO V3). Relying on the human body image to be recognized obtained by the image acquisition device, after inputting it into the set human body detection model, a human body detection frame (Body_boxex) can be generated.
  • backbone backbone network
  • the detection framework can be an object detection algorithm (You Only Look Once V3, YOLO V3).
  • key points are extracted for different parts of the human body based on the different parts of the human body tissue structure corresponding to the human body detection frame.
  • the human body detection frame is generated based on the human body image in the image to be recognized, and contains sample images of key points to be collected corresponding to different parts of the upper body of the human body. After inputting the human body detection frame into the key point detection model of the human body, different The key points corresponding to the parts.
  • the key points corresponding to each part are combined to obtain all the key points of the upper body of the human body that need to be extracted.
  • the confidence interval can be understood as the possible value range of the preset probability sample. Further, the accuracy of the preset value range of the probability sample , which can be understood as confidence.
  • the position information of the key points of the upper body of the human body can be used confidence is achieved.
  • the extracted key points of the upper body correspond to different organizational structures of the upper body of the human body.
  • the correct position information of the key points corresponding to each organizational structure can be obtained. Based on the position information, it is possible to obtain The confidence interval corresponding to the position information of each key point.
  • the confidence degree of each key point of the upper body of the human body is determined based on the actually extracted position information of the key points of the upper body of the human body and the obtained confidence interval corresponding to each key point.
  • a corresponding confidence threshold can be set for each key point. When each key point of the extracted upper body is greater than its corresponding confidence threshold, it can be determined that the currently extracted key points of the upper body of the human body are accurate and available. the key points.
  • an input matrix is constructed based on the collected position information of the key points of the upper body of the human body and the corresponding confidence levels.
  • the format of the construction matrix is set according to the preset input format of the human action recognition model.
  • a graph convolutional neural network model (Graph Convolutional Network, GCN) can be used to realize human action recognition.
  • GCN Graph Convolutional Network
  • a 16 ⁇ 3 input matrix can be constructed, where 16 represents the number of key point data contained in each input matrix is 16, and 3 represents each key point data in the input matrix The two numerical values of the corresponding coordinate information (x, y) and the confidence corresponding to each key point.
  • connection relationship between the key points can be obtained.
  • the first image feature is generated based on the extracted position information of the key points and the connection relationship between the key points. Therefore, it is necessary to accurately extract and express the connection relationship between the key points and the key points.
  • connection relationship between the key points of the upper body and the key points can be represented by an adjacency matrix.
  • the logical structure of the adjacency matrix can be divided into two parts, one of which is the vertices, which is the data corresponding to the extracted key points of the upper body in the disclosed embodiment, and can be stored in a one-dimensional array.
  • the other part is the edge, which in the embodiment of the present disclosure can reflect the key points of the upper body and the connection relationship data between the key points, which can be stored in a two-dimensional array.
  • the two-dimensional array storing the connection relationship between the key points of the upper body and the key points is the adjacency matrix to be constructed in the embodiments of the present disclosure.
  • the value of the element in the adjacency matrix corresponding to the connection is 1; when there is no direct connection between the two key points, the connection The relationship corresponds to the value of the element in the adjacency matrix being 0. Based on the above settings, the adjacency matrix corresponding to all the key points of the upper body and the connection relationship between the key points can be obtained.
  • a degree matrix can be constructed, and the degree matrix can be used to reflect the number of edges that are connected to the extracted key points.
  • the degree matrix is a diagonal matrix, and the vertices included in it are the data of the key points of the upper body in the embodiment of the present disclosure, and each element on the diagonal represents the degree of each vertex, which is the data of the key points of the embodiment of the present disclosure.
  • the number of edges that are connected to each key point of the upper body in . Based on the above settings, a degree matrix corresponding to the number of edges connecting all the key points of the upper body can be obtained.
  • the input matrix can determine the position information of each key point of the upper body extracted
  • the adjacency matrix can determine the connection relationship between each key point of the upper body
  • the degree matrix can determine the edge that has a connection relationship with each key point of the upper body. quantity.
  • the first image feature can be obtained based on the following formula:
  • X is the input matrix
  • A is the adjacency matrix
  • D is the degree matrix.
  • the adjacency matrix and the degree matrix represent the connection relationship between the key points of the upper body, which corresponds to the bone connection relationship between the joints of the human body.
  • the hand-raising recognition method proposed in this disclosure by constructing an input matrix, an adjacency matrix, and a degree matrix, accurately represents the location of the extracted key points of the upper body and the connection relationship between them. Further, based on the input matrix, adjacency matrix Generating the first image feature with the degree matrix improves the accuracy of the first image feature, thereby ensuring the accuracy of the first score obtained based on the first image feature, so that the human body's hand-raising action can be accurately recognized.
  • FIG. 4 is a schematic flowchart of a method for raising hands recognition in another embodiment of the present disclosure. As shown in FIG. 4, the method includes:
  • the key points of the hand of the human body may include key points corresponding to the elbow, key points corresponding to the wrist, and key points corresponding to the fingers of the human body.
  • the key points of the human hand can be obtained from all the key points of the upper body based on the key points and the position information of the key points.
  • the position information of each key point of the hand is determined, and further, based on the relative position information between the key points and the key points, the key points of the elbow and the key points of the wrist are determined .
  • the correct position information of the elbow key point and the wrist key point can be obtained, and the confidence interval of the corresponding position information of the elbow key point and the wrist key point can be obtained based on the position information. Further, based on the actually obtained location information of the elbow key point and the wrist key point, the confidence of the elbow key point and the wrist key point is obtained.
  • a corresponding confidence threshold can be set. When the confidence of the elbow key point and the wrist key point is greater than the set threshold, it can be understood that the positions of the elbow key point and the wrist key point are reasonable and correct. .
  • S403. Determine a hand detection frame based on the elbow key point and the wrist key point, and extract a second image feature from the hand detection frame.
  • the position of the human palm can be further determined, and then the hand detection frame of the human body can be generated.
  • the hand detection frame can be input into the human hand state detection model to identify whether the palm of the human body is in an open state.
  • connection relationship between the elbow key point and the wrist key point may correspond to the bone connection relationship between the human elbow joint and the wrist joint.
  • connection relationship between the key points of the elbow and the key points of the wrist of the human body is the key point corresponding to the key point of the wrist and the end point of the longest finger.
  • connection relationship There is a proportional relationship between the connection relationship, and the two are the same.
  • connection relationship between the elbow key point and the wrist key point After obtaining the connection relationship between the elbow key point and the wrist key point, based on the above proportional relationship, the connection relationship between the wrist key point of the human body and the key point corresponding to the longest finger end point can be obtained. Based on the above two connections The relationship can generate the hand detection box of the human body.
  • a second image feature of the human hand is generated.
  • the second image feature can be used to identify whether the human hand is in an open state, and the extent of the open state.
  • the hand raising recognition method proposed in the present disclosure determines the hand detection frame of the human body through the key points of the elbow and the wrist, and then obtains the second image feature of the human hand from the hand detection frame. Further, based on the second image feature, the recognition of whether the human hand is in an open state can be realized, thereby effectively improving the accuracy of hand raising recognition.
  • FIG. 5 is a schematic flowchart of a hand-raising recognition method according to another embodiment of the present disclosure. As shown in FIG. 5 , the method includes:
  • connection relationship between the key points of the elbow and the key points of the wrist After determining the key points of the elbow and the key points of the wrist, connect them to obtain the connection relationship between the key points of the elbow and the key points of the wrist, where the connection relationship corresponds to the relationship between the elbow joint and the wrist joint of the human body
  • the bone connection relationship of is a straight line.
  • the elbow joint and wrist joint of the human body are connected to the forearm. From the elbow to the wrist, based on the extension line of the forearm, extending a certain distance from the wrist is the palm of the human body. Generally, there is a proportional relationship between the length of the forearm and the length of the palm of the human body.
  • the set distance is the distance between the key point of the wrist of the human body and the key point corresponding to the end point of the longest finger, which is the same as the length of the palm of the human body.
  • the value of the set distance can be determined by the length of the straight line connection relationship between the elbow key point and the wrist key point.
  • the distance between the elbow key point and the wrist key point is obtained, and the set distance is determined based on the above-mentioned proportional relationship between the forearm length and the palm length.
  • the finger end point (x, y) of the longest finger of the human hand can be obtained:
  • S503. Generate a hand detection frame based on the key point of the wrist and the end point of the finger.
  • the hand detection frame may be a detection frame with a regular shape, such as a rectangle. After obtaining the key points corresponding to the elbow key point, wrist key point and the longest finger end point, the position information of all hand key points can be determined, and the hand detection frame can be generated based on the position information of all hand key points.
  • the hand detection frame is set to be a rectangle
  • the distance between the key point of the elbow and the key point corresponding to the end point of the longest finger is taken as the length of the rectangle
  • the key point corresponding to the end point of the thumb with the palm open is The distance between the key points corresponding to the end point of the little finger is used as the width of the rectangle to generate a hand detection frame.
  • the hand-raising recognition method proposed in this disclosure obtains the end point of the finger through the extension line between the key point of the elbow and the key point of the wrist, and further generates a hand detection frame based on the key point of the wrist and the end point of the finger, so that it can be based on the points in the hand detection frame
  • the key points of the hand generate the second image features of the human hand, and then realize the detection and recognition of whether the human hand is in an open state, effectively improving the accuracy of hand-raising recognition.
  • the hand-raising recognition method proposed in the embodiment of the present disclosure combines the recognition of the human body's hand-raising action and the recognition of the state of the human palm's opening.
  • the first image feature extracted based on the key points of the upper body of the human body that can be used for the detection of raising the hand is used to obtain the first score
  • the second image feature extracted based on the key points of the human hand that can be used for the detection of the palm opening state is obtained.
  • the second score further, based on the first score and the second score, accurate hand recognition can be performed.
  • the first score and the second score can be analyzed respectively to obtain the detection result, as shown in FIG. 6 , which is a schematic flowchart of a method for raising hands recognition according to another embodiment of the present disclosure.
  • the method includes:
  • the human body can be determined to have performed a hand-raising action. If the first score is less than or equal to the corresponding first preset score threshold, it can be understood that the probability of the human body performing the hand-raising action in the current state is low, and other non-hand-raising actions are performed.
  • the first score is set to correspond to the angle between the raised arm of the human body and the desktop in the captured image to which the first image feature belongs, and the interval of the first score is [0, 1].
  • this score corresponds to the state where the angle between the arm of the human body and the table is 45° after the arm is raised, which can be understood as, when the arm of the human body is raised and the When the angle between the desktops is greater than 45°, the first score is greater than 0.5, and it can be confirmed that the human body has performed the hand-raising action.
  • the first score is greater than its corresponding first score threshold of 0.5, that is, the angle between the arm of the human body and the table is greater than 45° after the arm is raised in the current state, according to It can be seen from the corresponding relationship that when the first score is 0.8, the angle between the raised arm of the human body and the table is 72°. It can be determined that in the current state, in the collected image to which the first image feature belongs, the human body has performed a hand-raising action.
  • the palm of the human body is determined to be in an open state in the current state. If the second score is less than or equal to the corresponding second score threshold, it can be understood that the palm of the human body is in other states of the non-open state in the current state.
  • set the interval of the second score to [0, 1], and the first preset score threshold corresponding to the second score is 0.5. It can be understood that when the second score is greater than 0.5, the palm of the human body is determined to be in an open state .
  • the first score is greater than its corresponding first preset score threshold and the second score is greater than its corresponding first preset score threshold, it can be judged that the current human arm has performed a hand-raising action and the palm is in the open position.
  • the open state it can be determined that the human body in this state has performed a hand-raising action.
  • the hand-raising recognition method proposed in this disclosure through the analysis of the first score, determines whether the human body has performed the hand-raising action, and through the analysis of the second score, determines whether the human hand is in an open state, and then determines whether the hand-raising action is performed.
  • the action with the hand in the open state is identified as a hand-raising action, which combines the dual recognition of the human body posture and the state of the human hand, so that the recognition of the human body's hand-raising action can be realized in a relatively blurred and small distant view image. Effectively improve the accuracy of human hand recognition.
  • FIG. 7 is a schematic flowchart of a method for raising hands recognition according to another embodiment of the present disclosure.
  • the method includes:
  • the first score is fused with the second score to generate a third score. And set a second preset score threshold corresponding to the third score.
  • the third score is greater than the second preset score threshold, it can be determined that the human body has performed a hand-raising action in the current state while the palm is in an open state, and then it can be determined In this state, the human body performs a hand-raising action.
  • weights may be configured for the first score and the second score, and the first score and the second score are fused based on the weight value to generate the third score.
  • the third score may reflect the probability that the palm is in an open state when the human body performs the hand-raising action.
  • the interval of the third score is set as [0, 1].
  • the third score is 0.77
  • the corresponding probability of the human body performing the hand-raising action while the palm is in an open state is 0.77.
  • the second preset score threshold is set to 0.6, then when the third score is greater than 0.6, it can be determined that the human body has performed a hand-raising action while the palm is in an open state; when the third score is less than or equal to 0.6, it can be determined that The human body does not perform the hand raising motion.
  • the third score is greater than the second preset score threshold of 0.6, and it can be determined that the human body has performed a hand-raising action in the current state.
  • the hand-raising recognition method proposed in this disclosure obtains the third score based on the first score and the second score. Through the analysis of the third score, it is determined that the hand-raising action is performed while the palm is in an open state, so as to realize the recognition of the raised hand.
  • the recognition of hand movements combines the detection of the human body's hand-raising movement and the detection of the opening state of the human palm, which effectively improves the accuracy of hand-raising recognition.
  • an embodiment of the present disclosure also provides a hand-raising recognition device, since the hand-raising recognition device provided by the embodiment of the present disclosure is the same as that provided by the above-mentioned several embodiments
  • the hand recognition method is corresponding, so the implementation of the hand-raised recognition method is also applicable to the hand-raised recognition device provided by the embodiment of the present disclosure, which will not be described in detail in the following embodiments.
  • Fig. 8 is a schematic structural diagram of a hand-raising recognition device according to an embodiment of the present disclosure.
  • the hand-raising recognition device 800 includes an extraction module 81 and a recognition module 82, wherein:
  • the extraction module 81 is used to extract the key points of the upper body of the human body from the collected human body images, and extract the first feature image based on the key points of the upper body, input it into the human body action recognition model, and output the first score recognized as the human body performing the hand-raising action ;
  • the extraction module 81 is also used to determine the key points of the hands from the key points of the upper body;
  • the extraction module 81 is also used to extract the second image feature of the hand based on the key points of the hand, input it into the hand classification model, and output the second score that is recognized as the palm being in an open state;
  • the identification module 82 is configured to identify the hand-raising state according to the first score and the second score.
  • Fig. 9 is a schematic structural diagram of a hand-raising recognition device according to another embodiment of the present disclosure.
  • the hand-raising recognition device 900 includes an extraction module 91 and a recognition module 92, wherein:
  • extraction module 81 and the identification module 82 have the same structure and function as the extraction module 91 and the identification module 92 .
  • the extraction module 91 is also used to: determine the key points of the elbow and the key points of the wrist from the key points of the hand based on the position information of the key points; determine the hand detection based on the key points of the elbow and the wrist frame, and extract the second image features from the hand detection frame.
  • the extraction module 91 also includes:
  • a connection unit 911 configured to generate a connection line between the key point of the elbow and the key point of the wrist;
  • the extension unit 912 is used to extend the connecting line for a set distance along the direction from the elbow key point to the wrist key point, so as to determine the end point of the finger;
  • the generating unit 913 is configured to generate a hand detection frame based on the key points of the wrist and the end points of the fingers.
  • the extraction module 91 is further configured to: obtain the distance between the elbow key point and the wrist key point according to the positions of the elbow key point and the wrist key point, and determine the set distance according to the distance.
  • the extraction module 91 is further configured to: obtain the confidence of the elbow key point and the wrist key point, and determine that the confidence of the elbow key point and the wrist key point is greater than a set threshold.
  • the extraction module 91 is also used to: input the human body image into the human body detection model for human body detection, and output the human body detection frame; input the human body detection frame into the human body key point detection model for feature extraction, based on the extracted features Perform key point detection and output key points of the upper body.
  • the extraction module 91 also includes:
  • a matrix construction unit 914 configured to construct an input matrix based on the position information and confidence of key points of the upper body
  • the matrix construction unit 914 is also used to construct an adjacency matrix and a degree matrix based on the connection relationship between the key points of the upper body;
  • a feature extraction unit 915 configured to acquire the first image feature based on the input matrix, degree matrix and adjacency matrix.
  • the recognition module 92 is further configured to: determine that the human body is in the state of raising hands in response to the first score and the second score being greater than the respective first preset score thresholds; or, for the first score and the second score The two scores are fused to generate a third score, and in response to the third score being greater than a second preset score threshold, it is determined that the human body is in a state of raising hands.
  • the hand-raising recognition device proposed in this disclosure extracts the key points of the upper body of the human body in the captured image, generates the first image feature, and inputs the first image feature into the human body action recognition model to detect the hand-raising action and obtain the first score.
  • the second image feature is extracted through the key points of the human hand, and the second image feature is input into the human hand action recognition model to detect the open state of the palm to obtain the second score. Further, based on the first score and the second score, the identification of whether the human body performs a hand-raising action is realized.
  • the dual recognition of human body posture and human hand state is integrated, so that the human body's hand-raising action recognition can be realized in a relatively blurred and small distant view image, thereby effectively improving the accuracy of human body's hand-raising recognition.
  • FIG. 10 shows a schematic block diagram of an example electronic device 1000 that may be used to implement embodiments of the present disclosure.
  • Electronic device is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers.
  • Electronic devices may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smart phones, wearable devices, and other similar computing devices.
  • the components shown herein, their connections and relationships, and their functions, are by way of example only, and are not intended to limit implementations of the disclosure described and/or claimed herein.
  • the device 1000 includes a computing unit 1001 that can execute according to a computer program stored in a read-only memory (ROM) 1002 or loaded from a storage unit 1008 into a random-access memory (RAM) 1004. Various appropriate actions and treatments. In the RAM 1004, various programs and data necessary for the operation of the device 1000 can also be stored.
  • the computing unit 1001, ROM 1002, and RAM 1004 are connected to each other through a bus 1004.
  • An input/output (I/O) interface 1005 is also connected to the bus 1004 .
  • the I/O interface 1005 includes: an input unit 1006, such as a keyboard, a mouse, etc.; an output unit 1007, such as various types of displays, speakers, etc.; a storage unit 1008, such as a magnetic disk, an optical disk, etc. ; and a communication unit 1009, such as a network card, a modem, a wireless communication transceiver, and the like.
  • the communication unit 1009 allows the device 1000 to exchange information/data with other devices through a computer network such as the Internet and/or various telecommunication networks.
  • the computing unit 1001 may be various general-purpose and/or special-purpose processing components having processing and computing capabilities. Some examples of computing units 1001 include, but are not limited to, central processing units (CPUs), graphics processing units (GPUs), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processing processor (DSP), and any suitable processor, controller, microcontroller, etc.
  • the calculation unit 1001 executes the various methods and processes described above, such as the method of raising a hand recognition.
  • the raised hand recognition method may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as storage unit 1008 .
  • part or all of the computer program may be loaded and/or installed on the device 1000 via the ROM 1002 and/or the communication unit 1009.
  • the computer program When the computer program is loaded into the RAM 1004 and executed by the computing unit 1001, one or more steps of the method for raising hands recognition described above can be performed.
  • the computing unit 1001 may be configured in any other appropriate way (for example, by means of firmware) to execute the method for recognizing raised hands.
  • Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips Implemented in a system of systems (SOC), load programmable logic device (CPLD), computer hardware, firmware, software, and/or combinations thereof.
  • FPGAs field programmable gate arrays
  • ASICs application specific integrated circuits
  • ASSPs application specific standard products
  • SOC system of systems
  • CPLD load programmable logic device
  • computer hardware firmware, software, and/or combinations thereof.
  • programmable processor can be special-purpose or general-purpose programmable processor, can receive data and instruction from storage system, at least one input device, and at least one output device, and transmit data and instruction to this storage system, this at least one input device, and this at least one output device an output device.
  • Program codes for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special purpose computer, or other programmable data processing devices, so that the program codes, when executed by the processor or controller, make the functions/functions specified in the flow diagrams and/or block diagrams Action is implemented.
  • the program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.
  • a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device.
  • a machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium.
  • a machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing.
  • machine-readable storage media would include one or more wire-based electrical connections, portable computer discs, hard drives, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, compact disk read only memory (CD-ROM), optical storage, magnetic storage, or any suitable combination of the foregoing.
  • RAM random access memory
  • ROM read only memory
  • EPROM or flash memory erasable programmable read only memory
  • CD-ROM compact disk read only memory
  • magnetic storage or any suitable combination of the foregoing.
  • the systems and techniques described herein can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user. ); and a keyboard and pointing device (eg, a mouse or a trackball) through which a user can provide input to the computer.
  • a display device e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor
  • a keyboard and pointing device eg, a mouse or a trackball
  • Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and can be in any form (including Acoustic input, speech input or, tactile input) to receive input from the user.
  • the systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., as a a user computer having a graphical user interface or web browser through which a user can interact with embodiments of the systems and techniques described herein), or including such backend components, middleware components, Or any combination of front-end components in a computing system.
  • the components of the system can be interconnected by any form or medium of digital data communication, eg, a communication network. Examples of communication networks include: local area networks (LANs), wide area networks (WANs), the Internet, and blockchain networks.
  • a computer system may include clients and servers.
  • Clients and servers are generally remote from each other and typically interact through a communication network.
  • the relationship of client and server arises by computer programs running on the respective computers and having a client-server relationship to each other.
  • the server can be a cloud server, also known as a cloud computing server or a cloud host. ), there are defects such as high management difficulty and weak business scalability.
  • the server can also be a server of a distributed system, or a server combined with a blockchain
  • steps may be reordered, added or deleted using the various forms of flow shown above.
  • each step described in the present disclosure may be executed in parallel, sequentially, or in a different order, as long as the desired result of the technical solution disclosed in the present disclosure can be achieved, no limitation is imposed herein.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Multimedia (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • General Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • Biomedical Technology (AREA)
  • Software Systems (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Biophysics (AREA)
  • Social Psychology (AREA)
  • Psychiatry (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Image Analysis (AREA)

Abstract

一种举手识别方法、装置、电子设备及存储介质，方法包括：从采集的人体图像中提取人体的上半身关键点，并基于上半身关键点提取第一图像特征，输入人体动作识别模型中，输出被识别为人体执行抬手动作的第一得分；从上半身关键点中确定手部关键点；基于手部关键点，提取人手的第二图像特征，输入人手分类模型中，输出被识别为手掌处于张开状态的第二得分；根据第一得分和第二得分进行举手状态的识别。

Description

举手识别方法、装置、电子设备及存储介质
相关申请的交叉引用
本公开基于申请号为202110519356.6、申请日为2021年05月12日的中国专利申请提出，并要求该中国专利申请的优先权，该中国专利申请的全部内容在此引入本公开作为参考。
技术领域
本公开涉及图像处理领域，尤其涉及计算机视觉和深度学习等人工智能领域。
背景技术
随着技术的发展，很多场景存在举手识别的需求，比如课堂上对于学生的举手识别。相关技术中，通常是将采集到的待识别图像直接输入训练检测模型，针对待识别图像直接进行图像识别。
一般情况下，由于图像采集设备的布放位置，所获取到的待识别图像为远景图像，其中，人手部位通常较小，且不够清晰，人体举手动作特征相对模糊，使得举手识别的准确率较低。因此，如何实现在清晰度不够的远景图像中进行高准确率的举手识别成为了需要解决的问题。
发明内容
本公开提出了一种用于举手识别方法、装置、电子设备以及存储介质、计算机程序产品。
根据本公开的一方面，提出了一种举手识别方法，包括：从采集的人体图像中提取人体的上半身关键点，并基于所述上半身关键点提取第一图像特征，输入人体动作识别模型中，输出被识别为人体执行抬手动作的第一得分；从所述上半身关键点中确定手部关键点；基于所述手部关键点，提取人手的第二图像特征，输入人手分类模型中，输出被识别为手掌处于张开状态的第二得分；根据所述第一得分和所述第二得分进行举手状态的识别。
根据本公开的第二方面，提出了一种举手识别装置，包括：提取模块，用于从采集的人体图像中提取人体的上半身关键点，并基于所述上半身关键点提取第一图像特征，输入人体动作识别模型中，输出被识别为人体执行抬手动作的第一得分；所述提取模块，还用于从所述上半身关键点中确定手部关键点；所述提取模块，还用于基于所述手部关键点，提取人手的第二图像特征，输入人手分类模型中，输出被识别为手掌处于张开状态的第二得分；识别模块，用于根据所述第一得分和所述第二得分进行举手状态的识别。
根据本公开的第三方面，提出了一种电子设备，包括：至少一个处理器；以及与所述至少一个处理器通信连接的存储器；其中，所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器能够执行上述第一方面提出的举手识别方法。
根据本公开的第四方面，提出了一种存储有计算机指令的非瞬时计算机可读存储介质，其中，所述计算机指令用于使所述计算机执行上述第一方面提出的举手识别方法。
根据本公开的第五方面，提出了一种计算机程序产品，包括计算机程序，所述计算机程序在被处理器执行时实现上述第一方面提出的举手识别方法。
应当理解，本部分所描述的内容并非旨在标识本公开的实施例的关键或重要特征，也不用于限制本公开的范围。本公开的其它特征将通过以下的说明书而变得容易理解。
附图说明
附图用于更好地理解本方案，不构成对本公开的限定。其中：
图1是本公开一实施例的举手识别方法的流程示意图；
图2是本公开另一实施例的举手识别方法的流程示意图；
图3是本公开另一实施例的举手识别方法的流程示意图；
图4是本公开另一实施例的举手识别方法的流程示意图；
图5是本公开另一实施例的举手识别方法的流程示意图；
图6是本公开另一实施例的举手识别方法的流程示意图；
图7是本公开另一实施例的举手识别方法的流程示意图；
图8是本公开一实施例的举手识别装置的结构示意图；
图9是本公开另一实施例的举手识别装置的结构示意图；
图10是本公开一实施例的电子设备的示意性框图。
具体实施方式
以下结合附图对本公开的示范性实施例做出说明，其中包括本公开实施例的各种细节以助于理解，应当将它们认为仅仅是示范性的。因此，本领域普通技术人员应当认识到，可以对这里描述的实施例做出各种改变和修改，而不会背离本公开的范围和精神。同样，为了清楚和简明，以下的描述中省略了对公知功能和结构的描述。
图像处理(Image Processing)，用计算机对图像进行分析，以达到所需结果的技术。又称影像处理。图像处理一般指数字图像处理。数字图像是指用工业相机、摄像机、扫描仪等设备经过拍摄得到的一个大的二维数组，该数组的元素称为像素，其值称为灰度值。图像处理技术一般包括图像压缩，增强和复原，匹配、描述和识别3个部分。
深度学习(Deep Learning，简称DL)，是机器学习(Machine Learning，简称ML)领域中一个新的研究方向，它被引入机器学习使其更接近于最初的目标——人工智能。深度学习是学习样本数据的内在律和表示层次，这些学习过程中获得的信息对诸如文字，图像和声音等数据的解释有很大的帮助。它的最终目标是让机器能够像人一样具有分析学习能力，能够识别文字、图像和声音等数据。深度学习是一个复杂的机器学习算法，在语音和图像识别方面取得的效果，远远超过先前相关技术。
计算机视觉(Computer Vision)，是一门研究如何使机器“看”的科学，更进一步的说，就是指用摄影机和电脑代替人眼对目标进行识别、跟踪和测量等机器视觉，并进一步做图形处理，使电脑处理成为更适合人眼观察或传送给仪器检测的图像。作为一个科学学科，计算机视觉研究相关的理论和技术，试图建立能够从图像或者多维数据中获取‘信息’的人工智能系统。这里所指的信息指Shannon定义的，可以用来帮助做一个“决定”的信息。因为感知可以看作是从感官信号中提取信息，所以计算机视觉也可以看作是研究如何使人工系统从图像或多维数据中“感知”的科学。
人工智能(Artificial Intelligence，简称AI)，是研究使计算机来模拟人生的某些思维过程和智能行为(如学习、推理、思考、规划等)的学科，既有硬件层面的技术，也有软件层面的技术。人工智能硬件技术一般包括计算机视觉技术、语音识别技术、自然语言处理技术以及及其学习/深度学习、大数据处理技术、知识图谱技术等几大方面。
手势识别(Gesture recognition)手势识别在计算机科学中，是通过数学算法来识别人类手势的一个议题。手势识别可以来自人的身体各部位的运动，但一般是指脸部和手的运动。用户可以使用简单的手势来控制或与设备交互，让计算机理解人类的行为。其核心技术为手势分割、手势分析以及手势识别。手势识别可以被视为计算机理解人体语言的方式，从而在机器和人之间搭建比原始文本用户界面或甚至图形用户界面更丰富的桥梁。
智慧教育，即教育信息化，是指在教育领域(教育管理、教育教学和教育科研)全面深入地运用现代信息技术来促进教育改革与发展的过程。其技术特点是数字化、网络化、智能化和多媒体化，基本特征是开放、共享、交互、协作、泛在。以教育信息化促进教育现代化，用信息技术改变传统模式。
图1是本公开一实施例的举手识别方法的流程示意图，如图1所示，该方法包括：
S101，从采集的人体图像中提取人体的上半身关键点，并基于上半身关键点提取第一图像特征，输入人体动作识别模型中，输出被识别为人体执行抬手动作的第一得分。
人体的运动动作执行依赖于骨骼、骨骼肌、骨关节等多种人体结构，针对人体的上肢，其基 本运动形式可以为推、拉等基础动作。在每个动作的运动过程中，骨关节为与其相连接的骨骼的转动提供了支点，因此，在本公开实施例中，可以将每个骨关节视为一个关键点，连接骨关节与骨关节之间的骨骼、骨骼肌等人体结构可以视为关键点之间的连接，进而使得可以基于提取到的关键点之间相对位置的变化实现人体动作的识别。
一般情况下，举手识别可以用于手势识别和智慧教育场景下，比如上课场景、投票决议场景等等。可以在场景中布放图像采集装置，依托于图像采集装置获取待识别的人体图像，进而实现对图像中的人体的上肢关键点的采集。
可选地，可以通过卷积神经网络(Convolutional Neural Networks，CNN)等算法进行图像中人体的上肢关键点的采集。
本公开实施例中，采集到人体上半身的关键点后，可以将提取到的关键点基于人体的基本结构进行连接，进而生成被采集的人体的上半身的图像特征，即为第一图像特征，其中，第一图像特征可以是基于人体上半身的关键点生成的拓扑结构图像特征。
其中，提取到的人体上半身的关键点，可以包括人体的头部、鼻子、颈、左眼、左耳、右眼、右耳、左肩、左肘、左腕、右肩、右肘、右腕、胸、左髋、右髋等等。
进一步地，如图2所示，图2为人体关键点的分布示意图，设定针对采集图像中的人体提取到图2中的人体关键点，其中，关键点1为人体的上半身的左肩关节，关键点2为人体的左胳膊上的肘关节，关键点3为左手腕关节，将关键点1、关键点2以及关键点3之间基于人体的骨骼结构相连接，即为关键点1与关键点2之间相连接，关键点2与关键点3之间相连接，关键点1与关键点3之间没有直接的连接。进一步地，可以生成关键点1、关键点2以及关键点3生成采集图像中的人体左臂的图像特征。
可以设置一个人体动作识别模型，该模型可以基于第一图像特征实现人体抬手动作的识别，比如时空图卷积网络模型(Spatial Temporal Graph Convolutional Networks for Skeleton Based Action Reco，gcn)，该模型基于人体的骨骼图序列制定，每一个关键点存在对应的人体的一个关节。
实现中，可以对人体动作识别模型进行一个初始的架构，进而根据输入的人体关键点的拓扑结构特征进行人体动作识别模型的训练，将训练样本人体关键点的拓扑结构特征输入至人体动作识别模型中，通过其特征提取层获取人体关键点的拓扑结构特征中携带的人体姿态信息，比如手臂抬起幅度等等，进而实现对人体关键点的拓扑结构特征中的人体所执行的动作的识别。通过多轮次的迭代，进而获取到成熟且精准的人体动作识别模型。
进一步地，将获取到的第一图像特征输入至成熟的人体动作识别模型中，通过模型的输出结果判断第一图像特征所属的采集图像中的人体是否执行了抬手动作。其中，人体动作识别模型输出的结果为分数，输入第一图像特征后获取到的输出结果即为本公开实施例中的第一得分。
S102，从上半身关键点中确定手部关键点。
基于采集的人体图像中提取到的上半身的关键点，包括人体上半身的全部关节以及其它可以体现人体运动动作的部位和/或组织结构。可以根据提取到的关键点之间的相对位置，可以确定人体手部位置的关键点。
本公开实施例中，人体手部的关键点可以包括人体的肘关节对应的关键点、腕关节对应的关键点、指关节对应的关键点等等，并且，可以将肘关节对应的关键点作为本公开实施例中的人体手部关键点与其他组织结构部位对应的关键点之间的分界。
一般情况下，人体手部的关键点中，肘关节对应的关键点与肩关节对应的关键点之间存在直接连接的关系，因此，可以将人体手部的关键点视为一个整体，通过该整体关键点与人体上半身其他组织结构对应的关键点之间的相对位置，从全部的上半身关键点中确定人体的手部关键点。
S103，基于手部关键点，提取人手的第二图像特征，输入人手分类模型中，输出被识别为手掌处于张开状态的第二得分。
人体的上肢的动作中，胳膊抬起的动作可以是举手也可以是其他动作，一般情况下，在教育场景、投票场景等需要人体执行举手动作的场景，人体除了手臂抬起之外，手掌也会处于张开状态。基于该设定，在人体手臂抬起的上肢动作中，需要进行进一步的筛选，避免人体执行抬手动 作且手掌处于非张开状态的动作的噪声干扰，进而提高人体举手识别的准确率。
基于获取到的手部关键点，将手部关键点与关键点之间基于人体的基础结构相连接。比如，肘关节对应的关键点需要与腕关节对应的关键点沿手肘至手腕方向相连接，腕关节对应的关键点需要与距离其最近的指关节对应的关键点沿手腕至手指方向相连接。
进一步地，基于手部关键点的位置以及关键点之间的连接关系，生成人体的手部的图像特征，即为第二图像特征。将第二图像特征输入预设的人体手部状态的识别模型，获取到模型的输出结果，并基于结果判断人体的手部是否处于张开状态。
其中，人体的手部状态识别模型的输出结果为分数，输入第二图像特征后获取到的即为第二图像特征所属的采集图像中的人体手掌是否处于张开状态的第二得分。
S104，根据第一得分和第二得分进行举手状态的识别。
本公开实施例中，第一得分存在一个数值区间，该区间内的数值的大小可以代表人体执行抬手动作的概率。
比如，设定第一得分A所属的区间为[0，1]，则第一得分A∈[0，1]。一般情况下，设定当A＝0.23时，即人体执行抬手动作的概率为0.23％，可以理解为，上述取值的第一得分对应的人体，执行了属于非抬手动作的其他动作。再比如，设定当A＝0.98时，即人体执行抬手动作的概率为0.98，可以理解为，上述取值的第一得分对应的人体，执行抬手的动作。
同样的，第二得分存在一个数值区间，该区间内的数值大小可以代表人体手部张开的概率。
比如，设定第二得分B所属的区间为[0，1]，则第二得分B∈[0，1]。一般情况下，设定当B＝0.17时，即为人体的手部处于张开状态的概率为0.17，可以理解为，上述取值的第二得分对应的人体的手部处于非张开状态的其他状态。再比如，设定当B＝0.93时，即为人体的手部处于张开状态的概率为0.93，可以理解为，上述取值的第二得分对应的人体的手部处于张开状态。
进一步地，基于第一得分判断人体的上肢是否执行抬手动作，基于第二得分判断人体的手部是否处于张开状态，当确定人体的执行抬手动作的同时手部处于张开状态，则可以确定人体执行了举手动作。
本公开提出的举手识别方法，通过人体上半身的关键点的提取并生成第一图像特征，将第一图像特征输入人体动作识别模型进行抬手动作是否执行的检测，获取相应地第一得分。通过对上半身关键点进行筛选，获取人体手部的关键点并生成第二图像特征，将第二图像特征输入人体手部状态识别模型进行手部状态是否张开的检测，获取相应地第二得分。进一步地，基于第一得分与第二得分，实现对于人体举手动作的识别。本公开中，通过对于人体上半身关键点的提取，实现对于抬手动作的检测，通过对于手部关键点的采集，实现对于人体手掌是否张开的状态检测，进一步地，融合对人体姿态以及人手状态的双重识别，使得可以在相对模糊且人体较小的远景图像中实现人体的举手动作识别，有效提高了人体举手识别的准确率。
上述实施例中，第一图像特征的获取方法可结合图3进一步理解，图3为本公开另一实施例的举手识别方法的流程示意图，如图3所示，该方法包括：
S301，将人体图像输入人体检测模型中进行人体检测，输出人体检测框。
为了实现对于第一图像特征的提取，需要设置一个人体检测模型，该模型可以基于采集到的待识别的人体图像生成人体检测框。
可选地，人体检测模型可以是轻量检测模型，其中，模型的主干网络(backbone)可以为卷积神经网络模型(ShuffleNet V2)，检测框架可以为对象检测算法(You Only Look Once V3，YOLO V3)。依托于图像采集设备获取到的待识别的人体图像，将其输入至设定的人体检测模型后，可以生成人体检测框(Body_boxex)。
S302，将人体检测框输入人体关键点检测模型中进行特征提取，基于提取的特征进行关键点检测，输出上半身关键点。
为了实现人体的关键点的提取，需要将人体检测框输入人体关键点检测模型。本公开实施例中，基于人体检测框对应的人体组织结构部位的不同，分别对人体不同的部位进行关键点提取。
人体检测框基于待识别图像中的人体图像生成，包含有人体上半身的对应的不同部位的待采 集关键点的样本图像，将人体检测框输入至人体关键点检测模型后，可以获取到人体上半身不同部位对应的关键点。
进一步地，将每个部位对应的关键点组合，获取到所需提取的全部的人体上半身的关键点。
S303，基于上半身关键点的位置信息和置信度，构建输入矩阵。
一般情况下，统计学中针对某一个概率样本存在对应的置信区间，该置信区间可以理解为预设的概率样本可能的取值范围，进一步地，该概率样本预设的取值范围的准确程度，可以理解为置信度。
为了保证提取到的人体上半身关键点的所处位置的准确性，获取全部上半身的关键点后，需要对关键点的位置信息进行进一步的确认，可选地，可以通过人体上半身关键点的位置信息的置信度实现。
本公开实施例中，提取到的上半身关键点分别对应不同的人体上半身的组织结构，根据人体的基础结构，可以获取每个组织结构对应的关键点的正确位置信息，基于该位置信息，可以获取每个关键点的位置信息对应的置信区间。
进一步地，基于实际提取的人体上半身的关键点的位置信息与获取到的每个关键点对应的置信区间，确定每个人体上半身关键点的置信度。
可以为每个关键点设置一个对应的置信度阈值，当提取到的上半身的各个关键点均大于其对应的置信度阈值时，可以确定当前提取到的人体上半身的关键点位置准确，为有效可用的关键点。
进一步地，基于采集到的人体上半身的关键点的位置信息以及对应的置信度，构建输入矩阵。其中，构建矩阵的格式根据预设的人体动作识别模型的输入格式设置。
可选地，可以使用图卷积神经网络模型(Graph Convolutional Network，GCN)实现人体动作的识别。根据GCN分类模型输入矩阵的要求，可以构建一个16×3的输入矩阵，其中，16代表每个输入矩阵中所包含的关键点数据的数量为16个，3代表输入矩阵中每个关键点数据对应的坐标信息(x，y)的两个数值以及每个关键点对应的置信度。
S304，基于上半身关键点之间的连接关系，构建邻接矩阵和度矩阵。
确定上半身关键点的位置信息后，可以获取关键点与关键点之间的连接关系。一般情况下，上半身关键点存在一个或者多个可以与其相连接的边。
本公开实施例中，第一图像特征基于提取到的关键点的位置信息以及关键点与关键点之间的连接关系生成，因此，需要准确的提取表现关键点与关键点之间的连接关系。
进一步地，上半身的关键点与关键点之间的连接关系，可以用邻接矩阵体现。邻接矩阵的逻辑结构可以分为两个部分，其中一部分是顶点，在本公开实施例中即为提取到的上半身的关键点对应的数据，可以用一个一维数组存放。另一部分是边，在本公开实施例中即为可以体现上半身的关键点与关键点之间的连接关系数据，可以用一个二维数组存放。其中，存放上半身的关键点与关键点之间连接关系的二维数组，即为本公开实施例需要构建的邻接矩阵。
一般情况下，当两个关键点之间存在直接连接关系时，则该连接关系对应邻接矩阵中的元素的数值为1，当两个关键点之间不存在直接连接的关系时，则该连接关系对应邻接矩阵中的元素的数值为0。基于上述设定，可以获取全部的上半身关键点与关键点之间的连接关系对应的邻接矩阵。
进一步地，基于确定的上半身关键点与关键点之间的连接关系，可以构建度矩阵，度矩阵可以用于体现与提取到的关键点存在连接关系的边的数量。一般情况下，度矩阵是对角矩阵，其中所包括的顶点即为本公开实施例中的上半身的关键点的数据，对角上的各个元素代表了各个顶点的度，即为本公开实施例中的与各个上半身关键点存在连接关系的边的数量。基于上述设定，可以获取与全部的上半身关键点存在连接关系的边的数量对应的度矩阵。
S305，基于输入矩阵、度矩阵和邻接矩阵，获取第一图像特征。
本公开实施例中，输入矩阵可以确定提取到的上半身各个关键点的位置信息，邻接矩阵可以确定上半身各个关键点之间的连接关系，度矩阵可以确定与上半身每个关键点存在连接关系的边的数量。综合上述三个矩阵可以生成上半身关键点对应的图像特征，为第一图像特征。
进一步地，第一图像特征可以基于如下公式获取：
F (X，A)＝D -1AX
其中，X为输入矩阵、A为邻接矩阵、D为度矩阵，邻接矩阵与度矩阵所表征的上半身关键点与关键点之间的连接关系，对应人体的关节与关节之间的骨骼连接关系。
本公开提出的举手识别方法，通过构建输入矩阵、邻接矩阵与度矩阵，准确表现了提取到的上半身关键点的所处位置以及相互之间的连接关系，进一步地，基于输入矩阵、邻接矩阵与度矩阵生成第一图像特征，提高了第一图像特征的准确程度，进而使得基于第一图像特征获取的第一得分的准确程度得以保证，使得人体的抬手动作可以被精准识别。
上述实施例中，关于第二图像特征的提取可结合图4进一步理解，图4为本公开另一实施例的举手识别方法的流程示意图，如图4所示，该方法包括：
S401，基于关键点的位置信息，从手部关键点中确定出手肘关键点和手腕关键点。
本公开实施例中，人体的手部关键点可以包括人体的手肘对应的关键点、手腕对应的关键点以及手指对应的关键点等等。获取到全部的人体上半身的关键点后，可以基于关键点与关键点的位置信息，从全部的上半身关键点中获取人体手部的关键点。
基于每个手部关键点对应的手部组织结构，确定各个手部关键点的位置信息，进一步地，基于关键点与关键点之间的相对位置信息，确定手肘关键点，以及手腕关键点。
S402，获取手肘关键点和手腕关键点的置信度，确定手肘关键点和手腕关键点的置信度均大于设定阈值。
获取到手肘关键点以及手腕关键点后，需要对其位置信息进行进一步的确定。根据人体的基础结构，可以获取手肘关键点与手腕关键点的正确位置信息，可以基于该位置信息获取手肘关键点与手腕关键点的对应的位置信息的置信区间。进一步地，基于实际获取到的手肘关键点与手腕关键点的位置信息，获取手肘关键点与手腕关键点的置信度。
进一步地，可以设置一个对应的置信度阈值，当手肘关键点与手腕关键点的置信度大于设定阈值时，可以理解为，手肘关键点与手腕关键点的所处位置是合理正确的。
进一步地，确定获取置信度均大于预设的置信度阈值的手肘关键点以及手腕关键点。
S403，基于手肘关键点和手腕关键点确定手部检测框，并从手部检测框内提取第二图像特征。
基于手肘关键点以及手腕关键点，可以进一步确定人体手掌的位置，进而生成人体的手部检测框。可以将手部检测框输入人体手部状态检测模型，进而识别人体的手掌是否处于张开状态。
本公开实施例中，手肘关键点与手腕关键点之间的连接关系，可以对应人体肘关节与腕关节之间的骨骼连接关系。
一般情况下，每个人的手掌长度与前臂长度存在一定的比例关系，同样地，人体的手肘关键点、手腕关键点之间的连接关系与手腕关键点、最长的手指终点对应的关键点之间的连接关系存在比例关系，二者相同。
获取到手肘关键点以及手腕关键点之间的连接关系后，可以基于上述比例关系，获取到人体的手腕关键点与最长的手指终点对应的关键点之间的连接关系，基于上述两个连接关系可以生成人体的手部检测框。
进一步地，基于手部检测框内包含的人体手部的关键点，生成人体手部的第二图像特征。
其中，第二图像特征可以用于识别人体手部是否处于张开状态，以及处于张开状态的程度。
本公开提出的举手识别方法，通过手肘关键点以及手腕关键点，确定人体的手部检测框，进而从手部检测框中获取人体手部的第二图像特征。进一步地，可以基于第二图像特征实现人体手部是否处于张开状态的识别，从而有效提高举手识别的准确程度。
上述实施例中，手部检测框的获取方法可结合图5进一步理解，图5为本公开另一实施例的举手识别方法的流程示意图，如图5所示，该方法包括：
S501，生成手肘关键点与手腕关键点之间的连线。
确定手肘关键点与手腕关键点以后，将二者之间相连接，进而获取手肘关键点与手腕关键点之间的连接关系，其中，该连接关系对应人体的肘关节与腕关节之间的骨骼连接关系，为直线。
S502，沿着从手肘关键点向手腕关键点的方向延伸连线设定距离，以确定手指终点。
根据人体的基础结构，人体的肘关节与腕关节之间以前臂相连接，从手肘往手腕方向，基于前臂的延长线上，从手腕延伸出一定的距离，即为人体的手掌部分。一般情况下，人体的前臂长度与手掌长度存在比例关系。
相应地，确定手肘关键点与手腕关键点之间的直线连接关系后，沿手肘关键点至手腕关键点方向，基于两个关键点之间的直线连接关系进行延长，并延长至设定距离，该设定距离为人体手腕关键点至最长的手指终点对应的关键点之间的距离，与人体的手掌长度相同。
可以通过手肘关键点与手腕关键点之间的直线连接关系的长度确定设定距离的值。
进一步地，根据手肘关键点与手腕关键点的位置，获取手肘关键点与手腕关键点之间的距离，并基于上述前臂长度与手掌长度的比例关系，确定设定距离。
比如，设定手肘关键点为(x1,y1)，手腕关键点为(x2,y2)，则将(x1,y1)与(x2,y2)之间用直线相连接，再基于手肘关键点至手腕关键点方向进行延长。设定手肘至手腕之间的连接线长度为d1，手腕至指尖的连接线中最长的长度为d2，则d1与d2之间的比例关系
Figure PCTCN2022088032-appb-000001
设定p值为
Figure PCTCN2022088032-appb-000002
进一步地，基于如下的两点公式，可以获取到人体手部的最长手指的手指终点(x,y)：
Figure PCTCN2022088032-appb-000003
S503，基于手腕关键点和手指终点，生成手部检测框。
一般情况下，人体手部结构的手指终点可以多个，不同手指的终点与手腕关键点之间的距离是不同的。
本公开实施例中，手部检测框可以是具有规则形状的检测框，比如矩形等等。获取到手肘关键点、手腕关键点以及最长的手指终点对应的关键点后，可以确定全部的手部关键点的位置信息，基于所有的手部关键点的位置信息，生成手部检测框。
比如，设定手部检测框为矩形，则将手肘关键点至最长的手指终点对应的关键点之间的距离作为矩形的长，将手掌张开状态下的拇指终点对应的关键点与小指终点对应的关键点之间的距离作为矩形的宽，生成手部检测框。
本公开提出的举手识别方法，通过手肘关键点与手腕关键点之间的延长线获取手指终点，进一步基于手腕关键点与手指终点生成手部检测框，使得可以基于手部检测框内的手部关键点生成人体手部的第二图像特征，进而实现对于人体手部是否处于张开状态的检测识别，有效提高了举手识别的准确程度。
本公开实施例提出的举手识别方法，融合了人体抬手动作的识别以及人体手掌张开状态的识别。其中，基于人体上半身关键点提取到的可以用于抬手动作检测的第一图像特征获取第一得分，基于人体手部关键点提取到的可以用于手掌张开状态检测的第二图像特征获取第二得分，进一步地，基于第一得分和第二得分可以进行准确的举手识别。
可选地，可以分别对第一得分和第二得分进行分析，以获取检测结果，如图6所示，图6为本公开另一实施例的举手识别方法的流程示意图，该方法包括：
S601，响应于第一得分和第二得分均大于各自的第一预设得分阈值，则确定人体处于举手状态。
第一得分与第二得分分别存在一个对应的得分阈值，为第一预设得分阈值。
当第一得分大于其对应的第一预设得分阈值时，可以理解为，当前的第一图像特征中，人体可以被确定执行了抬手动作。若第一得分小于或者等于其对应的第一预设得分阈值，可以理解为， 当前状态下人体执行抬手动作的概率较小，且执行了其他非抬手的动作。
比如，设定第一得分与第一图像特征所属的采集图像中的人体手臂抬起后与桌面之间的角度为对应关系，第一得分所属区间为[0，1]。
设定第一得分对应的第一预设得分阈值为0.5，则该分数与人体的手臂抬起后与桌面之间角度为45°的状态相对应，可以理解为，当人体手臂抬起后与桌面之间角度大于45°时，第一得分大于0.5，进而可以确认人体执行了抬手动作。进一步地，设定当前第一得分为0.8，则第一得分大于其对应的第一得分阈值0.5，也就是说，当前状态下人体的手臂抬起后与桌面之间的角度大于45°，根据对应关系可知，第一得分为0.8时人体手臂抬起与桌面之间的角度为72°，可以确定，当前状态下，第一图像特征所属的采集图像中，人体执行了抬手动作。
同样地，当第二得分大于其对应的第一预设阈值时，可以理解为，当前状态下人体的手掌确定处于张开的状态。若第二得分小于或者等于其对应的第二得分阈值，可以理解为，当前状态下人体的手掌处于非张开状态的其他状态。
比如，设定第二得分所属区间为[0，1]，第二得分对应的第一预设得分阈值为0.5，可以理解为，当第二得分大于0.5时，人体的手掌确定处于张开状态。
进一步地，当第一得分大于其对应的第一预设得分阈值，同时第二得分大于其对应的第一预设得分阈值时，可以判断，当前的人体手臂执行了抬手动作且手掌处于张开状态，可以确定该状态下的人体执行了举手动作。
本公开提出的举手识别方法，通过对第一得分的分析，确定人体是否执行了抬手动作，通过对第二得分的分析，确定人体手部是否处于张开状态，进而将确定执行抬手动作且手部处于张开状态的动作确定识别为举手动作，融合了对人体姿态以及人手状态的双重识别，使得可以在相对模糊且人体较小的远景图像中实现人体的举手动作识别，有效提高了人体举手识别的准确率。
可选地，可以融合第一得分与第二得分一起分析，以获取检测结果，如图7所示，图7为本公开另一实施例的举手识别方法的流程示意图，该方法包括：
S701，对第一得分和第二得分进行融合，生成第三得分，响应于第三得分大于第二预设得分阈值，则确定人体处于举手状态。
将第一得分与第二得分进行融合，生成第三得分。并设置一个第三得分对应的第二预设得分阈值，当第三得分大于第二预设得分阈值时，可以确定，当前状态下人体执行了抬手动作同时手掌处于张开状态，进而可以确定该状态下人体执行了举手动作。
可选地，可以为第一得分与第二得分配置权重，基于权重值进行第一得分与第二得分的融合，以生成第三得分。其中，第三得分可以体现人体执行抬手动作的同时手掌处于张开状态的概率。
比如，设定第三得分的所属区间为[0，1]。设定第一得分为0.8，权重占比为0.7，第二得分为0.7，权重占比为0.3，则第三得分可以为0.8×0.7+0.7×0.3＝0.56+0.21＝0.77，可以理解为，第三得分为0.77时所对应的人体执行抬手动作同时手掌处于张开状态的概率为0.77。进一步地，设定第二预设得分阈值为0.6，则当第三得分大于0.6时，可以确定人体执行了抬手动作同时手掌处于张开状态，当第三得分小于或者等于0.6时，可以确定人体并未执行举手动作。本示例中，获取到第三得分为0.77，则第三得分大于第二预设得分阈值0.6，可以确定，当前状态下人体执行了举手动作。
本公开提出的举手识别方法，基于第一得分与第二得分获取第三得分，通过对第三得分的分析，确定执行了抬手动作的同时手掌处于张开状态的动作，从而实现对于举手动作的识别，融合了人体的抬手动作的检测以及人体手掌部位张开状态的检测，有效提高了举手识别的准确程度。
与上述几种实施例提供的举手识别方法相对应，本公开的一个实施例还提供一种举手识别装置，由于本公开实施例提供的举手识别装置与上述几种实施例提供的举手识别方法相对应，因此在举手识别方法的实施方式也适用于本公开实施例提供的举手识别装置，在下述实施例中不再详细描述。
图8为本公开一实施例的举手识别装置的结构示意图，如图8所示，举手识别装置800，包括提取模块81、识别模块82，其中：
提取模块81，用于从采集的人体图像中提取人体的上半身关键点，并基于上半身关键点提取第一特征图像，输入人体动作识别模型中，输出被识别为人体执行抬手动作的第一得分；
提取模块81，还用于从上半身关键点中确定手部关键点；
提取模块81，还用于基于手部关键点，提取人手的第二图像特征，输入人手分类模型中，输出被识别为手掌处于张开状态的第二得分；
识别模块82，用于根据第一得分和第二得分进行举手状态的识别。
图9为本公开另一实施例的举手识别装置的结构示意图，如图9所示，举手识别装置900，包括提取模块91、识别模块92，其中：
需要说明的是，提取模块81、识别模块82与提取模块91、识别模块92，具备相同的结构和功能。
本公开实施例中，提取模块91，还用于：基于关键点的位置信息，从手部关键点中确定出手肘关键点和手腕关键点；基于手肘关键点和手腕关键点确定手部检测框，并从手部检测框内提取第二图像特征。
本公开实施例中，提取模块91，还包括：
连接单元911，用于生成手肘关键点与手腕关键点之间的连线；
延伸单元912，用于沿着从手肘关键点向手腕关键点的方向延伸连线设定距离，以确定手指终点；
生成单元913，用于基于手腕关键点和手指终点，生成手部检测框。
本公开实施例中，提取模块91，还用于：根据手肘关键点与手腕关键点的位置，获取手肘关键点与手腕关键点之间的距离，根据距离确定设定距离。
本公开实施例中，提取模块91，还用于：获取手肘关键点和手腕关键点的置信度，确定手肘关键点和手腕关键点的置信度均大于设定阈值。
本公开实施例中，提取模块91，还用于：将人体图像输入人体检测模型中进行人体检测，输出人体检测框；将人体检测框输入人体关键点检测模型中进行特征提取，基于提取的特征进行关键点检测，输出上半身关键点。
本公开实施例中，提取模块91，还包括：
矩阵构建单元914，用于基于上半身关键点的位置信息和置信度，构建输入矩阵；
矩阵构建单元914，还用于基于上半身关键点之间的连接关系，构建邻接矩阵和度矩阵；
特征提取单元915，用于基于输入矩阵、度矩阵和邻接矩阵，获取第一图像特征。
本公开实施例中，识别模块92，还用于：响应于第一得分和第二得分均大于各自的第一预设得分阈值，则确定人体处于举手状态；或者，对第一得分和第二得分进行融合，生成第三得分，响应于第三得分大于第二预设得分阈值，则确定人体处于举手状态。
本公开提出的举手识别装置，提取采集图像中的人体的上半身关键点，生成第一图像特征，将第一图像特征输入人体动作识别模型进行抬手动作检测，获取第一得分。通过人体的手部关键点提取第二图像特征，将第二图像特征输入人体手部动作识别模型进行手掌张开状态检测，获取第二得分。进一步地，基于第一得分与第二得分，实现对于人体是否执行举手动作的识别。本公开中，融合了对人体姿态以及人手状态的双重识别，使得可以在相对模糊且人体较小的远景图像中实现人体的举手动作识别，从而有效提高人体举手识别的准确率。
图10示出了可以用来实施本公开的实施例的示例电子设备1000的示意性框图。电子设备旨在表示各种形式的数字计算机，诸如，膝上型计算机、台式计算机、工作台、个人数字助理、服务器、刀片式服务器、大型计算机、和其它适合的计算机。电子设备还可以表示各种形式的移动装置，诸如，个人数字处理、蜂窝电话、智能电话、可穿戴设备和其它类似的计算装置。本文所示的部件、它们的连接和关系、以及它们的功能仅仅作为示例，并且不意在限制本文中描述的和/或者要求的本公开的实现。
如图10所示，设备1000包括计算单元1001，其可以根据存储在只读存储器(ROM)1002中的计算机程序或者从存储单元1008加载到随机访问存储器(RAM)1004中的计算机程序，来 执行各种适当的动作和处理。在RAM 1004中，还可存储设备1000操作所需的各种程序和数据。计算单元1001、ROM 1002以及RAM 1004通过总线1004彼此相连。输入/输出(I/O)接口1005也连接至总线1004。
设备1000中的多个部件连接至I/O接口1005，包括：输入单元1006，例如键盘、鼠标等；输出单元1007，例如各种类型的显示器、扬声器等；存储单元1008，例如磁盘、光盘等；以及通信单元1009，例如网卡、调制解调器、无线通信收发机等。通信单元1009允许设备1000通过诸如因特网的计算机网络和/或各种电信网络与其他设备交换信息/数据。
计算单元1001可以是各种具有处理和计算能力的通用和/或专用处理组件。计算单元1001的一些示例包括但不限于中央处理单元(CPU)、图形处理单元(GPU)、各种专用的人工智能(AI)计算芯片、各种运行机器学习模型算法的计算单元、数字信号处理器(DSP)、以及任何适当的处理器、控制器、微控制器等。计算单元1001执行上文所描述的各个方法和处理，例如举手识别方法。例如，在一些实施例中，举手识别方法可被实现为计算机软件程序，其被有形地包含于机器可读介质，例如存储单元1008。在一些实施例中，计算机程序的部分或者全部可以经由ROM 1002和/或通信单元1009而被载入和/或安装到设备1000上。当计算机程序加载到RAM 1004并由计算单元1001执行时，可以执行上文描述的举手识别方法一个或多个步骤。备选地，在其他实施例中，计算单元1001可以通过其他任何适当的方式(例如，借助于固件)而被配置为执行举手识别方法。
本文中以上描述的系统和技术的各种实施方式可以在数字电子电路系统、集成电路系统、场可编程门阵列(FPGA)、专用集成电路(ASIC)、专用标准产品(ASSP)、芯片上系统的系统(SOC)、负载可编程逻辑设备(CPLD)、计算机硬件、固件、软件、和/或它们的组合中实现。这些各种实施方式可以包括：实施在一个或者多个计算机程序中，该一个或者多个计算机程序可在包括至少一个可编程处理器的可编程系统上执行和/或解释，该可编程处理器可以是专用或者通用可编程处理器，可以从存储系统、至少一个输入装置、和至少一个输出装置接收数据和指令，并且将数据和指令传输至该存储系统、该至少一个输入装置、和该至少一个输出装置。
用于实施本公开的方法的程序代码可以采用一个或多个编程语言的任何组合来编写。这些程序代码可以提供给通用计算机、专用计算机或其他可编程数据处理装置的处理器或控制器，使得程序代码当由处理器或控制器执行时使流程图和/或框图中所规定的功能/操作被实施。程序代码可以完全在机器上执行、部分地在机器上执行，作为独立软件包部分地在机器上执行且部分地在远程机器上执行或完全在远程机器或服务器上执行。
在本公开的上下文中，机器可读介质可以是有形的介质，其可以包含或存储以供指令执行系统、装置或设备使用或与指令执行系统、装置或设备结合地使用的程序。机器可读介质可以是机器可读信号介质或机器可读储存介质。机器可读介质可以包括但不限于电子的、磁性的、光学的、电磁的、红外的、或半导体系统、装置或设备，或者上述内容的任何合适组合。机器可读存储介质的更具体示例会包括基于一个或多个线的电气连接、便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦除可编程只读存储器(EPROM或快闪存储器)、光纤、便捷式紧凑盘只读存储器(CD-ROM)、光学储存设备、磁储存设备、或上述内容的任何合适组合。
为了提供与用户的交互，可以在计算机上实施此处描述的系统和技术，该计算机具有：用于向用户显示信息的显示装置(例如，CRT(阴极射线管)或者LCD(液晶显示器)监视器)；以及键盘和指向装置(例如，鼠标或者轨迹球)，用户可以通过该键盘和该指向装置来将输入提供给计算机。其它种类的装置还可以用于提供与用户的交互；例如，提供给用户的反馈可以是任何形式的传感反馈(例如，视觉反馈、听觉反馈、或者触觉反馈)；并且可以用任何形式(包括声输入、语音输入或者、触觉输入)来接收来自用户的输入。
可以将此处描述的系统和技术实施在包括后台部件的计算系统(例如，作为数据服务器)、或者包括中间件部件的计算系统(例如，应用服务器)、或者包括前端部件的计算系统(例如，具有图形用户界面或者网络浏览器的用户计算机，用户可以通过该图形用户界面或者该网络浏览 器来与此处描述的系统和技术的实施方式交互)、或者包括这种后台部件、中间件部件、或者前端部件的任何组合的计算系统中。可以通过任何形式或者介质的数字数据通信(例如，通信网络)来将系统的部件相互连接。通信网络的示例包括：局域网(LAN)、广域网(WAN)、互联网和区块链网络。
计算机系统可以包括客户端和服务器。客户端和服务器一般远离彼此并且通常通过通信网络进行交互。通过在相应的计算机上运行并且彼此具有客户端-服务器关系的计算机程序来产生客户端和服务器的关系。服务端可以是云服务器，又称为云计算服务器或云主机，是云计算服务体系中的一项主机产品，以解决了传统物理主机与VPS服务(“Virtual Private Server”，或简称“VPS”)中，存在的管理难度大，业务扩展性弱的缺陷。服务器也可以为分布式系统的服务器，或者是结合区块链的服务器
应该理解，可以使用上面所示的各种形式的流程，重新排序、增加或删除步骤。例如，本公开中记载的各步骤可以并行地执行也可以顺序地执行也可以不同的次序执行，只要能够实现本公开公开的技术方案所期望的结果，本文在此不进行限制。
上述具体实施方式，并不构成对本公开保护范围的限制。本领域技术人员应该明白的是，根据设计要求和其他因素，可以进行各种修改、组合、子组合和替代。任何在本公开的精神和原则之内所作的修改、等同替换和改进等，均应包含在本公开保护范围之内。

Claims (19)

  1. 一种举手识别方法，包括：
    从采集的人体图像中提取人体的上半身关键点，并基于所述上半身关键点提取第一图像特征，输入人体动作识别模型中，输出被识别为人体执行抬手动作的第一得分；
    从所述上半身关键点中确定手部关键点；
    基于所述手部关键点，提取人手的第二图像特征，输入人手分类模型中，输出被识别为手掌处于张开状态的第二得分；
    根据所述第一得分和所述第二得分进行举手状态的识别。
  2. 根据权利要求1所述的举手识别方法，其中，所述基于所述手部关键点，提取人手的第二图像特征，包括：
    基于所述关键点的位置信息，从所述手部关键点中确定出手肘关键点和手腕关键点；
    基于所述手肘关键点和手腕关键点确定手部检测框，并从所述手部检测框内提取所述第二图像特征。
  3. 根据权利要求2所述的举手识别方法，其中，所述基于所述手肘关键点和手腕关键点确定手部检测框，包括：
    生成所述手肘关键点与所述手腕关键点之间的连线；
    沿着从所述手肘关键点向所述手腕关键点的方向延伸所述连线设定距离，以确定手指终点；
    基于所述手腕关键点和所述手指终点，生成所述手部检测框。
  4. 根据权利要求3所述的举手识别方法，其中，所述沿着从所述手肘关键点向所述手腕关键点的方向延伸所述连线设定距离之前，还包括：
    根据所述手肘关键点与所述手腕关键点的位置，获取所述手肘关键点与所述手腕关键点之间的距离，根据所述距离确定所述设定距离。
  5. 根据权利要求2所述的举手识别方法，其中，所述基于所述手肘关键点和手腕关键点确定手部检测框之前，还包括：
    获取所述手肘关键点和所述手腕关键点的置信度，确定所述手肘关键点和所述手腕关键点的置信度均大于设定阈值。
  6. 根据权利要求1所述的举手识别方法，其中，所述从采集的人体图像中提取人体的上半身关键点，还包括：
    将所述人体图像输入人体检测模型中进行人体检测，输出人体检测框；
    将所述人体检测框输入人体关键点检测模型中进行特征提取，基于提取的特征进行关键点检测，输出所述上半身关键点。
  7. 根据权利要求1或6所述的举手识别方法，其特征在于，所述基于所述上半身关键点提取第一图像特征，包括：
    基于所述上半身关键点的位置信息和置信度，构建输入矩阵；
    基于所述上半身关键点之间的连接关系，构建邻接矩阵和度矩阵；
    基于所述输入矩阵、所述度矩阵和所述邻接矩阵，获取所述第一图像特征。
  8. 根据权利要求1-5任一项所述的举手识别方法，其中，所述根据所述第一得分和所述第二得分进行举手状态的识别，包括：
    响应于所述第一得分和所述第二得分均大于各自的第一预设得分阈值，则确定人体处于举手状态；或者，
    对所述第一得分和所述第二得分进行融合，生成第三得分，响应于所述第三得分大于第二预设得分阈值，则确定所述人体处于举手状态。
  9. 一种举手识别装置，包括：
    提取模块，用于从采集的人体图像中提取人体的上半身关键点，并基于所述上半身关键点提取第一图像特征，输入人体动作识别模型中，输出被识别为人体执行抬手动作的第一得分；
    所述提取模块，还用于从所述上半身关键点中确定手部关键点；
    所述提取模块，还用于基于所述手部关键点，提取人手的第二图像特征，输入人手分类模型中，输出被识别为手掌处于张开状态的第二得分；
    识别模块，用于根据所述第一得分和所述第二得分进行举手状态的识别。
  10. 根据权利要求9所述的举手识别装置，其中，所述提取模块，还用于：
    基于所述关键点的位置信息，从所述手部关键点中确定出手肘关键点和手腕关键点；
    基于所述手肘关键点和手腕关键点确定手部检测框，并从所述手部检测框内提取所述第二图像特征。
  11. 根据权利要求10所述的举手识别装置，其中，所述提取模块，还包括：
    连接单元，用于生成所述手肘关键点与所述手腕关键点之间的连线；
    延伸单元，用于沿着从所述手肘关键点向所述手腕关键点的方向延伸所述连线设定距离，以确定手指终点；
    生成单元，用于基于所述手腕关键点和所述手指终点，生成所述手部检测框。
  12. 根据权利要求11所述的举手识别装置，其中，所述提取获取模块，还用于：
    根据所述手肘关键点与所述手腕关键点的位置，获取所述手肘关键点与所述手腕关键点之间的距离，根据所述距离确定所述设定距离。
  13. 根据权利要求10所述的举手识别装置，其中，所述提取模块，还用于：
    获取所述手肘关键点和所述手腕关键点的置信度，确定所述手肘关键点和所述手腕关键点的置信度均大于设定阈值。
  14. 根据权利要求9所述的举手识别装置，其中，所述提取模块，还用于：
    将所述人体图像输入人体检测模型中进行人体检测，输出人体检测框；
    将所述人体检测框输入人体关键点检测模型中进行特征提取，基于提取的特征进行关键点检测，输出所述上半身关键点。
  15. 根据权利要求9或14所述的举手识别装置，其特征在于，所述提取模块，还包括：
    矩阵构建单元，用于基于所述上半身关键点的位置信息和置信度，构建输入矩阵；
    所述矩阵构建单元，还用于基于所述上半身关键点之间的连接关系，构建邻接矩阵和度矩阵；
    特征提取单元，用于基于所述输入矩阵、所述度矩阵和所述邻接矩阵，获取所述第一图像特征。
  16. 根据权利要求9-13任一项所述的举手识别装置，其中，所述识别模块，还用于：
    响应于所述第一得分和所述第二得分均大于各自的第一预设得分阈值，则确定人体处于举手状态；或者，
    对所述第一得分和所述第二得分进行融合，生成第三得分，响应于所述第三得分大于第二预设得分 阈值，则确定所述人体处于举手状态。
  17. 一种电子设备，包括：
    至少一个处理器；以及
    与所述至少一个处理器通信连接的存储器；其中，
    所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器能够执行权利要求1-8中任一项所述的方法。
  18. 一种存储有计算机指令的非瞬时计算机可读存储介质，其中，所述计算机指令用于使所述计算机执行根据权利要求1-8中任一项所述的方法。
  19. 一种计算机程序产品，包括计算机程序，所述计算机程序在被处理器执行时实现根据权利要求1-8中任一项所述的方法。
PCT/CN2022/088032 2021-05-12 2022-04-20 举手识别方法、装置、电子设备及存储介质 Ceased WO2022237481A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110519356.6A CN113221745B (zh) 2021-05-12 2021-05-12 举手识别方法、装置、电子设备及存储介质
CN202110519356.6 2021-05-12

Publications (1)

Publication Number Publication Date
WO2022237481A1 true WO2022237481A1 (zh) 2022-11-17

Family

ID=77095243

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/088032 Ceased WO2022237481A1 (zh) 2021-05-12 2022-04-20 举手识别方法、装置、电子设备及存储介质

Country Status (2)

Country Link
CN (1) CN113221745B (zh)
WO (1) WO2022237481A1 (zh)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115966024A (zh) * 2023-01-05 2023-04-14 北京达佳互联信息技术有限公司 动作捕捉方法、装置、电子设备及介质
CN116311529A (zh) * 2023-03-27 2023-06-23 杭州海康威视系统技术有限公司 一种拆解行为检测方法、电子设备及存储介质
CN117746491A (zh) * 2023-02-09 2024-03-22 书行科技(北京)有限公司 动作识别方法及装置、电子设备及计算机可读存储介质
CN120612214A (zh) * 2025-08-12 2025-09-09 南京览众智能科技有限公司 基于ai的课堂异常预警方法、系统、设备及存储介质

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113221745B (zh) * 2021-05-12 2023-09-01 北京百度网讯科技有限公司 举手识别方法、装置、电子设备及存储介质
CN114360047B (zh) * 2021-11-29 2026-03-10 深圳市鸿合创新信息技术有限责任公司 举手手势识别方法、装置、电子设备及存储介质
CN114451903A (zh) * 2021-12-15 2022-05-10 杭州沧澜医疗科技有限公司 一种基于姿态估计网络的骨龄仪辅助摆位方法及装置
CN114429647A (zh) * 2022-01-21 2022-05-03 上海浦东发展银行股份有限公司 一种递进式人物交互识别方法及系统
CN114546114A (zh) * 2022-02-15 2022-05-27 美的集团(上海)有限公司 移动机器人的控制方法、控制装置及移动机器人
CN114677753B (zh) * 2022-03-07 2025-05-27 北京京东尚科信息技术有限公司 人体部位的检测方法、动作识别方法、装置和电子设备
CN114842712B (zh) * 2022-04-12 2023-10-17 汕头大学 一种基于手势识别的手语教学系统
CN114863565B (zh) * 2022-05-11 2025-09-05 杭州萤石软件有限公司 一种手势动作识别方法、系统及电子设备
CN115641539B (zh) * 2022-11-11 2026-04-24 智洋创新科技股份有限公司 一种违规操作工具的检测方法、嵌入式终端及存储介质
CN115937971B (zh) * 2022-11-18 2023-09-08 广东保伦电子股份有限公司 一种举手投票识别方法及装置
CN119851352A (zh) * 2025-03-19 2025-04-18 深圳麦哲科技有限公司 一种手势识别方法及系统

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170315615A1 (en) * 2014-12-19 2017-11-02 Hewlett-Packard Development Company, L.P. Gesture library
CN108038452A (zh) * 2017-12-15 2018-05-15 厦门瑞为信息技术有限公司 一种基于局部图像增强的家电手势快速检测识别方法
CN110399822A (zh) * 2019-07-17 2019-11-01 思百达物联网科技(北京)有限公司 基于深度学习的举手动作识别方法、装置及存储介质
CN111126339A (zh) * 2019-12-31 2020-05-08 北京奇艺世纪科技有限公司 手势识别方法、装置、计算机设备和存储介质
CN112299172A (zh) * 2020-10-31 2021-02-02 成都新潮传媒集团有限公司 手势求救的识别方法、装置及存储介质
CN113221745A (zh) * 2021-05-12 2021-08-06 北京百度网讯科技有限公司 举手识别方法、装置、电子设备及存储介质
CN114360047A (zh) * 2021-11-29 2022-04-15 深圳市鸿合创新信息技术有限责任公司 举手手势识别方法、装置、电子设备及存储介质

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9524028B2 (en) * 2013-03-08 2016-12-20 Fastvdo Llc Visual language for human computer interfaces
DE112016001794T5 (de) * 2015-04-17 2018-02-08 Mitsubishi Electric Corporation Gestenerkennungsvorrichtung, Gestenerkennungsverfahren und Informationsverarbeitungsvorrichtung
CN107316059B (zh) * 2017-06-16 2020-07-28 陕西师范大学 学习者姿态识别方法

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170315615A1 (en) * 2014-12-19 2017-11-02 Hewlett-Packard Development Company, L.P. Gesture library
CN108038452A (zh) * 2017-12-15 2018-05-15 厦门瑞为信息技术有限公司 一种基于局部图像增强的家电手势快速检测识别方法
CN110399822A (zh) * 2019-07-17 2019-11-01 思百达物联网科技(北京)有限公司 基于深度学习的举手动作识别方法、装置及存储介质
CN111126339A (zh) * 2019-12-31 2020-05-08 北京奇艺世纪科技有限公司 手势识别方法、装置、计算机设备和存储介质
CN112299172A (zh) * 2020-10-31 2021-02-02 成都新潮传媒集团有限公司 手势求救的识别方法、装置及存储介质
CN113221745A (zh) * 2021-05-12 2021-08-06 北京百度网讯科技有限公司 举手识别方法、装置、电子设备及存储介质
CN114360047A (zh) * 2021-11-29 2022-04-15 深圳市鸿合创新信息技术有限责任公司 举手手势识别方法、装置、电子设备及存储介质

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115966024A (zh) * 2023-01-05 2023-04-14 北京达佳互联信息技术有限公司 动作捕捉方法、装置、电子设备及介质
CN115966024B (zh) * 2023-01-05 2026-01-23 北京达佳互联信息技术有限公司 动作捕捉方法、装置、电子设备及介质
CN117746491A (zh) * 2023-02-09 2024-03-22 书行科技(北京)有限公司 动作识别方法及装置、电子设备及计算机可读存储介质
CN116311529A (zh) * 2023-03-27 2023-06-23 杭州海康威视系统技术有限公司 一种拆解行为检测方法、电子设备及存储介质
CN120612214A (zh) * 2025-08-12 2025-09-09 南京览众智能科技有限公司 基于ai的课堂异常预警方法、系统、设备及存储介质

Also Published As

Publication number Publication date
CN113221745B (zh) 2023-09-01
CN113221745A (zh) 2021-08-06

Similar Documents

Publication Publication Date Title
WO2022237481A1 (zh) 举手识别方法、装置、电子设备及存储介质
CN113033566B (zh) 模型训练方法、识别方法、设备、存储介质及程序产品
WO2023016007A1 (zh) 人脸识别模型的训练方法、装置及计算机程序产品
JP7015152B2 (ja) キーポイントデータに関する加工装置、方法及びプログラム
CN113240778B (zh) 虚拟形象的生成方法、装置、电子设备和存储介质
CN111767846A (zh) 图像识别方法、装置、设备和计算机存储介质
WO2022227768A1 (zh) 动态手势识别方法、装置、设备以及存储介质
US20220139061A1 (en) Model training method and apparatus, keypoint positioning method and apparatus, device and medium
CN112347769A (zh) 实体识别模型的生成方法、装置、电子设备及存储介质
WO2021258588A1 (zh) 一种人脸图像识别方法、装置、设备及存储介质
CN111860362A (zh) 生成人脸图像校正模型及校正人脸图像的方法和装置
WO2022257614A1 (zh) 物体检测模型的训练方法、图像检测方法及其装置
CN114399424A (zh) 模型训练方法及相关设备
CN114550313A (zh) 图像处理方法、神经网络及其训练方法、设备和介质
CN112508004A (zh) 一种文字识别方法、装置、电子设备及存储介质
KR20220124273A (ko) 동적 제스처 인식 방법, 장치, 기기 및 저장 매체
CN111862031A (zh) 一种人脸合成图检测方法、装置、电子设备及存储介质
CN115393488A (zh) 虚拟人物表情的驱动方法、装置、电子设备和存储介质
CN111768005A (zh) 轻量级检测模型的训练方法、装置、电子设备及存储介质
KR102488517B1 (ko) 헤어스타일 변환 방법, 장치, 기기 및 저장 매체
CN112200169B (zh) 用于训练模型的方法、装置、设备以及存储介质
CN116543452A (zh) 手势识别、手势交互方法及装置
CN116453222A (zh) 目标对象姿态确定方法、训练方法、装置以及存储介质
CN110032270A (zh) 一种基于手势识别的人机交互方法
CN116185205A (zh) 非接触手势交互方法和装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22806451

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22806451

Country of ref document: EP

Kind code of ref document: A1