WO2020220809A1 - 目标对象的动作识别方法、装置和电子设备 - Google Patents

目标对象的动作识别方法、装置和电子设备 Download PDF

Info

Publication number
WO2020220809A1
WO2020220809A1 PCT/CN2020/076742 CN2020076742W WO2020220809A1 WO 2020220809 A1 WO2020220809 A1 WO 2020220809A1 CN 2020076742 W CN2020076742 W CN 2020076742W WO 2020220809 A1 WO2020220809 A1 WO 2020220809A1
Authority
WO
WIPO (PCT)
Prior art keywords
target object
action
key points
image
visibility
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2020/076742
Other languages
English (en)
French (fr)
Inventor
罗国中
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing ByteDance Network Technology Co Ltd
Original Assignee
Beijing ByteDance Network Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing ByteDance Network Technology Co Ltd filed Critical Beijing ByteDance Network Technology Co Ltd
Priority to JP2021503605A priority Critical patent/JP7181375B2/ja
Priority to GB2102752.9A priority patent/GB2598015B/en
Priority to US17/260,072 priority patent/US12008167B2/en
Publication of WO2020220809A1 publication Critical patent/WO2020220809A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/90—Dynamic range modification of images or parts thereof
    • G06T5/92—Dynamic range modification of images or parts thereof based on global image properties
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/56—Extraction of image or video features relating to colour
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/19—Recognition using electronic means
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/61—Control of cameras or camera modules based on recognised objects
    • H04N23/611—Control of cameras or camera modules based on recognised objects where the recognised objects include parts of the human body
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/77—Circuits for processing the brightness signal and the chrominance signal relative to each other, e.g. adjusting the phase of the brightness signal relative to the colour signal, correcting differential gain or differential phase

Definitions

  • the present disclosure relates to the field of target recognition and image processing, and in particular to a method, device and electronic equipment for recognizing target objects.
  • smart terminals can be used to listen to music, play games, chat online, and take photos.
  • the camera technology of the smart terminal the camera pixel has reached more than 10 million pixels, with higher definition and the camera effect comparable to professional cameras.
  • the state change of the target object between multiple frames can generally be used to determine the simple actions of the target object, such as the distance of the palm, etc.
  • An action recognition method for a target object comprising: obtaining an original image from an image source, the original image including a target object; recognizing the target object from the original image; detecting multiple key points of the target object ; Judging the visibility attributes of the multiple key points by the detected key points, where the visibility attributes are used to indicate whether the key points are occluded; according to the combined value of the visibility attributes of the multiple key points Identify the action of the target object.
  • the method further includes: determining whether the action of the target object is the first action; if the target object The action of is the first action, and the original image is processed using the first process corresponding to the first action.
  • the identifying the target object from the original image includes: acquiring a current image frame of the original image; extracting image features in the image frame to form a characteristic image; generating a plurality of characteristics in the characteristic image Rectangular frame; determine the number of image features of the target object contained in each rectangular frame; output the rectangular frame containing the largest number of image features of the target object as the circumscribed frame of the target object.
  • the detecting multiple key points of the target object includes: inputting the identified target object into a key point detection model; the key point detection model outputting the value of each key point on the original image coordinate.
  • the judging the visibility attributes of the multiple key points based on the detected key points, wherein the visibility attributes are used to indicate whether the key points are occluded including: detecting multiple key points
  • the target object inputs a visibility judgment model; the visibility judgment model outputs the visibility probability of each of the plurality of key points; compares the visibility probability with a first threshold and outputs the value of the visibility attribute.
  • the action of identifying the target object according to the combined value of the visibility attributes of the multiple key points includes: obtaining a reference value of the combined value of the visibility attributes of the multiple key points, wherein the The reference value corresponds to a specific action; the combined value of the visibility attribute is compared with the reference value; if the combined value matches the reference value, the specific action corresponding to the reference value is output as recognized The target's actions.
  • the action of identifying the target object according to the combined value of the visibility attributes of the plurality of key points includes: obtaining a reference value for change of the combined value of the visibility attributes of the plurality of key points, wherein The change reference value of the combined value corresponds to a specific action; obtains two combined values of the visibility attributes of the multiple key points in two consecutive image frames; calculates the change value of the combined value through the two combined values; The change value of the combination value is compared with the change reference value of the combination value; if the change value of the combination value matches the change reference value of the combination value, output the specific action corresponding to the change reference value Is the action of the identified target object.
  • processing the original image using a first process corresponding to the first action includes: responding to the action of the target object when the first action is An action is to obtain the first process and the parameters of the first process corresponding to the first action; use the first process and the parameters of the first process to process the original image to obtain the processed image.
  • the first process is to reduce the brightness of the image
  • the parameter of the first process is the percentage of brightness reduction
  • the original image is processed using the first process and the parameters of the first process to obtain a processed image , Including: obtaining the three color components of the three channels of each pixel of the original image in the RGB color space; subtracting the percentage of the color component values from the three color components of each pixel to obtain the processed each The three color component values of each pixel.
  • An action recognition device for a target object including:
  • the original image acquisition module is used to obtain the original image from the image source, the original image includes the target object; the target object recognition module is used to recognize the target object from the original image; the key point detection module is used to Detect multiple key points of the target object; a visibility judgment module is used to judge the visibility attributes of the multiple key points through the detected key points, wherein the visibility attribute is used to indicate whether the key points are Obscured; an action recognition module for recognizing the action of the target object according to the combined value of the visibility attributes of the multiple key points.
  • the device further includes: a first action judging module, configured to judge whether the action of the target object is the first action; a first processing module, configured to determine whether the action of the target object is the first action, Then, the original image is processed using the first process corresponding to the first action.
  • the target object recognition module further includes: an image frame acquisition module for acquiring the current image frame of the original image; a feature extraction module for extracting image features in the image frame to form a feature image; rectangular frame generation Module, used to generate multiple rectangular frames in the feature image; Quantity judgment module, used to judge the number of image features of the target object contained in each rectangular frame; External frame output module, used to include the target object The rectangular frame with the largest number of image features is output as the outer frame of the target object.
  • the key point detection module further includes: a first input module, configured to input the identified target object into a key point detection model; a first output module, configured to output each key point detection model The coordinates of the key point on the original image.
  • the visibility judgment module further includes: a second input module for inputting the target object with multiple key points detected into the visibility judgment model; the visibility probability judgment module for the visibility judgment model output The visibility probability of each of the plurality of key points; a second output module, configured to compare the visibility probability with a first threshold and output the value of the visibility attribute.
  • the action recognition module further includes: a reference value obtaining module for obtaining a reference value of a combined value of the visibility attributes of the multiple key points, wherein the reference value corresponds to a specific action; comparison of reference values A module for comparing the combined value of the visibility attribute with the reference value; a first identification module for outputting the specific action corresponding to the reference value if the combined value matches the reference value Is the action of the identified target object.
  • the action recognition module further includes: a change reference value obtaining module, configured to obtain a change reference value of a combined value of the visibility attributes of the plurality of key points, wherein the change reference value of the combined value and a specific Action correspondence; a combined value obtaining module for obtaining two combined values of the visibility attributes of the multiple key points in two consecutive image frames; a change value calculation module for calculating a combined value from the two combined values The change value of the combined value; the change reference value comparison module, which is used to compare the change value of the combined value with the change reference value of the combined value; the second identification module, which is used to if the change value of the combined value is The change reference value of the combined value matches, and the specific action corresponding to the change reference value is output as the action of the identified target object.
  • a change reference value obtaining module configured to obtain a change reference value of a combined value of the visibility attributes of the plurality of key points, wherein the change reference value of the combined value and a specific Action correspondence
  • a combined value obtaining module for
  • the first processing module further includes: a first processing and parameter acquisition module, configured to acquire the first processing and first processing corresponding to the first action in response to the first action when the target object moves. Processing parameters; a first processing sub-module for processing the original image using the first processing and the first processing parameters to obtain a processed image.
  • the first processing is to reduce the brightness of the image
  • the parameter of the first processing is the percentage of the brightness reduction
  • the first processing sub-module includes: a color acquisition module for acquiring each pixel in the original image The three color components of the three channels in the RGB color space; the brightness processing module is used to subtract the percentage of the color component values from the three color components of each pixel to obtain the processed three color components of each pixel The color component value.
  • An electronic device comprising: a memory for storing non-transitory computer readable instructions; and a processor for running the computer readable instructions so that the processor realizes the action recognition of any of the aforementioned target objects when executed The steps described in the method.
  • a computer-readable storage medium for storing non-transitory computer-readable instructions.
  • the non-transitory computer-readable instructions When executed by a computer, the computer can execute the steps in any of the above methods.
  • the present disclosure discloses a method, device and electronic equipment for recognizing the action of a target object.
  • the action recognition method of the target object includes: obtaining an original image from an image source, the original image including the target object; recognizing the target object from the original image; detecting multiple keys of the target object Point; judging the visibility attributes of the multiple key points by the detected key points, where the visibility attributes are used to indicate whether the key points are occluded; according to the combination of the visibility attributes of the multiple key points
  • the value identifies the action of the target object.
  • the present disclosure judges the action of the target object through the visibility of key points, and solves the technical problem of inaccurate recognition of complex actions in the prior art.
  • FIG. 1 is a schematic flowchart of a method for recognizing a target object's action according to an embodiment of the present disclosure
  • FIG. 2 is a further schematic flowchart of step S102 of the method for recognizing the action of a target object according to an embodiment of the present disclosure
  • FIG. 3 is a further schematic flowchart of step S103 of the method for recognizing the action of a target object according to an embodiment of the present disclosure
  • step S104 is a further schematic flowchart of step S104 of the method for recognizing the action of a target object according to an embodiment of the present disclosure
  • FIG. 5 is a further schematic flowchart of step S105 of the method for recognizing the action of a target object according to an embodiment of the present disclosure
  • FIG. 6 is a further schematic flowchart of step S105 of the method for recognizing the action of a target object according to an embodiment of the present disclosure
  • Fig. 7 is a schematic structural diagram of a target object action recognition device according to an embodiment of the present disclosure.
  • Fig. 8 is a schematic structural diagram of an electronic device provided according to an embodiment of the present disclosure.
  • the embodiment of the present disclosure provides a method for recognizing the action of a target object.
  • the target object action recognition method provided in this embodiment can be executed by a computing device, the computing device can be implemented as software, or as a combination of software and hardware, and the computing device can be integrated in a server or terminal. Equipment etc.
  • the method for recognizing the action of the target object mainly includes the following steps S101 to S105. among them:
  • Step S101 Obtain an original image from an image source, the original image includes a target object;
  • the image source is a local storage space or a network storage space
  • the obtaining of the original image from the image source includes obtaining the original image from the local storage space or obtaining the original image from the network storage space, no matter where it is obtained
  • the original image it is first necessary to obtain the storage address of the original image, and then obtain the original image from the storage address.
  • the original image includes multiple frames of images.
  • the original image can be a video or a picture with dynamic effects, as long as it is with Images with multiple frames may be the original images in the present disclosure.
  • the image source may be an image sensor, and acquiring the original image from the image source includes acquiring the original image from the image sensor.
  • the image sensor refers to various devices that can collect images, and typical image sensors are video cameras, cameras, and cameras.
  • the image sensor may be a camera on a mobile terminal, such as a front or rear camera on a smart phone.
  • the original image collected by the camera may be directly displayed on the display screen of the phone.
  • the original image includes a target object
  • the target object may be one or more specific objects, such as human hands, human faces, human bodies, various animals, and so on.
  • the user uses the camera of the mobile terminal to take an image of his human body.
  • the mobile terminal displays the captured image on the display device of the mobile terminal.
  • the user can make various actions in front of the camera.
  • the posture of the human hand in the human body image collected by the camera is detected.
  • the identification and detection of the target object will be described in the next few steps, and will not be repeated here.
  • Step S102 Identify the target object from the original image
  • identifying the target object from the original image may include the following steps:
  • Step S201 Obtain the current image frame of the original image
  • Step S202 Extract image features in the image frame to form a feature image
  • Step S203 Generate multiple rectangular frames in the characteristic image
  • Step S204 Determine the number of image features of the target object contained in each rectangular frame
  • Step S205 Output the rectangular frame containing the largest number of image features of the target object as the circumscribed frame of the target object.
  • the current image frame of the original image is obtained.
  • the image frame is essentially a picture.
  • one second video image can include 30 frames, that is, one picture is obtained every 33 ms.
  • the current image frame is the picture at the current moment when the video image is played.
  • step S202 the image features are extracted from the image frame to form a feature image, where the image feature can be extracted by convolution, and the set convolution kernel is used to extract and calculate the pixels on the image frame to form the feature
  • An image such as a 10*10 video frame, is extracted by a 3*3 convolution kernel with a step size of 1, and an 8*8 feature image can be generated.
  • the image frame can be convolved with multi-layer convolution kernels to generate smaller feature images, or multiple convolution kernels can be used to generate different feature images, or the two methods can be combined to produce more With smaller and more feature images, these feature images can finally be fused together to generate a feature image.
  • the method for generating the characteristic image can use any one or a combination of the above, which is not limited here.
  • a plurality of rectangular frames are generated in the feature image to frame the feature of the target object.
  • the characteristic image may be divided into a plurality of rectangular regions, each rectangular region includes a center point, and a plurality of rectangular frames of preset size are generated to surround the center point, wherein the center point of the rectangular frame is connected to the center point.
  • the center points of the rectangular areas coincide, thereby a plurality of rectangles surrounding the rectangular area or surrounded by the rectangular area can be generated.
  • step S204 it is determined that the rectangle generated in step S203 contains the number of image features of the target object.
  • the number may be represented by the confidence that the target object is included in the rectangle, and the higher the confidence is, the more image features of the target object are included in the rectangle.
  • step S205 the rectangular frame including the largest number of image features of the target object is output as the circumscribed frame of the target object.
  • Step S103 Detect multiple key points of the target object
  • the detecting multiple key points of the target object may include:
  • Step S301 Input the identified target object into the key point detection model
  • Step S302 The key point detection model outputs the coordinates of each key point on the original image.
  • the key point detection model is a pre-trained model for detecting key points
  • training the key point detection model requires the use of multiple images of target objects with key points as the training atlas, typically ,
  • the target object is a human hand
  • the key points are 22 key points numbered 0-21.
  • the 22 key points on the human hand in the picture are marked as the training atlas and input into the model.
  • the coordinates of the predicted key points are compared with the coordinates of the key points on the training atlas, and the errors of the two are fed back to the key point detection model.
  • the key point detection model adjusts the model parameters according to the error and continues to use the pictures in the training atlas for prediction.
  • the target object identified in step S102 is input into the above-mentioned trained key point detection model, and the key point detection model outputs the coordinates of each numbered key point on the original image.
  • the target object input here can be the target object image in the outer frame in step S102, so that the image is smaller and the recognition speed is faster. Because the size and center point of the outer frame are already output when outputting. In this way, the coordinates of the key points relative to the original image can be calculated according to the position and size of the outer frame. The calculation method is not repeated here.
  • Step S104 Judging the visibility attributes of the multiple key points based on the detected key points, where the visibility attributes are used to indicate whether the key points are blocked;
  • judging the visibility attributes of the plurality of key points based on the detected key points may include the following steps:
  • Step S401 Input the target object with multiple key points detected into the visibility judgment model
  • Step S402 the visibility judgment model outputs the visibility probability of each of the multiple key points
  • Step S403 Compare the visibility probability with a first threshold and output the value of the visibility attribute.
  • a visibility judgment model needs to be used.
  • the model is trained using annotated training set.
  • the annotated training set includes pictures of the target object in various angles and poses, and annotated the target Whether each key point in the object is visible, that is, whether it is occluded.
  • the output of the model is the probability that the key points are visible. If the probability is higher than a threshold, it can be determined If it is consistent with the label, if it is different, the loss function is calculated and the error is fed back to the model to adjust the model parameters.
  • the parameters are adjusted repeatedly until the output result matches the label, and the model training is completed.
  • the target object with multiple key points detected is input into the visibility judgment model, and then the visibility judgment model outputs the visibility probability of each of the multiple key points, and the visibility probability
  • the threshold can be 0.8, that is, when the output probability is greater than 0.8, the key point is considered visible, and when the output probability is lower than 0.8, the key point is considered invisible.
  • Step S105 Identify the action of the target object according to the combined value of the visibility attributes of the multiple key points.
  • the action of identifying the target object according to the combined value of the visibility attributes of the multiple key points may include the following steps:
  • Step S501 Obtain a reference value of the combined value of the visibility attributes of the multiple key points, where the reference value corresponds to a specific action;
  • Step S502 comparing the combined value of the visibility attribute with the reference value
  • Step S503 If the combined value matches the reference value, output the specific action corresponding to the reference value as the action of the identified target object.
  • the value of the output visibility attribute may be a combined value of the visibility attribute, that is, a vector value.
  • the reference value of the combined value of the visibility attribute of the action of one or more target objects can be preset in the configuration file, which can also be a vector, such as a 1*N vector, after obtaining the combined value of the visibility attribute from Acquire the reference value of the combined value of the one or more visibility attributes in the configuration file, and then compare the combined value of the visibility attribute with the reference value.
  • the comparison may be to compare two The similarity of two vectors, commonly used methods can use Pearson correlation coefficient, Euclidean distance, cosine similarity, Manhattan distance, etc. to judge the similarity of two vectors, if the similarity is higher than a preset second threshold , It is determined that the combined value matches the reference value, and the specific action corresponding to the reference value is output as the action of the identified target object. At this point, the action of the target object is recognized.
  • the action of identifying the target object according to the combined value of the visibility attributes of the multiple key points may further include the following steps:
  • Step S601 Obtain a change reference value of the combined value of the visibility attributes of the multiple key points, where the change reference value of the combined value corresponds to a specific action;
  • Step S602 Obtain two combined values of the visibility attributes of the multiple key points in two consecutive image frames
  • Step S603 Calculate the change value of the combined value through the two combined values
  • Step S604 Compare the change value of the combined value with the change reference value of the combined value
  • Step S605 If the change value of the combination value matches the change reference value of the combination value, output the specific action corresponding to the change reference value as the action of the identified target object.
  • a dynamic action of a target object can be judged, such as a changing action.
  • the target object is a human hand
  • a finger snapping action of the human hand can be identified, and the finger snapping action requires at least two The state of each hand can be recognized.
  • the value of the output visibility attribute may be a combined value of the visibility attribute, that is, a vector value.
  • the change reference value of the combined value of the visibility attribute of the action of one or more target objects can be preset in the configuration file, which can also be two vectors, such as two 1*N vectors, which respectively indicate that the target object is in two The reference value of the combined value of the visibility attribute in each state.
  • the change value may be two combined values A new vector formed end to end, such as a 1*2N vector combined by two 1*N vectors.
  • the obtained change reference value of the combined value of the visibility attribute may also be a combined vector of two 1*N vectors.
  • the similarity is higher than a preset third threshold, it is determined that the change value of the combined value matches the change reference value, and the specific action corresponding to the change reference value is output as the action of the identified target object . At this point, the action of the target object is recognized.
  • step S105 it may further include the following steps:
  • the original image is processed using a first process corresponding to the first action.
  • the first process is to reduce the brightness of the image
  • the parameter of the first process is the percentage of brightness reduction
  • the original image is processed by using the first process and the parameters of the first process to obtain a processed image
  • the three color components of each pixel are respectively subtracted from the percentage color component values to obtain the processed three color component values of each pixel.
  • the target object may be a human hand
  • the action may be snapping a finger
  • the first processing is to reduce the brightness of the image
  • the first processing parameter is 50%, when it is used to shoot a video using the camera of a mobile terminal, when When a human hand appears in the video, and the human hand is recognized as making a finger snapping action, the brightness of the video on the screen of the mobile terminal is reduced by 50% to achieve an effect of turning off the light by snapping the finger.
  • the method for recognizing the action of the target object of the present disclosure can realize the triggering of any special effect.
  • the above-mentioned effect of turning on the finger and turning off the light is only an example, and the scheme of triggering other effects will not be repeated.
  • the present disclosure discloses a method, device and electronic equipment for recognizing the action of a target object.
  • the action recognition method of the target object includes: obtaining an original image from an image source, the original image including the target object; recognizing the target object from the original image; detecting multiple keys of the target object Point; judging the visibility attributes of the multiple key points by the detected key points, where the visibility attributes are used to indicate whether the key points are occluded; according to the combination of the visibility attributes of the multiple key points
  • the value identifies the action of the target object.
  • the present disclosure judges the action of the target object through the visibility of key points, and solves the technical problem of inaccurate recognition of complex actions in the prior art.
  • the device embodiments of the present disclosure can be used to perform the steps implemented by the method embodiments of the present disclosure.
  • the embodiment of the present disclosure provides an image processing device.
  • the device can execute the steps described in the above-mentioned embodiment of the target object action recognition method.
  • the device 700 mainly includes: an original image acquisition module 701, a target object recognition module 702, a key point detection module 703, a visibility judgment module 704, and an action recognition module 705. among them,
  • the original image acquisition module 701 is configured to acquire an original image from an image source, and the original image includes a target object;
  • the target object recognition module 702 is configured to recognize the target object from the original image
  • the key point detection module 703 is used to detect multiple key points of the target object
  • the visibility judgment module 704 is configured to judge the visibility attributes of the multiple key points based on the detected key points, where the visibility attributes are used to indicate whether the key points are blocked;
  • the action recognition module 705 is configured to recognize the action of the target object according to the combined value of the visibility attributes of the multiple key points.
  • the device 700 further includes:
  • the first action judgment module is used to judge whether the action of the target object is the first action
  • the first processing module is configured to, if the action of the target object is a first action, use a first process corresponding to the first action to process the original image.
  • target object recognition module 702 further includes:
  • Image frame acquisition module for acquiring the current image frame of the original image
  • the feature extraction module is used to extract image features in the image frame to form a feature image
  • a rectangular frame generating module configured to generate a plurality of rectangular frames in the characteristic image
  • the quantity judgment module is used to judge the quantity of the image features of the target object contained in each rectangular frame
  • the outer frame output module is used to output the rectangular frame containing the largest number of image features of the target object as the outer frame of the target object.
  • the key point detection module 703 further includes:
  • the first input module is used to input the identified target object into the key point detection model
  • the first output module is used for the key point detection model to output the coordinates of each key point on the original image.
  • the visibility judgment module 704 further includes:
  • the second input module is used to input the target object with multiple key points detected into the visibility judgment model
  • the visibility probability judgment module is used for the visibility judgment model to output the visibility probability of each of the multiple key points;
  • the second output module is configured to compare the visibility probability with a first threshold and output the value of the visibility attribute.
  • the action recognition module 705 further includes:
  • a reference value obtaining module configured to obtain a reference value of a combined value of the visibility attributes of the multiple key points, where the reference value corresponds to a specific action
  • a reference value comparison module configured to compare the combined value of the visibility attribute with the reference value
  • the first recognition module is configured to output the specific action corresponding to the reference value as the action of the identified target object if the combined value matches the reference value.
  • the action recognition module 705 further includes:
  • a change reference value obtaining module configured to obtain a change reference value of a combined value of the visibility attributes of the multiple key points, wherein the change reference value of the combined value corresponds to a specific action
  • a combined value obtaining module configured to obtain two combined values of the visibility attributes of the multiple key points in two consecutive image frames
  • a change value calculation module configured to calculate a change value of the combined value through the two combined values
  • a change reference value comparison module configured to compare the change value of the combined value with the change reference value of the combined value
  • the second recognition module is configured to output the specific action corresponding to the change reference value as the action of the identified target object if the change value of the combined value matches the change reference value of the combined value.
  • the first processing module further includes:
  • the first processing and parameter acquisition module is configured to acquire the first processing and the parameters of the first processing corresponding to the first action in response to the first action of the target object;
  • the first processing sub-module is configured to use the first processing and the parameters of the first processing to process the original image to obtain a processed image.
  • the first processing is to reduce image brightness
  • a parameter of the first processing is a percentage of brightness reduction
  • the first processing submodule includes:
  • the color acquisition module is used to acquire the three color components of the three channels in the RGB color space of each pixel of the original image
  • the brightness processing module is configured to subtract the color component values of the percentage from the three color components of each pixel to obtain the processed three color component values of each pixel.
  • the device shown in FIG. 7 can execute the methods of the embodiments shown in FIG. 1 to FIG. 6.
  • parts that are not described in detail in this embodiment please refer to the related descriptions of the embodiments shown in FIG. 1 to FIG. 6.
  • For the implementation process and technical effects of the technical solution please refer to the description in the embodiment shown in FIG. 1 to FIG. 6, which will not be repeated here.
  • FIG. 8 shows a schematic structural diagram of an electronic device 800 suitable for implementing embodiments of the present disclosure.
  • Electronic devices in the embodiments of the present disclosure may include, but are not limited to, mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablets), PMPs (portable multimedia players), vehicle-mounted terminals (for example, Mobile terminals such as car navigation terminals) and fixed terminals such as digital TVs, desktop computers, etc.
  • the electronic device shown in FIG. 8 is only an example, and should not bring any limitation to the function and scope of use of the embodiments of the present disclosure.
  • the electronic device 800 may include a processing device (such as a central processing unit, a graphics processor, etc.) 801, which may be loaded into a random access device according to a program stored in a read-only memory (ROM) 802 or from a storage device 808
  • the program in the memory (RAM) 803 executes various appropriate actions and processing.
  • the RAM 803 also stores various programs and data required for the operation of the electronic device 800.
  • the processing device 801, the ROM 802, and the RAM 803 are connected to each other through a bus 804.
  • An input/output (I/O) interface 805 is also connected to the bus 804.
  • the following devices can be connected to the I/O interface 805: including input devices 806 such as touch screen, touch panel, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; including, for example, liquid crystal display (LCD), speakers, An output device 807 such as a vibrator; a storage device 808 such as a magnetic tape and a hard disk; and a communication device 809.
  • the communication device 809 may allow the electronic device 800 to perform wireless or wired communication with other devices to exchange data.
  • FIG. 8 shows an electronic device 800 having various devices, it should be understood that it is not required to implement or have all the illustrated devices. It may alternatively be implemented or provided with more or fewer devices.
  • an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart.
  • the computer program may be downloaded and installed from the network through the communication device 809, or installed from the storage device 808, or installed from the ROM 802.
  • the processing device 801 the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.
  • the aforementioned computer-readable medium in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two.
  • the computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or a combination of any of the above. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable Programmable read only memory (EPROM or flash memory), optical fiber, portable compact disk read only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.
  • a computer-readable storage medium may be any tangible medium that contains or stores a program, and the program may be used by or in combination with an instruction execution system, apparatus, or device.
  • a computer-readable signal medium may include a data signal propagated in a baseband or as a part of a carrier wave, and a computer-readable program code is carried therein. This propagated data signal can take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing.
  • the computer-readable signal medium may also be any computer-readable medium other than the computer-readable storage medium.
  • the computer-readable signal medium may send, propagate or transmit for use by or in combination with the instruction execution system, apparatus, or device. program.
  • the program code contained on the computer-readable medium can be transmitted by any suitable medium, including but not limited to: wire, optical cable, RF (Radio Frequency), etc., or any suitable combination of the above.
  • the above-mentioned computer-readable medium may be included in the above-mentioned electronic device; or it may exist alone without being assembled into the above-mentioned electronic device.
  • the aforementioned computer-readable medium carries one or more programs, and when the aforementioned one or more programs are executed by the electronic device, the electronic device: obtains an original image from an image source, and the original image includes a target object; Identify the target object from the original image; detect multiple key points of the target object; determine the visibility attributes of the multiple key points through the detected key points, where the visibility attributes are used Indicates whether the key point is occluded; the action of the target object is recognized according to the combined value of the visibility attributes of the multiple key points.
  • the computer program code used to perform the operations of the present disclosure may be written in one or more programming languages or a combination thereof.
  • the above-mentioned programming languages include object-oriented programming languages—such as Java, Smalltalk, C++, and also conventional Procedural programming language-such as "C" language or similar programming language.
  • the program code can be executed entirely on the user's computer, partly on the user's computer, executed as an independent software package, partly on the user's computer and partly executed on a remote computer, or entirely executed on the remote computer or server.
  • the remote computer can be connected to the user’s computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (for example, using an Internet service provider to pass Internet connection).
  • LAN local area network
  • WAN wide area network
  • each block in the flowchart or block diagram may represent a module, program segment, or part of code, and the module, program segment, or part of code contains one or more logic for implementing prescribed Function executable instructions.
  • the functions marked in the block may also occur in a different order from the order marked in the drawings. For example, two blocks shown in succession can actually be executed substantially in parallel, or they can sometimes be executed in the reverse order, depending on the functions involved.
  • each block in the block diagram and/or flowchart, and the combination of the blocks in the block diagram and/or flowchart can be implemented by a dedicated hardware-based system that performs the specified functions or operations Or it can be realized by a combination of dedicated hardware and computer instructions.
  • the units involved in the embodiments described in the present disclosure may be implemented in a software manner, or may be implemented in a hardware manner. Among them, the name of the unit does not constitute a limitation on the unit itself under certain circumstances.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Human Computer Interaction (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Psychiatry (AREA)
  • Social Psychology (AREA)
  • General Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Image Analysis (AREA)

Abstract

一种目标对象的动作识别方法、装置和电子设备。其中，所述目标对象的动作识别方法包括：从图像源获取原始图像，所述原始图像中包括目标对象（S101）；从所述原始图像中识别出所述目标对象（S102）；检测所述目标对象的多个关键点（S103）；通过检测到的关键点判断所述多个关键点的可见性属性，其中所述可见性属性用于表示所述关键点是否被遮挡（S104）；根据所述多个关键点的可见性属性的组合值识别所述目标对象的动作（S105）。该方法关键点的可见性来判断目标对象的动作，解决了现有技术中的对复杂动作识别不准确的技术问题。

Description

目标对象的动作识别方法、装置和电子设备
相关申请的交叉引用
本申请要求于2019年04月29日提交的，申请号为201910353500.6、发明名称为“目标对象的动作识别方法、装置和电子设备”的中国专利申请的优先权，该申请的全文通过引用结合在本申请中。
技术领域
本公开涉及目标识别以及图像处理领域，特别是涉及一种目标对象的动作识别方法、装置和电子设备。
背景技术
随着计算机技术的发展，智能终端的应用范围得到了广泛的提高，例如可以通过智能终端听音乐、玩游戏、上网聊天和拍照等。对于智能终端的拍照技术来说，其拍照像素已经达到千万像素以上，具有较高的清晰度和媲美专业相机的拍照效果。
目前在采用智能终端进行拍照时，不仅可以使用出厂时内置的拍照软件实现传统功能的拍照效果，还可以通过从网络端下载应用程序(Application，简称为：APP)来实现具有附加功能的拍照效果或者给视频加上特殊效果。在加入效果之前，有时候还会对用户的动作进行识别。
现有技术中一般可以使用多帧之间目标对象的状态变化来判断目标对象的简单动作，比如手掌的远近等，但是这样仅仅通过简单的状态的变化来判断目标对象的动作的方式无法识别一些比较复杂的动作。
发明内容
根据本公开的一个方面，提供以下技术方案：
一种目标对象的动作识别方法，包括：从图像源获取原始图像，所述原始图像中包括目标对象；从所述原始图像中识别出所述目标对象；检测所述目标对象的多个关键点；通过检测到的关键点判断所述多个关键点的 可见性属性，其中所述可见性属性用于表示所述关键点是否被遮挡；根据所述多个关键点的可见性属性的组合值识别所述目标对象的动作。
进一步的，在所述根据所述多个关键点的可见性属性的组合值识别所述目标对象的动作之后，还包括：判断所述目标对象的动作是否为第一动作；如果所述目标对象的动作是第一动作，则使用与所述第一动作对应的第一处理对所述原始图像进行处理。
进一步的，所述从所述原始图像中识别出所述目标对象，包括：获取原始图像的当前图像帧；抽取所述图像帧中的图像特征形成特征图像；在所述特征图像中生成多个矩形框；判断每个矩形框中所包含的目标对象的图像特征的数量；将包含目标对象的图像特征数量最多的矩形框输出作为目标对象的外接框。
进一步的，所述检测所述目标对象的多个关键点，包括：将所述识别出的目标对象输入关键点检测模型；所述关键点检测模型输出每个关键点在所述原始图像上的坐标。
进一步的，所述通过检测到的关键点判断所述多个关键点的可见性属性，其中所述可见性属性用于表示所述关键点是否被遮挡，包括：将检测出多个关键点的目标对象输入可见性判断模型；可见性判断模型输出所述多个关键点中的每一个的可见性概率；将所述可见性概率与第一阈值比较并输出所述可见性属性的值。
进一步的，所述根据所述多个关键点的可见性属性的组合值识别所述目标对象的动作，包括：获取所述多个关键点的可见性属性的组合值的参考值，其中所述参考值与特定动作对应；将所述可见性属性的组合值与所述参考值进行对比；如果所述组合值与所述参考值匹配，输出所述参考值所对应的特定动作为识别出的目标对象的动作。
进一步的，所述根据所述多个关键点的可见性属性的组合值识别所述目标对象的动作，包括：获取所述多个关键点的可见性属性的组合值的变化参考值，其中所述组合值的变化参考值与特定动作对应；获取连续两个图像帧中的所述多个关键点的可见性属性的两个组合值；通过所述两个组合值计算组合值的变化值；将所述组合值的变化值与所述组合值的变化参考值进行对比；如果所述组合值的变化值与所述组合值的变化参考值匹配，输出所述变化参考值所对应的特定动作为识别出的目标对象的动作。
进一步的，所述如果所述目标对象的动作是第一动作，则使用与所述第一动作对应的第一处理对所述原始图像进行处理，包括：响应于所述目 标对象的动作时第一动作，获取所述第一动作对应的第一处理及第一处理的参数；使用所述第一处理以及第一处理的参数对原始图像进行处理得到处理后的图像。
进一步的，所述第一处理为降低图像亮度，所述第一处理的参数为亮度降低的百分比，所述使用所述第一处理以及第一处理的参数对原始图像进行处理得到处理后的图像，包括：获取原始图像的每个像素在RGB颜色空间中的三通道的三个颜色分量；将所述每个像素的三个颜色分量分别减去所述百分比的颜色分量值得到处理后的每个像素的三个颜色分量值。
根据本公开的另一个方面，还提供以下技术方案：
一种目标对象的动作识别装置，包括：
原始图像获取模块，用于从图像源获取原始图像，所述原始图像中包括目标对象；目标对象识别模块，用于从所述原始图像中识别出所述目标对象；关键点检测模块，用于检测所述目标对象的多个关键点；可见性判断模块，用于通过检测到的关键点判断所述多个关键点的可见性属性，其中所述可见性属性用于表示所述关键点是否被遮挡；动作识别模块，用于根据所述多个关键点的可见性属性的组合值识别所述目标对象的动作。
进一步的，所述装置，还包括：第一动作判断模块，用于判断所述目标对象的动作是否为第一动作；第一处理模块，用于如果所述目标对象的动作是第一动作，则使用与所述第一动作对应的第一处理对所述原始图像进行处理。
进一步的，所述目标对象识别模块，还包括：图像帧获取模块，用于获取原始图像的当前图像帧；特征抽取模块，用于抽取所述图像帧中的图像特征形成特征图像；矩形框生成模块，用于在所述特征图像中生成多个矩形框；数量判断模块，用于判断每个矩形框中所包含的目标对象的图像特征的数量；外接框输出模块，用于将包含目标对象的图像特征数量最多的矩形框输出作为目标对象的外接框。
进一步的，所述关键点检测模块，还包括：第一输入模块，用于将所述识别出的目标对象输入关键点检测模型；第一输出模块，用于所述关键点检测模型输出每个关键点在所述原始图像上的坐标。
进一步的，所述可见性判断模块，还包括：第二输入模块，用于将检测出多个关键点的目标对象输入可见性判断模型；可见性概率判断模块，用于可见性判断模型输出所述多个关键点中的每一个的可见性概率；第二 输出模块，用于将所述可见性概率与第一阈值比较并输出所述可见性属性的值。
进一步的，所述动作识别模块，还包括：参考值获取模块，用于获取所述多个关键点的可见性属性的组合值的参考值，其中所述参考值与特定动作对应；参考值对比模块，用于将所述可见性属性的组合值与所述参考值进行对比；第一识别模块，用于如果所述组合值与所述参考值匹配，输出所述参考值所对应的特定动作为识别出的目标对象的动作。
进一步的，所述动作识别模块，还包括：变化参考值获取模块，用于获取所述多个关键点的可见性属性的组合值的变化参考值，其中所述组合值的变化参考值与特定动作对应；组合值获取模块，用于获取连续两个图像帧中的所述多个关键点的可见性属性的两个组合值；变化值计算模块，用于通过所述两个组合值计算组合值的变化值；变化参考值对比模块，用于将所述组合值的变化值与所述组合值的变化参考值进行对比；第二识别模块，用于如果所述组合值的变化值与所述组合值的变化参考值匹配，输出所述变化参考值所对应的特定动作为识别出的目标对象的动作。
进一步的，所述第一处理模块，还包括：第一处理及参数获取模块，用于响应于所述目标对象的动作时第一动作，获取所述第一动作对应的第一处理及第一处理的参数；第一处理子模块，用于使用所述第一处理以及第一处理的参数对原始图像进行处理得到处理后的图像。
进一步的，所述第一处理为降低图像亮度，所述第一处理的参数为亮度降低的百分比，所述第一处理子模块，包括：颜色获取模块，用于获取原始图像的每个像素在RGB颜色空间中的三通道的三个颜色分量；亮度处理模块，用于将所述每个像素的三个颜色分量分别减去所述百分比的颜色分量值得到处理后的每个像素的三个颜色分量值。
根据本公开的又一个方面，还提供以下技术方案：
一种电子设备，包括：存储器，用于存储非暂时性计算机可读指令；以及处理器，用于运行所述计算机可读指令，使得所述处理器执行时实现上述任一目标对象的动作识别方法所述的步骤。
根据本公开的又一个方面，还提供以下技术方案：
一种计算机可读存储介质，用于存储非暂时性计算机可读指令，当所述非暂时性计算机可读指令由计算机执行时，使得所述计算机执行上述任一方法中所述的步骤。
本公开公开了一种目标对象的动作识别方法、装置和电子设备。其中，所述目标对象的动作识别方法包括：从图像源获取原始图像，所述原始图像中包括目标对象；从所述原始图像中识别出所述目标对象；检测所述目标对象的多个关键点；通过检测到的关键点判断所述多个关键点的可见性属性，其中所述可见性属性用于表示所述关键点是否被遮挡；根据所述多个关键点的可见性属性的组合值识别所述目标对象的动作。本公开通关键点的可见性来判断目标对象的动作，解决了现有技术中的对复杂动作识别不准确的技术问题。
上述说明仅是本公开技术方案的概述，为了能更清楚了解本公开的技术手段，而可依照说明书的内容予以实施，并且为让本公开的上述和其他目的、特征和优点能够更明显易懂，以下特举较佳实施例,并配合附图，详细说明如下。
附图说明
图1为根据本公开一个实施例的目标对象的动作识别方法的流程示意图；
图2为根据本公开一个实施例的目标对象的动作识别方法的步骤S102进一步的流程示意图；
图3为根据本公开一个实施例的目标对象的动作识别方法的步骤S103进一步的流程示意图；
图4为根据本公开一个实施例的目标对象的动作识别方法的步骤S104进一步的流程示意图；
图5为根据本公开一个实施例的目标对象的动作识别方法的步骤S105进一步的流程示意图；
图6为根据本公开一个实施例的目标对象的动作识别方法的步骤S105进一步的流程示意图；
图7为根据本公开一个实施例的目标对象的动作识别装置的结构示意图；
图8为根据本公开实施例提供的电子设备的结构示意图。
具体实施方式
以下通过特定的具体实例说明本公开的实施方式，本领域技术人员可由本说明书所揭露的内容轻易地了解本公开的其他优点与功效。显然，所描述的实施例仅仅是本公开一部分实施例，而不是全部的实施例。本公开还可以通过另外不同的具体实施方式加以实施或应用，本说明书中的各项细节也可以基于不同观点与应用，在没有背离本公开的精神下进行各种修饰或改变。需说明的是，在不冲突的情况下，以下实施例及实施例中的特征可以相互组合。基于本公开中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本公开保护的范围。
需要说明的是，下文描述在所附权利要求书的范围内的实施例的各种方面。应显而易见，本文中所描述的方面可体现于广泛多种形式中，且本文中所描述的任何特定结构及/或功能仅为说明性的。基于本公开，所属领域的技术人员应了解，本文中所描述的一个方面可与任何其它方面独立地实施，且可以各种方式组合这些方面中的两者或两者以上。举例来说，可使用本文中所阐述的任何数目个方面来实施设备及/或实践方法。另外，可使用除了本文中所阐述的方面中的一或多者之外的其它结构及/或功能性实施此设备及/或实践此方法。
还需要说明的是，以下实施例中所提供的图示仅以示意方式说明本公开的基本构想，图式中仅显示与本公开中有关的组件而非按照实际实施时的组件数目、形状及尺寸绘制，其实际实施时各组件的型态、数量及比例可为一种随意的改变，且其组件布局型态也可能更为复杂。
另外，在以下描述中，提供具体细节是为了便于透彻理解实例。然而，所属领域的技术人员将理解，可在没有这些特定细节的情况下实践所述方面。
本公开实施例提供一种目标对象的动作识别方法。本实施例提供的所述目标对象的动作识别方法可以由一计算装置来执行，所述计算装置可以实现为软件，或者实现为软件和硬件的组合，所述计算装置可以集成设置在服务器、终端设备等中。如图1所示，所述目标对象的动作识别方法主要包括如下步骤S101至步骤S105。其中：
步骤S101：从图像源获取原始图像，所述原始图像中包括目标对象；
在本公开中，所述图像源为本地存储空间或者网络存储空间，所述从图像源获取原始图像，包括从本地存储空间中获取原始图像或者从网络存储空间中获取原始图像，无论从哪里获取原始图像，首选需要获取原始图像的存储地址，之后从该存储地址获取原始图像，所述原始图像包括多帧 图像，所述原始图像可以是视频也可以是带有动态效果的图片，只要是带有多帧的图像均可以是本公开中的原始图像。
在本公开中，所述图像源可以是图像传感器，所述从图像源获取原始图像，包括从图像传感器中采集原始图像。所述图像传感器指可以采集图像的各种设备，典型的图像传感器为摄像机、摄像头、相机等。在该实施例中，所述图像传感器可以是移动终端上的摄像头，比如智能手机上的前置或者后置摄像头，摄像头采集的原始图像可以直接显示在手机的显示屏上，在该步骤中，获取图像传感器所拍摄的视频，用于在下一步进一步识别图像中的目标对象。
在本公开中，所述原始图像中包括目标对象，所述目标对象可以是一个或者多个特定的对象，比如人手、人脸、人体、各种动物等等。
在一个典型的应用中，用户使用移动终端的摄像头拍摄自己的人体图像，移动终端将拍摄到的图像显示在移动终端的显示装置上，此时用户可以在摄像头前做出各种动作，移动终端通过摄像头采集到的人体图像中的人手姿态进行检测。对目标对象的识别和检测将在下边的几个步骤中描述，在此不再赘述。
步骤S102：从所述原始图像中识别出所述目标对象；
如图2所示，在本公开中，所述从所述原始图像中识别出所述目标对象，可以包括以下步骤：
步骤S201：获取原始图像的当前图像帧；
步骤S202：抽取所述图像帧中的图像特征形成特征图像；
步骤S203：在所述特征图像中生成多个矩形框；
步骤S204：判断每个矩形框中所包含的目标对象的图像特征的数量；
步骤S205：将包含目标对象的图像特征数量最多的矩形框输出作为目标对象的外接框。
其中在步骤S201中，获取原始图像的当前图像帧，图像帧实质上就是一张图片，如在一般的视频图像中，一秒钟的视频图像可以包括30帧，也就是每33ms获取一张图片，所述的当前图像帧就是当前时刻视频图像播放时间点上的图片。
在步骤S202中，从所述图像帧中抽取出图像特征形成特征图像，其中抽取图像特征可以使用卷积的方式，使用实现设置好的卷积核对图像帧上 的像素进行提取和计算，形成特征图像，如一个10*10的视频帧通过3*3的卷积核以步长为1进行特征提取，可以生成一个8*8的特征图像。可以理解的，可以对图像帧进行多层卷积核进行卷积，以生成更小的特征图像，也可以使用多个卷积核生成不同的特征图像，也可以将两种方式结合起来生生更小更多的特征图像，最终可以将这些特征图像融合在一起，生成一张特征图像。可以理解的是，特征图像的生成方法可以使用上述任一种或多种的组合，在此不做限制。
在步骤S203中，在所述特征图像中生成多个矩形框，用来框取所述目标对象的特征。具体的，可以将特征图像划分为多个矩形区域，每个矩形区域包括一个中心点，生成多个预设大小的矩形框来围绕所述中心点，其中所述矩形框的中心点与所述矩形区域的中心点重合，由此可以生成多个围绕所述矩形区域或者被所述矩形区域围绕的矩形。
在步骤S204中，判断在步骤S203中所生成的矩形中，包含目标对象的图像特征的数量。具体的，该数量可以使用该矩形中包括所目标对象的置信度来表示，置信度越高，该矩形中包括目标对象的图像特征的数量越多。
在步骤S205中，输出包括目标对象的图像特征的数量最多的矩形框作为目标对象的外接框。在步骤S204中，有可能包含很多置信度类似的矩形框，在步骤S205中可以以置信度排序，如置信度从1到0.5的矩形框，按置信度从大到小排序，舍弃置信度低于0.5的矩形框。之后按照顺序依次判断矩形框的重合程度，将重合程度大的矩形框去重，最终得到唯一的置信度最大的矩形框输出为目标对象的外接框。
可以理解的是，识别目标对象的方法有很多，对于不同的目标对象还可以进行进一步的优化，这不在本公开的讨论范围内，不再赘述，只要是可以从原始图像中识别出目标对象的方法，均可以应用于本公开中。
步骤S103：检测所述目标对象的多个关键点；
如图3所示，在本公开中，所述检测所述目标对象的多个关键点，可以包括：
步骤S301：将所述识别出的目标对象输入关键点检测模型；
步骤S302：所述关键点检测模型输出每个关键点在所述原始图像上的坐标。
在上述步骤中，所述关键点检测模型为预选训练的用于检测关键点的 模型，训练所述关键点检测模型需要使用多个标注了关键点的目标对象的图像作为训练图集，典型的，所述目标对象为人手，所述关键点为编号0-21的22个关键点，此时将图片中的人手上的22个关键点标注出来作为训练图集并输入模型中，将模型的预测关键点的坐标与训练图集上关键点的坐标进行对比，并将这两者的误差反馈给关键点检测模型，关键点检测模型根据误差调整模型参数之后继续使用训练图集中图片进行预测，直至对训练图集中的每个图片都能准确预测出与标注的关键点相同的坐标为止。将步骤S102中所识别出来的目标对象输入上述训练好的关键点检测模型，所述关键点检测模型输出每个编号的关键点在原始图像上的坐标。为了加快检测的速度，此处输入的目标对象可以是步骤S102中的外接框内的目标对象图像，这样图像较小，识别的速度较快，由于外接框的大小和中心点在输出时为已知的数值，这样可以根据外接框的位置和大小计算出关键点相对于原始图像的坐标，计算方法在此不再赘述。
步骤S104：通过检测到的关键点判断所述多个关键点的可见性属性，其中所述可见性属性用于表示所述关键点是否被遮挡；
如图4所示，在本公开中，所述通过检测到的关键点判断所述多个关键点的可见性属性，可以包括以下步骤：
步骤S401：将检测出多个关键点的目标对象输入可见性判断模型；
步骤S402：可见性判断模型输出所述多个关键点中的每一个的可见性概率；
步骤S403：将所述可见性概率与第一阈值比较并输出所述可见性属性的值。
在上述步骤中，需要使用可见性判断模型，所述模型使用标注训练集合来训练，所述标注训练集合中包括了所述目标对象在各种角度、各种姿态下的图片，并且标注了目标对象中的每个关键点是否可见，即是否被遮挡。将所述标注训练集合中的图片输入所述可见性判断模型中，并将模型的输出与图片的标注进行比较，所述模型的输出为关键点可见的概率，如果概率高于一个阈值可以确定与标注相符，如果不同，则计算损失函数并将误差反馈给模型用以调整模型参数之后再次将训练集合输入并输出结果，反复调整参数直至输出结果与标注相符，模型训练完成。
使用上述可见性判断模型时，将检测出多个关键点的目标对象输入可见性判断模型，之后可见性判断模型输出所述多个关键点中的每一个的可见性概率，将该可见性概率与一预设的第一阈值进行比较，该阈值可以为 0.8，即当输出的概率大于0.8时，认为该关键点可见，当输出的概率低于0.8时，认为该关键点不可见。之后将该概率通过激活函数输出，将关键点可见的值输出为1，将关键点不可见的值输出为0，这样得到一个可见性属性的值的向量，该向量为一个1*N的向量，其中N为关键点的个数，该向量中的元素的值为0或1，分别表示对应的关键点不可见和可见。
步骤S105：根据所述多个关键点的可见性属性的组合值识别所述目标对象的动作。
如图5所示，所述根据所述多个关键点的可见性属性的组合值识别所述目标对象的动作，可以包括如下步骤：
步骤S501：获取所述多个关键点的可见性属性的组合值的参考值，其中所述参考值与特定动作对应；
步骤S502：将所述可见性属性的组合值与所述参考值进行对比；
步骤S503：如果所述组合值与所述参考值匹配，输出所述参考值所对应的特定动作为识别出的目标对象的动作。
在上述步骤中，可以判断一个目标对象的静态动作，如一个姿势，具体的，当目标对象为人手时，可以识别出人手的一个手势。在步骤S104中，输出的可见性属性的值可以是可见性属性的组合值，也就是一个向量值。可以在配置文件中预设一个或多个目标对象的动作的可见性属性的组合值的参考值，其也可以是一个向量，如1*N的向量，在得到可见性属性的组合值之后从所述配置文件中获取所述一个或多个可见性属性的组合值的参考值，之后比较所述可见性属性的组合值与所述参考值，在本公开中，所述比较可以是比较两个向量的相似度，常用的方法可以使用皮尔逊相关系数、欧几里得距离、余弦相似度、曼哈顿距离等等判断两个向量的相似度，如果相似度高于一个预设的第二阈值，则判断所述组合值与所述参考值匹配，则输出所述参考值所对应的特定动作为识别出的目标对象的动作。至此，所述目标对象的动作就被识别出来。
如图6所示，所述根据所述多个关键点的可见性属性的组合值识别所述目标对象的动作，还可以包括如下步骤：
步骤S601：获取所述多个关键点的可见性属性的组合值的变化参考值，其中所述组合值的变化参考值与特定动作对应；
步骤S602：获取连续两个图像帧中的所述多个关键点的可见性属性的两个组合值；
步骤S603：通过所述两个组合值计算组合值的变化值；
步骤S604：将所述组合值的变化值与所述组合值的变化参考值进行对比；
步骤S605：如果所述组合值的变化值与所述组合值的变化参考值匹配，输出所述变化参考值所对应的特定动作为识别出的目标对象的动作。
在上述步骤中，可以判断一个目标对象的动态的动作，如一个变化的动作，具体的，当目标对象为人手时，可以识别出人手的一个打响指的动作，其中打响指的动作至少需要两个手的状态才能识别。在步骤S104中，输出的可见性属性的值可以是可见性属性的组合值，也就是一个向量值。可以在配置文件中预设一个或多个目标对象的动作的可见性属性的组合值的变化参考值，其也可以是两个向量，如两个1*N的向量，分别表示目标对象在两个状态下的可见性属性的组合值的参考值。在上述步骤中，需要获取连续两个图像帧中的关键点的可见性属性的组合值，并将两个可见性属性的组合值组合成一个变化值，所述变化值可以是两个组合值首尾相连组成的一个新的向量，如两个1*N的向量组合成的1*2N的向量。此时，获取到的可见性属性的组合值的变化参考值也可以是两个1*N向量的组合向量。之后，比较组合值的变化值与所述组合值的变化参考值的相似度判断两者是否匹配，计算相似度的方法与上边相同。如果相似度高于一个预设的第三阈值，则判断所述组合值的变化值与所述变化参考值匹配，则输出所述变化参考值所对应的特定动作为识别出的目标对象的动作。至此，所述目标对象的动作就被识别出来。
可以理解的是，计算两个向量的相似度的方法有很多，本公开并不局限于上述方法，实际上根据实际需要还可以设置每个关键点在计算相似度时的权重等等，可以使用任何其他算法计算两个向量的相似度，甚至可以使用严格匹配的方式，本公开对此不做任何限制，在此也不再赘述。
进一步的，在步骤S105之后，还可以包括步骤：
判断所述目标对象的动作是否为第一动作；
如果所述目标对象的动作是第一动作，则使用与所述第一动作对应的第一处理对所述原始图像进行处理。
具体的，所述第一处理为降低图像亮度，所述第一处理的参数为亮度降低的百分比，所述使用所述第一处理以及第一处理的参数对原始图像进行处理得到处理后的图像，包括：
获取原始图像的每个像素在RGB颜色空间中的三通道的三个颜色分量；
将所述每个像素的三个颜色分量分别减去所述百分比的颜色分量值得到处理后的每个像素的三个颜色分量值。
上述步骤是对本公开目标对象的动作识别的一个具体的应用，其使用识别出的目标对象的动作触发一个图像处理操作，其图像处理操作的结果是将图像变暗。具体的，所述目标对象可以为人手，所述动作可以是打响指，第一处理为降低图像亮度，所述第一处理参数为50％，则当用于使用移动终端的摄像头拍摄视频，当视频中出现人手，并且人手被识别出做了打响指的动作，则移动终端中的屏幕中的视频的亮度被下降50％，以实现一种打响指关灯的效果。
可以理解的，基于本公开的目标对象的动作的识别方法可以实现任何特效的触发，上述打响指关灯的效果仅仅是举例，触发其他效果的方案不再赘述。
本公开公开了一种目标对象的动作识别方法、装置和电子设备。其中，所述目标对象的动作识别方法包括：从图像源获取原始图像，所述原始图像中包括目标对象；从所述原始图像中识别出所述目标对象；检测所述目标对象的多个关键点；通过检测到的关键点判断所述多个关键点的可见性属性，其中所述可见性属性用于表示所述关键点是否被遮挡；根据所述多个关键点的可见性属性的组合值识别所述目标对象的动作。本公开通关键点的可见性来判断目标对象的动作，解决了现有技术中的对复杂动作识别不准确的技术问题。
在上文中，虽然按照上述的顺序描述了上述方法实施例中的各个步骤，本领域技术人员应清楚，本公开实施例中的步骤并不必然按照上述顺序执行，其也可以倒序、并行、交叉等其他顺序执行，而且，在上述步骤的基础上，本领域技术人员也可以再加入其他步骤，这些明显变型或等同替换的方式也应包含在本公开的保护范围之内，在此不再赘述。
下面为本公开装置实施例，本公开装置实施例可用于执行本公开方法实施例实现的步骤，为了便于说明，仅示出了与本公开实施例相关的部分，具体技术细节未揭示的，请参照本公开方法实施例。
本公开实施例提供一种图像的处理装置。所述装置可以执行上述目标对象的动作识别方法实施例中所述的步骤。如图7所示，所述装置700主要包括：原始图像获取模块701、目标对象识别模块702、关键点检测模块703、可见性判断模块704和动作识别模块705。其中，
原始图像获取模块701，用于从图像源获取原始图像，所述原始图像中包括目标对象；
目标对象识别模块702，用于从所述原始图像中识别出所述目标对象；
关键点检测模块703，用于检测所述目标对象的多个关键点；
可见性判断模块704，用于通过检测到的关键点判断所述多个关键点的可见性属性，其中所述可见性属性用于表示所述关键点是否被遮挡；
动作识别模块705，用于根据所述多个关键点的可见性属性的组合值识别所述目标对象的动作。
进一步的，所述装置700，还包括：
第一动作判断模块，用于判断所述目标对象的动作是否为第一动作；
第一处理模块，用于如果所述目标对象的动作是第一动作，则使用与所述第一动作对应的第一处理对所述原始图像进行处理。
进一步的，所述目标对象识别模块702，还包括：
图像帧获取模块，用于获取原始图像的当前图像帧；
特征抽取模块，用于抽取所述图像帧中的图像特征形成特征图像；
矩形框生成模块，用于在所述特征图像中生成多个矩形框；
数量判断模块，用于判断每个矩形框中所包含的目标对象的图像特征的数量；
外接框输出模块，用于将包含目标对象的图像特征数量最多的矩形框输出作为目标对象的外接框。
进一步的，所述关键点检测模块703，还包括：
第一输入模块，用于将所述识别出的目标对象输入关键点检测模型；
第一输出模块，用于所述关键点检测模型输出每个关键点在所述原始图像上的坐标。
进一步的，所述可见性判断模块704，还包括：
第二输入模块，用于将检测出多个关键点的目标对象输入可见性判断模型；
可见性概率判断模块，用于可见性判断模型输出所述多个关键点中的每一个的可见性概率；
第二输出模块，用于将所述可见性概率与第一阈值比较并输出所述可见性属性的值。
进一步的，所述动作识别模块705，还包括：
参考值获取模块，用于获取所述多个关键点的可见性属性的组合值的参考值，其中所述参考值与特定动作对应；
参考值对比模块，用于将所述可见性属性的组合值与所述参考值进行对比；
第一识别模块，用于如果所述组合值与所述参考值匹配，输出所述参考值所对应的特定动作为识别出的目标对象的动作。
进一步的，所述动作识别模块705，还包括：
变化参考值获取模块，用于获取所述多个关键点的可见性属性的组合值的变化参考值，其中所述组合值的变化参考值与特定动作对应；
组合值获取模块，用于获取连续两个图像帧中的所述多个关键点的可见性属性的两个组合值；
变化值计算模块，用于通过所述两个组合值计算组合值的变化值；
变化参考值对比模块，用于将所述组合值的变化值与所述组合值的变化参考值进行对比；
第二识别模块，用于如果所述组合值的变化值与所述组合值的变化参考值匹配，输出所述变化参考值所对应的特定动作为识别出的目标对象的动作。
进一步的，所述第一处理模块，还包括：
第一处理及参数获取模块，用于响应于所述目标对象的动作时第一动作，获取所述第一动作对应的第一处理及第一处理的参数；
第一处理子模块，用于使用所述第一处理以及第一处理的参数对原始图像进行处理得到处理后的图像。
进一步的，所述第一处理为降低图像亮度，所述第一处理的参数为亮度降低的百分比，所述第一处理子模块，包括：
颜色获取模块，用于获取原始图像的每个像素在RGB颜色空间中的三通道的三个颜色分量；
亮度处理模块，用于将所述每个像素的三个颜色分量分别减去所述百 分比的颜色分量值得到处理后的每个像素的三个颜色分量值。
图7所示装置可以执行图1-图6所示实施例的方法，本实施例未详细描述的部分，可参考对图1-图6所示实施例的相关说明。所述技术方案的执行过程和技术效果参见图1-图6所示实施例中的描述，在此不再赘述。
下面参考图8，其示出了适于用来实现本公开实施例的电子设备800的结构示意图。本公开实施例中的电子设备可以包括但不限于诸如移动电话、笔记本电脑、数字广播接收器、PDA(个人数字助理)、PAD(平板电脑)、PMP(便携式多媒体播放器)、车载终端(例如车载导航终端)等等的移动终端以及诸如数字TV、台式计算机等等的固定终端。图8示出的电子设备仅仅是一个示例，不应对本公开实施例的功能和使用范围带来任何限制。
如图8所示，电子设备800可以包括处理装置(例如中央处理器、图形处理器等)801，其可以根据存储在只读存储器(ROM)802中的程序或者从存储装置808加载到随机访问存储器(RAM)803中的程序而执行各种适当的动作和处理。在RAM 803中，还存储有电子设备800操作所需的各种程序和数据。处理装置801、ROM 802以及RAM 803通过总线804彼此相连。输入/输出(I/O)接口805也连接至总线804。
通常，以下装置可以连接至I/O接口805：包括例如触摸屏、触摸板、键盘、鼠标、图像传感器、麦克风、加速度计、陀螺仪等的输入装置806；包括例如液晶显示器(LCD)、扬声器、振动器等的输出装置807；包括例如磁带、硬盘等的存储装置808；以及通信装置809。通信装置809可以允许电子设备800与其他设备进行无线或有线通信以交换数据。虽然图8示出了具有各种装置的电子设备800，但是应理解的是，并不要求实施或具备所有示出的装置。可以替代地实施或具备更多或更少的装置。
特别地，根据本公开的实施例，上文参考流程图描述的过程可以被实现为计算机软件程序。例如，本公开的实施例包括一种计算机程序产品，其包括承载在计算机可读介质上的计算机程序，所述计算机程序包含用于执行流程图所示的方法的程序代码。在这样的实施例中，所述计算机程序可以通过通信装置809从网络上被下载和安装，或者从存储装置808被安装，或者从ROM 802被安装。在所述计算机程序被处理装置801执行时，执行本公开实施例的方法中限定的上述功能。
需要说明的是，本公开上述的计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质或者是上述两者的任意组合。计算机可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导 体的系统、装置或器件，或者任意以上的组合。计算机可读存储介质的更具体的例子可以包括但不限于：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机访问存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑磁盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本公开中，计算机可读存储介质可以是任何包含或存储程序的有形介质，所述程序可以被指令执行系统、装置或者器件使用或者与其结合使用。而在本公开中，计算机可读信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式，包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读信号介质还可以是计算机可读存储介质以外的任何计算机可读介质，所述计算机可读信号介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。计算机可读介质上包含的程序代码可以用任何适当的介质传输，包括但不限于：电线、光缆、RF(射频)等等，或者上述的任意合适的组合。
上述计算机可读介质可以是上述电子设备中所包含的；也可以是单独存在，而未装配入所述电子设备中。
上述计算机可读介质承载有一个或者多个程序，当上述一个或者多个程序被所述电子设备执行时，使得所述电子设备：从图像源获取原始图像，所述原始图像中包括目标对象；从所述原始图像中识别出所述目标对象；检测所述目标对象的多个关键点；通过检测到的关键点判断所述多个关键点的可见性属性，其中所述可见性属性用于表示所述关键点是否被遮挡；根据所述多个关键点的可见性属性的组合值识别所述目标对象的动作。
可以以一种或多种程序设计语言或其组合来编写用于执行本公开的操作的计算机程序代码，上述程序设计语言包括面向对象的程序设计语言—诸如Java、Smalltalk、C++，还包括常规的过程式程序设计语言—诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络——包括局域网(LAN)或广域网(WAN)—连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。
附图中的流程图和框图，图示了按照本公开各种实施例的系统、方法 和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分，所述模块、程序段、或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意，在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个接连地表示的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或操作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
描述于本公开实施例中所涉及到的单元可以通过软件的方式实现，也可以通过硬件的方式来实现。其中，单元的名称在某种情况下并不构成对所述单元本身的限定。
以上描述仅为本公开的较佳实施例以及对所运用技术原理的说明。本领域技术人员应当理解，本公开中所涉及的公开范围，并不限于上述技术特征的特定组合而成的技术方案，同时也应涵盖在不脱离上述公开构思的情况下，由上述技术特征或其等同特征进行任意组合而形成的其它技术方案。例如上述特征与本公开中公开的(但不限于)具有类似功能的技术特征进行互相替换而形成的技术方案。

Claims (12)

  1. 一种目标对象的动作识别方法，包括：
    从图像源获取原始图像，所述原始图像中包括目标对象；
    从所述原始图像中识别出所述目标对象；
    检测所述目标对象的多个关键点；
    通过检测到的关键点判断所述多个关键点的可见性属性，其中所述可见性属性用于表示所述关键点是否被遮挡；
    根据所述多个关键点的可见性属性的组合值识别所述目标对象的动作。
  2. 如权利要求1所述的目标对象的动作识别方法，其中在所述根据所述多个关键点的可见性属性的组合值识别所述目标对象的动作之后，还包括：
    判断所述目标对象的动作是否为第一动作；
    如果所述目标对象的动作是第一动作，则使用与所述第一动作对应的第一处理对所述原始图像进行处理。
  3. 如权利要求1所述的目标对象的动作识别方法，其中所述从所述原始图像中识别出所述目标对象，包括：
    获取原始图像的当前图像帧；
    抽取所述图像帧中的图像特征形成特征图像；
    在所述特征图像中生成多个矩形框；
    判断每个矩形框中所包含的目标对象的图像特征的数量；
    将包含目标对象的图像特征数量最多的矩形框输出作为目标对象的外接框。
  4. 如权利要求1所述的目标对象的动作识别方法，其中所述检测所述目标对象的多个关键点，包括：
    将所述识别出的目标对象输入关键点检测模型；
    所述关键点检测模型输出每个关键点在所述原始图像上的坐标。
  5. 如权利要求1所述的目标对象的动作识别方法，其中所述通过检测到的关键点判断所述多个关键点的可见性属性，其中所述可见性属性用于 表示所述关键点是否被遮挡，包括：
    将检测出多个关键点的目标对象输入可见性判断模型；
    可见性判断模型输出所述多个关键点中的每一个的可见性概率；
    将所述可见性概率与第一阈值比较并输出所述可见性属性的值。
  6. 如权利要求1所述的目标对象的动作识别方法，其中所述根据所述多个关键点的可见性属性的组合值识别所述目标对象的动作，包括：
    获取所述多个关键点的可见性属性的组合值的参考值，其中所述参考值与特定动作对应；
    将所述可见性属性的组合值与所述参考值进行对比；
    如果所述组合值与所述参考值匹配，输出所述参考值所对应的特定动作为识别出的目标对象的动作。
  7. 如权利要求1所述的目标对象的动作识别方法，其中所述根据所述多个关键点的可见性属性的组合值识别所述目标对象的动作，包括：
    获取所述多个关键点的可见性属性的组合值的变化参考值，其中所述组合值的变化参考值与特定动作对应；
    获取连续两个图像帧中的所述多个关键点的可见性属性的两个组合值；
    通过所述两个组合值计算组合值的变化值；
    将所述组合值的变化值与所述组合值的变化参考值进行对比；
    如果所述组合值的变化值与所述组合值的变化参考值匹配，输出所述变化参考值所对应的特定动作为识别出的目标对象的动作。
  8. 如权利要求2所述的目标对象的动作识别方法，其中所述如果所述目标对象的动作是第一动作，则使用与所述第一动作对应的第一处理对所述原始图像进行处理，包括：
    响应于所述目标对象的动作时第一动作，获取所述第一动作对应的第一处理及第一处理的参数；
    使用所述第一处理以及第一处理的参数对原始图像进行处理得到处理后的图像。
  9. 如权利要求8所述的目标对象的动作识别方法，其中所述第一处理为降低图像亮度，所述第一处理的参数为亮度降低的百分比，所述使用所述第一处理以及第一处理的参数对原始图像进行处理得到处理后的图像， 包括：
    获取原始图像的每个像素在RGB颜色空间中的三通道的三个颜色分量；
    将所述每个像素的三个颜色分量分别减去所述百分比的颜色分量值得到处理后的每个像素的三个颜色分量值。
  10. 一种目标对象的动作识别装置，包括：
    原始图像获取模块，用于从图像源获取原始图像，所述原始图像中包括目标对象；
    目标对象识别模块，用于从所述原始图像中识别出所述目标对象；
    关键点检测模块，用于检测所述目标对象的多个关键点；
    可见性判断模块，用于通过检测到的关键点判断所述多个关键点的可见性属性，其中所述可见性属性用于表示所述关键点是否被遮挡；
    动作识别模块，用于根据所述多个关键点的可见性属性的组合值识别所述目标对象的动作。
  11. 一种电子设备，包括：
    存储器，用于存储计算机可读指令；以及
    处理器，用于运行所述计算机可读指令，使得所述处理器运行时实现根据权利要求1-9中任意一项所述的基于目标对象姿态的图像处理方法。
  12. 一种非暂态计算机可读存储介质，用于存储计算机可读指令，当所述计算机可读指令由计算机执行时，使得所述计算机执行权利要求1-9中任意一项所述的基于目标对象姿态的图像处理方法。
PCT/CN2020/076742 2019-04-29 2020-02-26 目标对象的动作识别方法、装置和电子设备 Ceased WO2020220809A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
JP2021503605A JP7181375B2 (ja) 2019-04-29 2020-02-26 目標対象の動作認識方法、装置及び電子機器
GB2102752.9A GB2598015B (en) 2019-04-29 2020-02-26 Action recognition method and device for target object, and electronic apparatus
US17/260,072 US12008167B2 (en) 2019-04-29 2020-02-26 Action recognition method and device for target object, and electronic apparatus

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910353500.6 2019-04-29
CN201910353500.6A CN110070063B (zh) 2019-04-29 2019-04-29 目标对象的动作识别方法、装置和电子设备

Publications (1)

Publication Number Publication Date
WO2020220809A1 true WO2020220809A1 (zh) 2020-11-05

Family

ID=67369495

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/076742 Ceased WO2020220809A1 (zh) 2019-04-29 2020-02-26 目标对象的动作识别方法、装置和电子设备

Country Status (5)

Country Link
US (1) US12008167B2 (zh)
JP (1) JP7181375B2 (zh)
CN (1) CN110070063B (zh)
GB (1) GB2598015B (zh)
WO (1) WO2020220809A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114387302A (zh) * 2021-12-28 2022-04-22 西安电子科技大学 图像目标物体的弧形表达方法、装置和设备
CN115359132A (zh) * 2022-10-21 2022-11-18 小米汽车科技有限公司 用于车辆的相机标定方法、装置、电子设备及存储介质

Families Citing this family (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110070063B (zh) * 2019-04-29 2020-06-30 北京字节跳动网络技术有限公司 目标对象的动作识别方法、装置和电子设备
CN113225613B (zh) * 2020-01-21 2022-07-08 北京达佳互联信息技术有限公司 图像识别、视频直播方法和装置
CN112232194A (zh) * 2020-10-15 2021-01-15 广州云从凯风科技有限公司 一种单目标人体关键点检测方法、系统、设备及介质
CN113821103B (zh) * 2021-08-20 2023-12-05 苏州创捷传媒展览股份有限公司 一种屏上物体识别展示系统
CN113496512B (zh) * 2021-09-06 2021-12-17 北京字节跳动网络技术有限公司 用于内窥镜的组织腔体定位方法、装置、介质及设备
CN115790619A (zh) * 2021-09-10 2023-03-14 罗伯特·博世有限公司 用于基于传感器可视性处理对象检测结果的处理系统、单元和方法
WO2023079783A1 (ja) * 2021-11-05 2023-05-11 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ 行動認識方法、行動認識装置、及び行動認識プログラム
CN115272745A (zh) * 2022-06-23 2022-11-01 浙江大华技术股份有限公司 一种属性识别方法、设备和存储介质
CN117746273A (zh) * 2022-09-15 2024-03-22 北京字跳网络技术有限公司 数据处理方法、装置、设备、存储介质及计算机程序产品
CN118037295A (zh) * 2022-11-02 2024-05-14 腾讯科技（深圳）有限公司 生物支付处理方法、装置、电子设备及存储介质
CN118864315A (zh) * 2023-04-28 2024-10-29 北京字跳网络技术有限公司 图像修复方法、装置、电子设备及存储介质
WO2025046697A1 (ja) * 2023-08-28 2025-03-06 日本電気株式会社 判定システム、判定装置及び判定方法
CN120550406B (zh) * 2025-07-31 2026-01-30 赛力斯汽车有限公司 游戏交互控制方法、装置、计算机设备和存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120170804A1 (en) * 2010-12-31 2012-07-05 Industrial Technology Research Institute Method and apparatus for tracking target object
CN108986169A (zh) * 2018-07-06 2018-12-11 北京字节跳动网络技术有限公司 用于处理图像的方法和装置
CN109063653A (zh) * 2018-08-07 2018-12-21 北京字节跳动网络技术有限公司 图像处理方法和装置
CN109492581A (zh) * 2018-11-09 2019-03-19 中国石油大学(华东) 一种基于tp-stg框架的人体动作识别方法
CN110070063A (zh) * 2019-04-29 2019-07-30 北京字节跳动网络技术有限公司 目标对象的动作识别方法、装置和电子设备

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7990421B2 (en) * 2008-07-18 2011-08-02 Sony Ericsson Mobile Communications Ab Arrangement and method relating to an image recording device
KR101038323B1 (ko) * 2009-09-24 2011-06-01 주식회사 팬택 영상인식기법을 이용한 화면 프레임 제어장치
JP6417702B2 (ja) * 2014-05-01 2018-11-07 富士通株式会社 画像処理装置、画像処理方法および画像処理プログラム
US10599920B2 (en) * 2017-02-01 2020-03-24 Epilog Imaging Systems Automated digital magnifier system with hand gesture controls
CN107357427A (zh) * 2017-07-03 2017-11-17 南京江南博睿高新技术研究院有限公司 一种用于虚拟现实设备的手势识别控制方法
EP3467707B1 (en) 2017-10-07 2024-03-13 Tata Consultancy Services Limited System and method for deep learning based hand gesture recognition in first person view
US10296102B1 (en) * 2018-01-31 2019-05-21 Piccolo Labs Inc. Gesture and motion recognition using skeleton tracking
CN110163048B (zh) * 2018-07-10 2023-06-02 腾讯科技（深圳）有限公司 手部关键点的识别模型训练方法、识别方法及设备

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120170804A1 (en) * 2010-12-31 2012-07-05 Industrial Technology Research Institute Method and apparatus for tracking target object
CN108986169A (zh) * 2018-07-06 2018-12-11 北京字节跳动网络技术有限公司 用于处理图像的方法和装置
CN109063653A (zh) * 2018-08-07 2018-12-21 北京字节跳动网络技术有限公司 图像处理方法和装置
CN109492581A (zh) * 2018-11-09 2019-03-19 中国石油大学(华东) 一种基于tp-stg框架的人体动作识别方法
CN110070063A (zh) * 2019-04-29 2019-07-30 北京字节跳动网络技术有限公司 目标对象的动作识别方法、装置和电子设备

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114387302A (zh) * 2021-12-28 2022-04-22 西安电子科技大学 图像目标物体的弧形表达方法、装置和设备
CN115359132A (zh) * 2022-10-21 2022-11-18 小米汽车科技有限公司 用于车辆的相机标定方法、装置、电子设备及存储介质
CN115359132B (zh) * 2022-10-21 2023-03-24 小米汽车科技有限公司 用于车辆的相机标定方法、装置、电子设备及存储介质

Also Published As

Publication number Publication date
US20210281744A1 (en) 2021-09-09
CN110070063A (zh) 2019-07-30
JP7181375B2 (ja) 2022-11-30
JP2021531589A (ja) 2021-11-18
GB2598015A (en) 2022-02-16
GB2598015B (en) 2023-07-05
GB202102752D0 (en) 2021-04-14
US12008167B2 (en) 2024-06-11
CN110070063B (zh) 2020-06-30

Similar Documents

Publication Publication Date Title
JP7181375B2 (ja) 目標対象の動作認識方法、装置及び電子機器
JP7238141B2 (ja) 顔と手を関連付けて検出する方法及び装置、電子機器、記憶媒体及びコンピュータプログラム
CN112101305B (zh) 多路图像处理方法、装置及电子设备
US9727156B2 (en) Method for recognizing biometrics information and electronic device thereof
JP7262659B2 (ja) 目標対象物マッチング方法及び装置、電子機器並びに記憶媒体
CN106651955B (zh) 图片中目标物的定位方法及装置
CN110084204B (zh) 基于目标对象姿态的图像处理方法、装置和电子设备
US20150149925A1 (en) Emoticon generation using user images and gestures
WO2020186935A1 (zh) 虚拟对象的显示方法、装置、电子设备和计算机可读存储介质
CN110796664B (zh) 图像处理方法、装置、电子设备及计算机可读存储介质
CN110070551B (zh) 视频图像的渲染方法、装置和电子设备
US20140232748A1 (en) Device, method and computer readable recording medium for operating the same
US11924520B2 (en) Subtitle border-crossing processing method and apparatus, and electronic device
CN111314626A (zh) 用于处理视频的方法和装置
CN108960213A (zh) 目标跟踪方法、装置、存储介质及终端
CN107133361B (zh) 手势识别方法、装置和终端设备
WO2023155393A1 (zh) 特征点匹配方法、装置、电子设备、存储介质和计算机程序产品
WO2020156058A1 (zh) 基于终端的图像拖拽变形实现方法和装置
WO2022095318A1 (zh) 字符检测方法、装置、电子设备、存储介质及程序
WO2020215854A1 (zh) 渲染图像的方法、装置、电子设备和计算机可读存储介质
CN110222576B (zh) 拳击动作识别方法、装置和电子设备
JP7560207B2 (ja) オブジェクトの表示方法、装置、電子機器及びコンピュータ可読型記憶媒体
CN112085733B (zh) 图像处理方法、装置、电子设备和计算机可读介质
CN109040588A (zh) 人脸图像的拍照方法、装置、存储介质及终端
CN113963000B (zh) 图像分割方法、装置、电子设备及程序产品

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20798833

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021503605

Country of ref document: JP

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 202102752

Country of ref document: GB

Kind code of ref document: A

Free format text: PCT FILING DATE = 20200226

WWE Wipo information: entry into national phase

Ref document number: 2102752.9

Country of ref document: GB

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 18.02.2022)

122 Ep: pct application non-entry in european phase

Ref document number: 20798833

Country of ref document: EP

Kind code of ref document: A1

WWG Wipo information: grant in national office

Ref document number: 2102752.9

Country of ref document: GB

WWG Wipo information: grant in national office

Ref document number: 202117002689

Country of ref document: IN