WO2024188293A1 - 模型构建、对象识别方法、装置、设备、介质及产品 - Google Patents

模型构建、对象识别方法、装置、设备、介质及产品 Download PDF

Info

Publication number
WO2024188293A1
WO2024188293A1 PCT/CN2024/081599 CN2024081599W WO2024188293A1 WO 2024188293 A1 WO2024188293 A1 WO 2024188293A1 CN 2024081599 W CN2024081599 W CN 2024081599W WO 2024188293 A1 WO2024188293 A1 WO 2024188293A1
Authority
WO
WIPO (PCT)
Prior art keywords
limb
network
recognition
object recognition
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2024/081599
Other languages
English (en)
French (fr)
Inventor
董登科
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Zitiao Network Technology Co Ltd
Original Assignee
Beijing Zitiao Network Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Zitiao Network Technology Co Ltd filed Critical Beijing Zitiao Network Technology Co Ltd
Publication of WO2024188293A1 publication Critical patent/WO2024188293A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/107—Static hand or arm
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition

Definitions

  • the embodiments of the present disclosure relate to the field of image processing technology, and in particular to a model building, object recognition method, device, equipment, medium and product.
  • Image-based gesture recognition refers to identifying the hand movements made by people in an image. By identifying hand movements, gesture control can be performed to improve interaction efficiency.
  • hand movement recognition requires first identifying the area where the hand is located in the image, also known as the hand frame, then extracting the local image corresponding to the hand frame, and then recognizing the gesture from the local image.
  • the method of identifying hand frames in images mainly uses a large number of training images, including images and labels of hand frames in images, to train the image recognition network model.
  • an embodiment of the present disclosure provides a model building method, comprising:
  • the limb recognition model includes a limb feature extraction network with known parameters, and the limb recognition model is used to recognize limbs;
  • a target recognition model is constructed, and the target recognition model is used to perform recognition of local parts of the limbs in the target image.
  • an object recognition method including:
  • the target recognition model includes a limb feature extraction network with known parameters and an object recognition network obtained through training; the object recognized by the object recognition network is a local part of a limb;
  • an object recognition result corresponding to the image limb feature of the image to be recognized is determined, and the target recognition model is obtained by adopting the model construction method described in the first aspect.
  • an embodiment of the present disclosure provides a model building device, including:
  • a first acquisition unit is used to acquire a limb recognition model, wherein the limb recognition model includes a limb feature extraction network with known parameters, and the limb recognition model is used to recognize limbs;
  • a feature extraction unit used for inputting a training image into the limb feature extraction network to obtain image limb features
  • a model training unit used to determine an object recognition result of a training image through an object recognition network based on the image limb features, and update the object recognition network based on the object recognition result, wherein the object recognized by the object recognition network is a local part of a limb;
  • the model building unit is used to build a target recognition model based on the limb feature extraction network and the updated object recognition network, and the target recognition model is used to perform recognition of local parts of the limbs in the target image.
  • an object recognition device including:
  • An image receiving unit configured to receive an image to be recognized in response to an image recognition request
  • a second acquisition unit is used to acquire a target recognition model obtained through training, wherein the target recognition model includes a limb feature extraction network with known parameters and an updated object recognition network;
  • a feature extraction unit configured to extract the image limb features of the image to be identified based on the limb feature extraction network with known parameters
  • the object recognition unit is used to determine the object recognition result corresponding to the image limb feature of the image to be recognized based on the object recognition network obtained by training, and the target recognition model is obtained by adopting the model construction method described in the first aspect.
  • an embodiment of the present disclosure provides an electronic device, including: a processor and a memory;
  • the memory stores computer-executable instructions
  • the processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the model building method described in the first aspect and various possible designs of the first aspect or the object recognition method described in the second aspect and various possible designs of the second aspect.
  • an embodiment of the present disclosure provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions.
  • the model building method described in the first aspect and various possible designs of the first aspect or the model building method described in the second aspect and various possible designs of the second aspect are implemented.
  • Object recognition methods are implemented.
  • an embodiment of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the model building method described in the first aspect and various possible designs of the first aspect or the object recognition method described in the second aspect and various possible designs of the second aspect.
  • a computer program comprising: instructions, which, when executed by a processor, cause the processor to execute the model building method or the object recognition method according to the aforementioned embodiment;
  • FIG1 is a diagram of an application network architecture of an object recognition method provided by an embodiment of the present disclosure
  • FIG2 is a flow chart of some embodiments of a model building method provided by an embodiment of the present disclosure
  • FIG3 is a flowchart of some other embodiments of a model building method provided by an embodiment of the present disclosure.
  • FIG4 is an example diagram of a target recognition model provided by an embodiment of the present disclosure.
  • FIG5 is a flowchart of some embodiments of an object recognition method provided by an embodiment of the present disclosure.
  • FIG6 is a schematic diagram of the structure of an image processing device provided by an embodiment of the present disclosure.
  • FIG7 is a schematic diagram of the structure of an object recognition device provided by an embodiment of the present disclosure.
  • FIG8 is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present disclosure.
  • the technical solution disclosed in the present invention can be applied to the recognition scenarios of key parts of an image, such as a hand frame and a palm.
  • the limb feature extraction network in the limb recognition model is involved in the training process of the recognition network of objects such as a hand frame and a palm.
  • the construction of the target recognition model can be completed, thereby improving the construction efficiency of the recognition model.
  • hand recognition is an important branch in the field of artificial intelligence, which can be applied to many fields such as home appliances, transportation, finance, and entertainment.
  • a processing flow of first identifying the hand frame, extracting a partial image of the hand frame, and identifying the hand gesture in the partial image can generally be adopted. From this process, it can be seen that the recognition accuracy of the hand frame plays an important role in the hand recognition accuracy.
  • the hand frame recognition model is generally obtained through learning by machine learning models, neural networks, etc.
  • the hand frame recognition model can be trained using images and training data corresponding to the hand frame annotations in the image. A large amount of training data is required in this training process. Even if the hand frame recognition model obtained by training based on a large amount of training data is not accurate, the hand recognition accuracy is not high, resulting in low hand recognition accuracy.
  • the technical solution disclosed in the present invention takes into account that the hands are essentially partial features of the human body, and currently, the limb recognition model also needs to be trained. Therefore, the feature extraction network in the limb recognition model can be used on the basis of the limb recognition model obtained through training. After the feature extraction network extracts features from the image, a regression network for the hands, palms, and other parts is added to realize the extraction of local parts of the limbs.
  • the feature extraction network in the limb recognition model can be used to extract limb features, and the limb features learned by the limb recognition model can be applied to the learning process of the object recognition network of the local parts, which can not only improve the extraction accuracy of the local parts of the limbs, but also reduce the complexity of the training of the target recognition model.
  • a limb recognition model can be obtained, and the limb recognition model can include a limb feature extraction network with known parameters.
  • the limb recognition model can also include a limb recognition network with known parameters, and the limb recognition network can recognize the limb as a whole based on the features extracted by the limb feature extraction network.
  • the limb recognition model can be a model obtained through training, and the feature extraction network can be reused on the basis of the limb recognition model. By inputting the training image into the limb feature extraction network, the image limb features can be obtained, and the image limb features can be used for object recognition of the object recognition network, and the object recognition network is updated by the obtained object recognition results, so that the limb feature extraction network can be reused.
  • the object recognized by the object recognition network can be a representation of a local limb part in the image to be recognized, such as a hand frame, a palm, etc. that represents the hand.
  • the limb feature extraction network can be directly used to extract the corresponding features, and the extracted limb features can be directly used to train the object recognition network, which can greatly reduce the difficulty of building the target recognition model.
  • the target recognition model can be constructed using the limb feature extraction network and the updated object recognition network, and the local parts of the limb can be more accurately extracted through the target recognition model, and the target recognition model can be quickly and accurately obtained. This technical solution can achieve the effect of improving both training efficiency and effectively improving training accuracy.
  • FIG. 1 is an application network architecture diagram according to the object recognition method disclosed in the present invention.
  • the application network architecture according to the embodiment of the present invention may include an electronic device and a client connected to the electronic device through a local area network or a wide area network.
  • the electronic device can be a server of the type of personal computer, ordinary server, super personal computer, cloud server, etc., and the specific type of the electronic device is not too limited in the present invention.
  • the client can be, for example, a terminal device such as a mobile phone, a tablet computer, a personal computer, a smart home appliance, a wearable device, etc., and the specific type of the client is not too limited in the present invention. As shown in FIG.
  • the cloud server 1 can train the target recognition model based on the technical solution of the present invention.
  • Any first client 2 can request the cloud server 1 to obtain the target recognition model after training.
  • the client can send a recognition request to the cloud server to obtain the target recognition model after training.
  • the cloud server can also directly receive the image to be recognized, and recognize the target object in the object to be recognized based on the object recognition method disclosed in the present invention.
  • the limb feature extraction network 11 and the limb recognition network 12 in the limb recognition model can be fixed.
  • An object recognition network 13 is added after the limb feature extraction network 11.
  • the object recognition network 13 can be, for example, one or more of a hand frame recognition network and a palm recognition network.
  • the limb feature extraction network 11 and the limb recognition network 12 are fixed, that is, the limb feature extraction network 11 and the limb recognition network 12 do not participate in the training, and the object recognition network 13 only needs to be trained.
  • the model building method may include the following steps 201 to 204 .
  • a limb recognition model is obtained, where the limb recognition model includes a limb feature extraction network with known parameters, and the limb recognition network is used to recognize limbs.
  • a limb recognition model can be used to recognize limbs in an image.
  • the limbs in the image can specifically refer to the entire limb, which can include the entire region of the limb and/or the key points of the limb.
  • the entire region of the limb can be a rectangular frame that can contain the entire human body, and the key points of the limb can include the position points or coordinate points corresponding to various local parts of the limb.
  • the local parts of the limb can include: the hands, feet, upper and lower limbs, head and other local parts of the limb.
  • the limb recognition model can be obtained by pre-training.
  • the limb recognition model may include a limb feature extraction network with known parameters and a limb recognition network with known parameters.
  • the limb feature extraction network can be used to extract limb features of an input image.
  • the limb recognition network can perform regression calculation on the limb features to obtain corresponding human body frames or limb key points.
  • the specific information obtained by the limb recognition network can be set according to task requirements.
  • the limb recognition model can be obtained by building a network structure of the convolutional neural network, recurrent neural network, etc. In this embodiment, there are no excessive restrictions on the specific algorithm used in the limb recognition network.
  • step 202 the training image is input into a limb feature extraction network to obtain image limb features.
  • the limb feature extraction network in step 202 is a trained limb feature extraction network.
  • the training image may include a two-dimensional training image and/or a three-dimensional training image.
  • the limb feature extraction network may be directly used to extract limb features of the training image.
  • the image limb features may be overall features of the limb in the training image.
  • step 203 based on the image limb features, the object recognition result of the training image is determined through the object recognition network, and the object recognition network is updated based on the object recognition result.
  • the object recognized by the object recognition network is a local part of the limb.
  • the object recognition network can identify the local limb parts of the input image limb features to obtain an object recognition result.
  • the object recognition result is the recognition result of a local part of the limb in the training image.
  • the local part of the limb may include, for example, the hand, the center point of the hand, the eye and/or the center point of the eye.
  • the specific types of the local part of the limb are not limited too much.
  • updating the object recognition network can be to determine the recognition error based on the object recognition result, judge whether the object recognition network meets the convergence condition according to the recognition error, in response to judging that the object recognition meets the convergence condition, determine that the current object recognition network is the object recognition network that has been updated, in response to judging that the object recognition does not meet the convergence condition, update the object recognition network based on the recognition error, and return to step 203 to continue execution.
  • a target recognition model is constructed based on the limb feature extraction network and the updated object recognition network, and the target recognition model is used to recognize the local part of the limb in the target image.
  • the target recognition model may include a limb feature extraction network and an updated object recognition network.
  • the limb feature extraction network and the updated object recognition network may be used to perform recognition of local parts of a limb, such as recognition of a limb hand, recognition of a local limb part such as a hand center point, etc.
  • the application steps of the target recognition model may include: extracting features of the input image through a limb feature extraction network, inputting features output by the limb feature extraction network into the updated object recognition network, identifying local parts of the limbs through the object recognition network, and realizing the comprehensive application of the limb feature extraction network and the object recognition network.
  • the target recognition model may also include: adding a feature enhancement network between the limb feature extraction network and the object recognition network, enhancing the output features of the limb feature extraction network through the feature enhancement network, and inputting the enhanced features into the object recognition network to identify local parts of the limbs.
  • a limb recognition model can be obtained, and the limb recognition model can include a limb feature extraction network with known parameters.
  • the limb recognition model can also include a limb recognition network with known parameters, and the limb recognition network can recognize the limb as a whole based on the features extracted by the limb feature extraction network.
  • the limb recognition model can be a trained model, and the feature extraction network can be reused on the basis of the limb recognition model.
  • the object recognized by the object recognition network can be a representation of a local limb part in the image to be recognized, such as a hand frame, palm, etc. that represent the hand.
  • the limb feature extraction network can be used directly to extract the corresponding features, and the object recognition network can be trained directly using the extracted limb features, which can greatly reduce the difficulty of building the target recognition model.
  • the target recognition model can be constructed using the limb feature extraction network and the updated object recognition network.
  • the local parts of the limbs can be extracted more accurately through the target recognition model, and the target recognition model can be obtained quickly and accurately. This technical solution can achieve the effect of both improving training efficiency and effectively improving training accuracy.
  • the object recognition network can be trained separately, which reduces the complexity of the network that needs to be trained. Therefore, for simpler network training, a small number of training images with local limb labels can be used.
  • the model construction method disclosed in the present invention high-precision training of the object recognition network can be completed through a small number of labeled samples, that is, training images.
  • a target recognition model with high recognition accuracy of local parts of the limbs is obtained, thereby achieving a higher model training effect at a lower labeling cost.
  • FIG. 3 a flowchart of some other embodiments of a model building method provided in the embodiments of the present disclosure is provided.
  • the difference from the aforementioned embodiments is that the object recognition result of the training image is determined based on the image limb features through an object recognition network, which may include steps 301 to 302.
  • step 301 candidate parameters are determined for the object recognition network; the candidate parameters are parameter values of each network function in the object recognition network.
  • the candidate parameters may be parameter values of each network function in the object recognition network.
  • the object recognition network may include multiple network functions, such as activation functions, excitation functions, etc.
  • each network function has a corresponding weight and an offset.
  • the candidate parameters may include the value of the weight and the value of the offset.
  • step 302 based on the object recognition network corresponding to the candidate parameters, an object recognition result corresponding to the image limb feature is determined.
  • the limb feature extraction network can be used as an encoder in the target recognition model. Extract limb features in the image.
  • the limb feature extraction network is known, and the image limb features of the training image can be extracted through the limb feature extraction network.
  • the image limb features can be vectors or matrices that characterize the limb features in the training image.
  • Step 302 may specifically include: inputting the image limb features into the object recognition network corresponding to the candidate parameters to obtain the object recognition result.
  • the image limb features may be feature enhanced, and the enhanced image limb features may be input into the object recognition network corresponding to the candidate parameters to obtain the object recognition result.
  • the image limb features may also be processed in other ways and then input into the object recognition network corresponding to the candidate parameters.
  • step 203 the object recognition network is updated based on the object recognition result, and steps 303 to 305 may also be included.
  • step 303 it is determined whether the object recognition network meets the convergence condition according to the object recognition result.
  • step 304 is executed, and in response to determining that the object recognition network does not meet the convergence condition, step 305 is executed.
  • step 304 the candidate parameters are determined to be target parameters, and the object recognition network corresponding to the target parameters is determined to be the updated object recognition network.
  • step 305 the candidate parameters are updated and the process returns to step 302 to continue execution.
  • the training image may be associated with a recognition result label, and the recognition result label may be the annotation result of the local limb in the image.
  • the recognition result label may be the annotation result of the real rectangular frame where the hand is located, referred to as the hand frame label.
  • the recognition result label may be the annotation result of the real coordinate point corresponding to the palm.
  • Determining whether the object recognition network satisfies the convergence condition based on the object recognition result may include: determining the recognition error through the object recognition result and the recognition result label corresponding to the training image, judging whether the object recognition network satisfies the convergence condition based on the recognition error, determining that the current object recognition network is an updated object recognition network in response to judging that the object recognition network satisfies the convergence condition based on the recognition error, and updating the object recognition network based on the recognition error in response to judging that the object recognition network does not satisfy the convergence condition based on the recognition error.
  • the object recognition network can perform feature regression and classification on the image limb features to obtain the corresponding object recognition results.
  • the object recognition results may include a rectangular frame corresponding to the area or the location point of the object.
  • the result type of the object recognition result can be specifically determined according to the object characteristics of the recognized object.
  • the object recognition result may be a hand frame.
  • the object recognition result can be used for loss calculation of the object recognition network to obtain a recognition error.
  • Judging whether the object recognition network meets the convergence condition according to the recognition error may include: judging whether the recognition error is less than a loss threshold, in response to judging that the recognition error is less than the loss threshold, determining that the recognition error meets the convergence condition, and in response to judging that the recognition error is greater than or equal to the loss threshold, determining that the recognition error does not meet the convergence condition.
  • updating the candidate parameters may include: updating the candidate parameters based on a gradient descent algorithm and a recognition error to obtain updated candidate parameters.
  • the image limb features of the training image can be extracted using the limb feature extraction network, and the object recognition network corresponding to the candidate parameters can be used to perform local limb recognition on the image limb features to obtain object recognition results.
  • the object recognition results are used to detect the convergence conditions of the object recognition network.
  • the current candidate parameters of the object recognition network can be evaluated for their use effect through the convergence conditions, so that the object recognition network can be applied to the limb feature extraction network during the training process, reducing the complexity of the network that needs to be trained and improving the training efficiency of the object recognition network.
  • the convergence conditions are used to judge whether the object recognition network has terminated training, accurately monitor the training process of the object recognition network, and improve the training accuracy of the object recognition network.
  • the difference from the above embodiments is that the object recognition network includes a hand frame recognition network; based on the image limb features, determining the object recognition result of the training image through the object recognition network may include:
  • a hand frame recognition result corresponding to the image limb feature is determined, and the hand frame parameters are candidate parameters of the hand frame recognition network.
  • Update the object recognition network based on the object recognition results including:
  • the hand frame parameters are updated, the hand frame recognition network corresponding to the hand frame parameters is returned, and the hand frame recognition result corresponding to the image limb feature is determined to continue execution.
  • determining a hand frame recognition result corresponding to an image limb feature may include: inputting the image limb feature into the hand frame recognition network corresponding to the hand frame parameter to obtain a hand frame recognition result.
  • the image limb feature may also be feature enhanced, and the feature-enhanced image limb feature is input into the hand frame recognition network corresponding to the hand frame parameter to obtain a hand frame recognition result.
  • the hand frame recognition network can be used to recognize the hand frame in the image limb features.
  • the hand frame recognition result can include a rectangular frame corresponding to the area where the hand is located in the image, which can be called a hand frame.
  • the object recognition network may also include recognition networks of other parts of the human body, such as a palm recognition network, a face recognition network, etc., which may be specifically determined according to the recognition purpose or recognition requirement of the recognition network.
  • hand frame recognition can be performed based on the hand frame recognition network corresponding to the hand frame parameters to obtain a hand frame recognition result.
  • the hand frame recognition result can be used to determine whether the object recognition network meets the convergence condition.
  • the hand frame parameters can be updated, that is, the object recognition network can be updated.
  • the hand frame recognition network corresponding to the hand frame parameters can be determined as the updated object recognition network.
  • the hand frame recognition result is used to determine whether the hand frame recognition network corresponding to the hand frame parameters meets the convergence condition, and then the hand frame recognition network is updated when the convergence condition is not met. This can achieve iterative update of the hand frame recognition network and obtain an accurate hand frame recognition network.
  • the object recognition network may further include: a palm recognition network and a hand frame recognition network.
  • the object recognition results of the training image are determined through the object recognition network, including:
  • the palm recognition result corresponding to the image limb features is determined, and the palm parameters are candidate parameters of the palm recognition network.
  • Update the object recognition network based on the object recognition results including:
  • the hand frame parameters and the palm parameters are updated, the hand frame recognition network corresponding to the hand frame parameters is returned, and the hand frame recognition result corresponding to the image limb feature is determined to continue execution.
  • the object recognition result may include a hand frame recognition result and a palm recognition result.
  • the palm recognition network may be used to recognize the center point of a human hand, and the palm recognition result may include palm position coordinates, which may refer to the coordinates of the palm in the training image or the image to be recognized.
  • the target recognition model may include a limb feature extraction network 401.
  • the input of the limb feature extraction network 401 may be an image, and the output may be connected to the hand frame recognition network 402, the palm recognition network 403, and the limb recognition network 404, respectively.
  • the features extracted by the limb feature extraction network 401 may be respectively input to the hand frame recognition network 402, the palm recognition network 403, and the limb recognition network 404, to obtain the hand frame recognition result output by the hand frame recognition network 402, the palm recognition result output by the palm recognition network 403, and the limb recognition result output by the limb recognition network 404, and the limb recognition result may include, for example, limb key points or limb regions.
  • the object recognition network may also include a palm recognition network.
  • the palm recognition network By setting up the palm recognition network, the effective use of the image limb features can be further realized.
  • adding the palm recognition network can combine the two key recognition objects of the hand, namely the hand frame and the palm point.
  • the two networks are jointly trained so that the training processes of the hand frame recognition network and the hand palm recognition network influence each other, thereby improving the recognition accuracy of the hand frame recognition network and the hand palm recognition network.
  • judging whether the object recognition network meets the convergence condition according to the hand frame recognition result and the hand palm recognition result includes:
  • the first loss value may be obtained by performing a difference calculation between the hand frame recognition result and the hand frame label. For example, the distance between the hand frame recognition result and the hand frame label may be calculated to obtain the first loss value.
  • the second loss value may be obtained by performing a difference calculation between the palm recognition result and the palm label. For example, the distance between the palm recognition result and the palm label may be calculated to obtain the second loss value.
  • the target loss value may be obtained by weighted summing the first loss value and the second loss value.
  • the first weight of the hand frame recognition network and the second weight of the palm recognition network may be determined, the first value may be obtained by multiplying the first weight and the first loss value, the second value may be obtained by multiplying the second weight and the second loss value, and the target loss value may be obtained by summing the first value and the second value.
  • the first weight and the second weight may be preset constants. In other embodiments, in order to improve the accuracy of model construction, the first weight and the second weight may be used as training parameters to participate in the training of the hand frame recognition network and the palm recognition network, and the weight value obtained in the last iteration is the training result of the first weight and the second weight.
  • the target loss value meeting the loss threshold may mean that the target loss value is less than or equal to the loss threshold.
  • the target loss value not meeting the loss threshold may mean that the target loss value is greater than the loss threshold.
  • the training effects of the hand frame recognition network and the palm recognition network can be evaluated respectively by calculating the first loss value of the hand frame recognition network and the second loss value of the palm recognition network.
  • the final target loss value is calculated by weighted summing the first loss value and the second loss value, so that the two loss values are calculated comprehensively, so that the hand frame recognition network and the palm recognition network can jointly judge the convergence conditions, and the hand frame recognition network and the palm recognition network can be trained at the same time.
  • the palm recognition network and the hand frame recognition network influence each other to improve the network recognition accuracy.
  • the step of determining the recognition result label corresponding to the training image in the object recognition network includes:
  • the recognition result labels corresponding to the training images in the object recognition network are generated.
  • the recognition result label can be determined according to the recognition object of the object recognition network. For example, when the recognition object of the object recognition network is the palm of a hand, the recognition result label can be a palm label. When the recognition object of the object recognition network is a hand frame, the recognition result label can be a hand frame label.
  • a palm label of a training image in an object recognition network may be generated based on the limb key point label.
  • a hand frame label of a training image in an object recognition network may be generated based on the limb key point label.
  • limb key point labels are obtained when training a limb recognition model using existing training images, and recognition result labels corresponding to the training images in the object recognition network are generated based on the limb key point labels.
  • the existing limb key point labels can be used through interpolation, fitting, etc. to achieve the labeling of recognition result labels corresponding to local parts of the limbs.
  • the labeling efficiency can be improved and the number of labels can be enriched by reusing the limb key point labels.
  • the method further includes:
  • the target loss value may be used in combination with a gradient descent algorithm to update the hand frame parameters and the palm parameters.
  • a gradient descent algorithm to update the hand frame parameters and the palm parameters.
  • other parameter update algorithms may also be used, such as a stochastic gradient descent algorithm, etc.
  • the specific algorithm used for parameter update in this embodiment is not limited too much.
  • the second loss value is calculated, including:
  • the mean square error loss function is used to calculate the second loss value corresponding to the palm recognition result and the palm annotation of the training image.
  • the regression loss function may include any one of the loss functions such as the mean absolute error (also known as the L1 loss function), the mean square error loss function (also known as the L2 loss function), the smoothed mean absolute error (Huber smoothed L1 loss function), etc.
  • the mean absolute error also known as the L1 loss function
  • the mean square error loss function also known as the L2 loss function
  • the smoothed mean absolute error Huber smoothed L1 loss function
  • the mean squared error loss function (MSE) is also called the L2 loss function, which can calculate the error of the sum of the squares of the difference between the predicted value and the target value.
  • MSE mean squared error loss function
  • the first loss value can be calculated using a regression loss function
  • the second loss value can be calculated using a mean square error function.
  • the hand frame recognition network may include a global average pooling layer and a fully connected layer.
  • the palm recognition network may be constructed based on a heat map algorithm.
  • the global average pooling layer can be constructed by the global average pooling algorithm.
  • the global average pooling algorithm can calculate the average of the raw data input by each channel, obtain the average corresponding to each channel, and use the average corresponding to each channel as the input of the fully connected layer.
  • the global average pooling algorithm can reduce the complexity of the data structure of each channel and improve the network calculation speed.
  • the fully connected layer can be constructed by the fully connected algorithm (FC), which can be used for the classification calculation of the hand frame.
  • FC fully connected algorithm
  • the fully connected layer can realize the fast and accurate classification of the hand frame.
  • the heatmap algorithm can be used to identify the palm point of the input feature. When the above algorithm is implemented, the corresponding object can be recognized.
  • the limb recognition model can be trained.
  • the training steps of the limb recognition model may include:
  • a limb recognition model is trained
  • a limb recognition model is trained based on the two-dimensional limb annotation data and the three-dimensional limb annotation data.
  • the two-dimensional limb annotation data may include training images and two-dimensional annotation data of the training images.
  • Two-dimensional annotation data may refer to data represented in two-dimensional data, such as two-dimensional coordinate points, two-dimensional rectangular boxes, etc.
  • Three-dimensional annotation data may refer to data that includes depth data of an image in addition to two-dimensional data, for example, depth data is added on the basis of two-dimensional coordinate points, and depth data of a two-dimensional rectangular box is added on the basis of a two-dimensional rectangular box.
  • the limb recognition model can perform limb recognition on an image and obtain limb recognition results.
  • the limbs involved in this case include but are not limited to limb key points, limb areas, etc.
  • limbs can also include cartoon characters, animals, etc. as subjects to be recognized.
  • the limb recognition model can be trained using two-dimensional limb annotation data, or the limb recognition model can be trained using two-dimensional limb annotation data and three-dimensional limb annotation data.
  • the limb recognition model can accurately recognize the limbs of the image, thereby improving the training efficiency and accuracy of the limb recognition model.
  • the limb recognition model further includes: a limb recognition network with known parameters;
  • a target recognition model is constructed, including:
  • a target recognition model is determined, and the target recognition model is used to perform recognition of limbs and local parts of limbs in the target image.
  • the parameters of the limb feature extraction network and the limb recognition network in the target recognition model remain unchanged, and the parameters of the limb feature extraction network and the limb recognition network are not iteratively updated, while the parameters of the object recognition network are continuously iteratively updated during training until the convergence conditions are met.
  • the limb recognition model is obtained to obtain the limb feature extraction network and the limb recognition network.
  • the limb recognition network can be used as a recognition branch of the target recognition model.
  • an object recognition network is added.
  • a recognition network branch is added to realize multi-task recognition, and multiple tasks can share the limb feature extraction network (backbone).
  • the object recognition method may include the following steps 501 to 504 .
  • step 501 in response to an image recognition request, an image to be recognized is received.
  • a trained target recognition model is obtained, where the target recognition model includes a limb feature extraction network with known parameters and a trained object recognition network.
  • step 503 based on a limb feature extraction network with known parameters, image limb features of the image to be identified are extracted.
  • step 504 based on the trained object recognition network, an object recognition result corresponding to the image limb feature of the image to be recognized is determined, and the target recognition model is obtained by using the model building method of any of the above embodiments.
  • the object recognition results include recognition results of limbs and limb parts.
  • the object recognition network may include: a palm recognition network and/or a hand frame recognition network.
  • the object recognition result corresponding to the image limb features of the image to be recognized may include: a palm recognition result obtained by the palm recognition network recognizing the image limb features of the image to be recognized, for example, it may include palm point coordinates and/or a hand frame recognition result obtained by the hand frame recognition network recognizing the image limb features of the image to be recognized, for example, it may include a hand frame.
  • the object recognition network may further include: a limb recognition network.
  • the object recognition result corresponding to the image limb features of the image to be recognized may also include: a limb recognition result obtained by the limb recognition network recognizing the image limb features of the image to be recognized, for example, a rectangular frame corresponding to a limb key point or limb area.
  • an image to be recognized can be received in response to an image recognition request.
  • the limb feature extraction network in the target recognition model can be used to extract the image limb features of the image to be recognized.
  • the object recognition network Through the object recognition network, the object recognition result corresponding to the image limb features of the image to be recognized can be obtained, so as to realize the recognition of limbs. Accurate identification of objects.
  • the model building device 600 may include a first acquisition unit 601 , a feature extraction unit 602 , a model training unit 603 and a model building unit 604 .
  • the first acquisition unit 601 is used to acquire a limb recognition model.
  • the limb recognition model includes a limb feature extraction network with known parameters.
  • the limb recognition model is used to recognize limbs.
  • the feature extraction unit 602 is used to input the training image into the limb feature extraction network to obtain the image limb features.
  • the model training unit 603 is used to determine the object recognition result of the training image based on the image limb features through the object recognition network, and update the object recognition network based on the object recognition result.
  • the object recognized by the object recognition network is a local part of the limb.
  • the model building unit 604 is used to build a target recognition model based on the limb feature extraction network and the updated object recognition network, and the target recognition model is used to perform recognition of local parts of the limbs in the target image.
  • the model training unit may include:
  • a first determination module is used to determine candidate parameters for the object recognition network, where the candidate parameters are parameter values of each network function in the object recognition network;
  • a first recognition module used to determine an object recognition result corresponding to the image limb feature based on an object recognition network corresponding to the candidate parameters
  • a first processing module is used to determine, in response to the object recognition result, that the object recognition network meets the convergence condition, determine the candidate parameters as the target parameters, and determine that the object recognition network corresponding to the target parameters is the updated object recognition network;
  • the second processing module is used to respond to the object recognition result, determine that the object recognition network does not meet the convergence condition, update the candidate parameters, return to the object recognition network corresponding to the candidate parameters, determine the object recognition result corresponding to the image limb feature and continue to execute.
  • the object recognition network includes a hand frame recognition network; the model training unit may further include:
  • a second determination module is used to determine a hand frame recognition result corresponding to the image limb feature based on a hand frame recognition network corresponding to the hand frame parameters, where the hand frame parameters are candidate parameters of the hand frame recognition network;
  • the first judgment module is used to judge whether the object recognition network meets the convergence condition according to the hand frame recognition result; in response to judging that the object recognition network meets the convergence condition according to the hand frame recognition result, determine that the hand frame recognition network corresponding to the hand frame parameters is the object recognition network that has completed the update; in response to judging that the object recognition network does not meet the convergence condition according to the hand frame recognition result, update the hand frame parameters, return to the hand frame recognition network corresponding to the hand frame parameters, and determine the hand frame recognition result corresponding to the image limb feature to continue execution.
  • the object recognition network includes: a palm recognition network and a hand frame recognition network.
  • Model training unit It may also include:
  • a third determination module is used to determine a hand frame recognition result corresponding to the image limb feature based on a hand frame recognition network corresponding to the hand frame parameters, where the hand frame parameters are candidate parameters of the hand frame recognition network;
  • a fourth determination module is used to determine the palm recognition result corresponding to the image limb feature based on the palm recognition network corresponding to the palm parameters, where the palm parameters are candidate parameters of the palm recognition network;
  • the second judgment module is used to judge whether the object recognition network meets the convergence condition according to the hand frame recognition result and the palm recognition result; in response to judging that the object recognition network meets the convergence condition according to the hand frame recognition result and the palm recognition result, determine that the hand frame recognition network corresponding to the hand frame parameters and the palm recognition network corresponding to the palm parameters are the object recognition networks that have completed the update; in response to judging that the object recognition network does not meet the convergence condition according to the hand frame recognition result and the palm recognition result, update the hand frame parameters and the palm parameters, return to the hand frame recognition network corresponding to the hand frame parameters, and determine the hand frame recognition result corresponding to the image limb feature to continue execution.
  • the second determination module includes:
  • a first calculation submodule used to calculate a first loss value according to a hand frame recognition result and a hand frame label of a training image
  • a second calculation submodule used for calculating a second loss value according to the palm recognition result and the palm label of the training image
  • a target calculation submodule used to perform a weighted summation of the first loss value and the second loss value to obtain a target loss value
  • a first judgment submodule configured to determine that the object recognition network satisfies a convergence condition in response to determining that the target loss value satisfies a loss threshold
  • the second judgment submodule is used to determine that the object recognition network does not meet the convergence condition in response to determining that the target loss value does not meet the loss threshold.
  • the second determination module further includes:
  • the label determination unit is used to determine the labels of limb key points when training a limb recognition model using a training image.
  • the label generation unit is used to generate the recognition result label corresponding to the training image in the object recognition network according to the limb key point label.
  • the second determination module further includes:
  • a first training unit is used to train and obtain a limb recognition model based on two-dimensional limb annotation data
  • the second training unit is used to train and obtain a limb recognition model based on the two-dimensional limb annotation data and the three-dimensional limb annotation data.
  • the limb recognition model further includes: a limb recognition network with known parameters;
  • Model building unit specifically for:
  • a target recognition model is constructed based on the limb feature extraction network, the limb recognition network and the object recognition network to be trained.
  • model building unit may be specifically used to:
  • a target recognition model is determined, and the target recognition model is used to perform recognition of limbs and local parts of limbs in the target image.
  • the object recognition device 700 may include an image receiving unit 701 , a second acquisition unit 702 , a feature extraction unit 703 and an object recognition unit 704 .
  • the image receiving unit 701 is used to receive an image to be recognized in response to an image recognition request.
  • the second acquisition unit 702 is used to acquire the target recognition model obtained through training, where the target recognition model includes a limb feature extraction network with known parameters and an updated object recognition network.
  • the feature extraction unit 703 is used to extract the image limb features of the image to be identified based on a limb feature extraction network with known parameters.
  • the object recognition unit 704 is used to determine the object recognition result corresponding to the image limb feature of the image to be recognized based on the object recognition network obtained through training.
  • the target recognition model is obtained by using the model construction method provided by any of the above embodiments.
  • the object recognition device provided in this embodiment can be used to execute the technical solution of the above method embodiment. Its implementation principle and technical effect are similar, and this embodiment will not be repeated here.
  • the embodiment of the present disclosure also provides an electronic device.
  • FIG8 it shows a schematic diagram of the structure of an electronic device 800 suitable for implementing the embodiment of the present disclosure
  • the electronic device 800 may be a terminal device or a server.
  • the terminal device may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (Portable Android Devices, PADs), portable multimedia players (PMPs), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc.
  • PDAs personal digital assistants
  • PADs Portable Android Devices, PADs
  • PMPs portable multimedia players
  • vehicle-mounted terminals such as vehicle-mounted navigation terminals
  • fixed terminals such as digital TVs, desktop computers, etc.
  • the electronic device shown in FIG8 is only an example and should not bring any limitation to the functions and scope of use of the embodiment of the present disclosure.
  • the electronic device 800 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 801, which may perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage device 808 to a random access memory (RAM) 803.
  • a processing device 801 e.g., a central processing unit, a graphics processing unit, etc.
  • RAM random access memory
  • Various programs and data required for the operation of the electronic device 800 are also stored in the RAM 803.
  • the processing device 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804.
  • An input/output (I/O) interface 805 is also connected to the bus 804.
  • the following devices can be connected to the I/O interface 805: input devices 806 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; input devices 806 including, for example, a liquid crystal display (LCD);
  • the electronic device 800 may include an output device 807 such as a display, LCD, speaker, vibrator, etc.; a storage device 808 such as a magnetic tape, a hard disk, etc.; and a communication device 809.
  • the communication device 809 may allow the electronic device 800 to communicate with other devices wirelessly or by wire to exchange data.
  • FIG. 8 shows an electronic device 800 with various devices, it should be understood that it is not required to implement or have all the devices shown. More or fewer devices may be implemented or provided instead.
  • an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart.
  • the computer program can be downloaded and installed from the network through the communication device 809, or installed from the storage device 808, or installed from the ROM 802.
  • the processing device 801 the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.
  • the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two.
  • the computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above.
  • Computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
  • a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device.
  • a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried.
  • This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above.
  • the computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device.
  • the program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.
  • the computer-readable medium may be included in the electronic device, or may exist independently without being installed in the electronic device.
  • the computer-readable medium carries one or more programs.
  • the electronic device executes the method shown in the above embodiment.
  • Computer program code for carrying out operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, Conventional procedural programming languages are also included, such as "C" or similar programming languages.
  • the program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server.
  • the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
  • LAN local area network
  • WAN wide area network
  • Internet service provider e.g., via the Internet using an Internet service provider
  • each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function.
  • the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved.
  • each square box in the block diagram and/or flow chart, and the combination of the square boxes in the block diagram and/or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
  • the units involved in the embodiments described in the present disclosure may be implemented by software or hardware.
  • the name of a unit does not limit the unit itself in some cases.
  • the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses".
  • exemplary types of hardware logic components include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.
  • FPGAs field programmable gate arrays
  • ASICs application specific integrated circuits
  • ASSPs application specific standard products
  • SOCs systems on chip
  • CPLDs complex programmable logic devices
  • a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment.
  • a machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium.
  • a machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing.
  • a more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
  • RAM random access memory
  • ROM read-only memory
  • EPROM or flash memory erasable programmable read-only memory
  • CD-ROM portable compact disk read-only memory
  • CD-ROM compact disk read-only memory
  • magnetic storage device or any suitable combination of the foregoing.
  • the present disclosure provides a model construction, object recognition method, device, equipment, medium and product to overcome Currently, hand frame training is difficult and the recognition accuracy is low.
  • a model building method comprising:
  • the limb recognition model includes a limb feature extraction network with known parameters, and the limb recognition model is used to recognize limbs;
  • the object recognition result of the training image is determined through the object recognition network, and the object recognition network is updated based on the object recognition result, and the object recognized by the object recognition network is a local part of the limb;
  • a target recognition model is constructed, and the target recognition model is used to perform recognition of local parts of the limbs in the target image.
  • determining the object recognition result of the training image through the object recognition network includes:
  • Update the object recognition network based on the object recognition results including:
  • the candidate parameters are updated, and the object recognition network corresponding to the candidate parameters is returned to determine the object recognition result corresponding to the image limb feature to continue execution.
  • the object recognition network includes a hand frame recognition network
  • the object recognition results of the training image are determined through the object recognition network, including:
  • Update the object recognition network based on the object recognition results including:
  • the hand frame parameters are updated, the hand frame recognition network corresponding to the hand frame parameters is returned, and the hand frame recognition result corresponding to the image limb feature is determined to continue execution.
  • the object recognition network includes: a palm recognition network and a hand frame recognition network;
  • the object recognition results of the training image are determined through the object recognition network, including:
  • the hand frame recognition results corresponding to the image limb features are determined.
  • the number is the candidate parameter of the hand frame recognition network
  • the palm recognition result corresponding to the image limb features is determined, and the palm parameters are candidate parameters of the palm recognition network;
  • Update the object recognition network based on the object recognition results including:
  • the hand frame parameters and the palm parameters are updated, the hand frame recognition network corresponding to the hand frame parameters is returned, and the hand frame recognition result corresponding to the image limb feature is determined to continue execution.
  • judging whether the object recognition network meets the convergence condition according to the hand frame recognition result and the hand palm recognition result includes:
  • the step of determining the recognition result label corresponding to the training image in the object recognition network includes:
  • the recognition result labels corresponding to the training images in the object recognition network are generated.
  • the training step of the limb recognition model includes:
  • a limb recognition model is trained
  • a limb recognition model is trained based on the two-dimensional limb annotation data and the three-dimensional limb annotation data.
  • the limb recognition model further includes: a limb recognition network with known parameters;
  • a target recognition model is constructed, including:
  • a target recognition model is determined, and the target recognition model is used to perform recognition of limbs and local parts of limbs in the target image.
  • an object recognition method comprising:
  • the target recognition model includes a limb feature extraction network with known parameters and a trained object recognition network; the object recognized by the object recognition network is a local part of a limb;
  • the limb features of the image to be identified are extracted;
  • an object recognition result corresponding to the image limb feature of the image to be recognized is determined, and the target recognition model is obtained by using the model construction method provided by any of the above embodiments.
  • a model building device comprising:
  • a first acquisition unit is used to acquire a limb recognition model
  • the limb recognition model includes a limb feature extraction network with known parameters, and the limb recognition model is used to recognize limbs;
  • a feature extraction unit used for inputting a training image into a limb feature extraction network to obtain image limb features
  • a model training unit is used to determine the object recognition result of the training image through the object recognition network based on the image limb features, and update the object recognition network based on the object recognition result, wherein the object recognized by the object recognition network is a local part of the limb;
  • the model building unit is used to build a target recognition model based on the limb feature extraction network and the updated object recognition network, and the target recognition model is used to perform recognition of local parts of the limbs in the target image.
  • an object recognition device comprising:
  • An image receiving unit configured to receive an image to be recognized in response to an image recognition request
  • a second acquisition unit is used to acquire a target recognition model obtained through training, wherein the target recognition model includes a limb feature extraction network with known parameters and an updated object recognition network;
  • a feature extraction unit used for extracting image limb features of an image to be identified based on a limb feature extraction network with known parameters
  • the object recognition unit is used to determine the object recognition result corresponding to the image limb feature of the image to be recognized based on the object recognition network obtained through training.
  • the target recognition model is obtained by using the model construction method provided by any of the above embodiments.
  • an electronic device comprising: at least one processor and a memory;
  • Memory stores computer-executable instructions
  • At least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the model building method or object recognition method provided by the aforementioned embodiments.
  • a computer-readable storage medium in which computer execution instructions are stored.
  • a processor executes the computer execution instructions, the model building method or object recognition method provided in the aforementioned embodiments is implemented.
  • a computer program product comprising: A computer program, when executed by a processor, implements the model building method or object recognition method provided by the aforementioned embodiment.
  • a computer program comprising: instructions, which, when executed by a processor, cause the processor to execute the model building method or the object recognition method according to the aforementioned embodiments.
  • the technical solution provided by this embodiment can obtain a limb recognition model.
  • the parameters of the limb feature extraction network in the limb recognition model are known and can be directly used to extract the image limb features of the training image.
  • the image limb features can be object recognized by the object recognition network to obtain the object recognition result of the training image.
  • the object recognition result can be used to update the object recognition network, and the object recognition network can be a local part of the object limb.
  • the image limb features that can be learned by the limb feature extraction network can be applied to the training of the object recognition network, so that the object recognition network can learn the image limb features and perform object recognition.
  • the object recognized by the object recognition network can be a local part of the limb.
  • the object recognition network can be updated using the object recognition result, reducing the number of updated networks, and then realizing fast and accurate training of the object recognition network. This technical solution can achieve the effect of improving training efficiency and effectively improving training accuracy.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • General Health & Medical Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • Artificial Intelligence (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Psychiatry (AREA)
  • Social Psychology (AREA)
  • Image Analysis (AREA)

Abstract

本公开实施例提供一种模型构建、对象识别方法、装置、设备、介质及产品，该方法可以包括：获取肢体识别模型，所述肢体识别模型包括参数已知的肢体特征提取网络，所述肢体识别模型用于识别肢体；将训练图像输入所述肢体特征提取网络，获得图像肢体特征；基于所述图像肢体特征，通过对象识别网络，确定训练图像的对象识别结果，并基于所述对象识别结果更新所述对象识别网络，所述对象识别网络识别的对象为肢体中的局部部位；基于所述肢体特征提取网络和完成更新的对象识别网络，构建目标识别模型，所述目标识别模型用于执行目标图像中肢体的局部部位的识别。本公开降低了目标识别模型训练的复杂度，提高了训练精度。

Description

模型构建、对象识别方法、装置、设备、介质及产品
相关申请的交叉引用
本申请是以中国申请号为202310267677.0，申请日为2023年3月14日、题目为“模型构建、对象识别方法、装置、设备、介质及产品”的申请为基础，并主张其优先权，该中国申请的公开内容在此作为整体引入本申请中。
技术领域
本公开实施例涉及图像处理技术领域，尤其涉及一种模型构建、对象识别方法、装置、设备、介质及产品。
背景技术
基于图像的手势识别可以指识别图像中人所作出的手部动作，通过识别手部动作可以进行手势控制，提高交互效率。目前，手部动作的识别需要先识别图像中手部所在区域，也可称手框，再提取手框对应的局部图像，然后从局部图像中识别手势。
目前识别图像中手框的方法主要为采用大量的训练图像，包括图像和图像中手框的标签对图像识别网络进行模型训练，
发明内容
第一方面，本公开实施例提供一种模型构建方法，包括：
获取肢体识别模型，所述肢体识别模型包括参数已知的肢体特征提取网络，所述肢体识别模型用于识别肢体；
将训练图像输入所述肢体特征提取网络，获得图像肢体特征；
基于所述图像肢体特征，通过对象识别网络，确定训练图像的对象识别结果，并基于所述对象识别结果更新所述对象识别网络，所述对象识别网络识别的对象为肢体中的局部部位；
基于所述肢体特征提取网络和完成更新的对象识别网络，构建目标识别模型，所述目标识别模型用于执行目标图像中肢体的局部部位的识别。
第二方面，本公开实施例提供一种对象识别方法，包括：
响应于图像识别请求，接收待识别图像；
获取训练获得的目标识别模型，所述目标识别模型包括参数已知的肢体特征提取网络和训练获得的对象识别网络；对象识别网络识别的对象为肢体中的局部部位；
基于所述参数已知的肢体特征提取网络，提取获得所述待识别图像的图像肢体特征；
基于所述训练获得的对象识别网络，确定所述待识别图像的图像肢体特征对应的对象识别结果，所述目标识别模型是采用第一方面所述的模型构建方法获得的。
第三方面，本公开实施例提供一种模型构建装置，包括：
第一获取单元，用于获取肢体识别模型，所述肢体识别模型包括参数已知的肢体特征提取网络，所述肢体识别模型用于识别肢体；
特征提取单元，用于将训练图像输入所述肢体特征提取网络，获得图像肢体特征；
模型训练单元，用于基于所述图像肢体特征，通过对象识别网络，确定训练图像的对象识别结果，并基于所述对象识别结果更新所述对象识别网络，所述对象识别网络识别的对象为肢体中的局部部位；
模型构建单元，用于基于所述肢体特征提取网络和完成更新的对象识别网络，构建目标识别模型，所述目标识别模型用于执行目标图像中肢体的局部部位的识别。
第四方面，本公开实施例提供一种对象识别装置，包括：
图像接收单元，用于响应于图像识别请求，接收待识别图像；
第二获取单元，用于获取训练获得的目标识别模型，所述目标识别模型包括参数已知的肢体特征提取网络和完成更新的对象识别网络；
特征提取单元，用于基于所述参数已知的肢体特征提取网络，提取获得所述待识别图像的图像肢体特征；
对象识别单元，用于基于所述训练获得的对象识别网络，确定所述待识别图像的图像肢体特征对应的对象识别结果，所述目标识别模型是采用第一方面所述的模型构建方法获得的。
第五方面，本公开实施例提供一种电子设备，包括：处理器以及存储器；
所述存储器存储计算机执行指令；
所述处理器执行所述存储器存储的计算机执行指令，使得所述至少一个处理器执行如上第一方面以及第一方面各种可能的设计所述的模型构建方法或第二方面以及第二方面各种可能的设计的对象识别方法。
第六方面，本公开实施例提供一种计算机可读存储介质，所述计算机可读存储介质中存储有计算机执行指令，当处理器执行所述计算机执行指令时，实现如上第一方面以及第一方面各种可能的设计所述的模型构建方法或第二方面以及第二方面各种可能的设计的 对象识别方法。
第七方面，本公开实施例提供一种计算机程序产品，包括计算机程序，所述计算机程序被处理器执行时实现如上第一方面以及第一方面各种可能的设计所述的模型构建方法或第二方面以及第二方面各种可能的设计的对象识别方法。
第八方面，根据本公开的一个或多个实施例，提供了一种计算机程序，包括：指令，所述指令当由处理器执行时使所述处理器执行根据前述实施例所述的模型构建方法或者对象识别方法
附图说明
为了更清楚地说明本公开实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作一简单地介绍，显而易见地，下面描述中的附图是本公开的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。
图1为本公开实施例提供的一种对象识别方法的一个应用网络架构图；
图2为本公开实施例提供的一种模型构建方法的一些实施例的流程图；
图3为本公开实施例提供的一种模型构建方法的又一些实施例的流程图；
图4为本公开实施例提供的一个目标识别模型的示例图；
图5为本公开实施例提供的一种对象识别方法的一些实施例的流程图；
图6为本公开实施例提供的一种图像处理装置的一个结构示意图；
图7为本公开实施例提供的一种对象识别装置的一个结构示意图；
图8为本公开实施例提供的一种电子设备的硬件结构示意图。
具体实施方式
为使本公开实施例的目的、技术方案和优点更加清楚，下面将结合本公开实施例中的附图，对本公开实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本公开一部分实施例，而不是全部的实施例。基于本公开中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本公开保护的范围。
本公开的技术方案可以应用于图像的手框、手心等关键部位的识别场景中，通过采用已训练获得的肢体识别模型，将肢体识别模型中的肢体特征提取网络参与到手框、手心等对象的识别网络的训练过程中，通过复用肢体特征提取网络，更新对象识别网络即可完成目标识别模型的构建，提高识别模型的构建效率。
相关技术中，手部识别为人工智能领域中的一个重要分支，可以应用于家电、交通、金融、娱乐等诸多领域。为了获得准确的手部识别结果，一般可以采用先识别手框、提取手框的局部图像、识别局部图像中的手部姿态的处理流程。由该流程可知，手框的识别精度对手部识别精度起到重要作用。目前，手框识别模型一般通过机器学习模型、神经网络等学习获得。一般可以采用图像和对图像中手框标注所对应的训练数据训练手框识别模型。这一训练过程中需要使用到大量的训练数据，即便是根据大量的训练数据训练获得的手框识别模型的精度也不高，导致手部识别精度不高。
目前手框识别网络训练过程中需要使用到大量训练数据完成网络的训练，在标签数据有限时，频繁出现手框识别不准确的问题。
因此，如何准确地识别图像中的手框是目前亟待解决的技术问题。
为了解决上述技术问题，本公开技术方案，考虑到手部本质为人体上的部分特征，而目前，肢体识别模型也是需要训练的，因此，可以在训练获得的肢体识别模型的基础上，采用该肢体识别模型中的特征提取网络。在特征提取网络对图像进行特征提取之后，再增加手部、手心等部位的回归网络，实现对肢体中局部部位的提取，这样既可以采用肢体识别模型中的特征提取网络提取肢体特征，将肢体识别模型学习到的肢体特征应用到局部部位的对象识别网络的学习过程，既提高肢体的局部部位的提取精度，又可以降低目标识别模型训练的复杂度。
本公开的实施例中，可以获取肢体识别模型，肢体识别模型可以包括参数已知的肢体特征提取网络。此外，肢体识别模型还可以包括参数已知的肢体识别网络，肢体识别网络可以对肢体特征提取网络提取的特征进行肢体整体的识别。肢体识别模型可以为训练获得的模型，在肢体识别模型的基础上可以复用特征提取网络。通过将训练图像输入到肢体特征提取网络，可以获得图像肢体特征，图像肢体特征可以用于对象识别网络的对象识别，并通过获得的对象识别结果对对象识别网络进行更新，使得可以重新利用肢体特征提取网络。对象识别网络识别的对象可以为待识别图像中局部肢体部位的表征，例如表征手部的手框、手心等。通过固定肢体特征提取网络的方式，可以直接使用肢体特征提取网络提取相应的特征，直接利用提取到的肢体特征进行对象识别网络的训练，可以极大减少目标识别模型的构建难度。同时，利用肢体特征提取网络和完成更新的对象识别网络可以构建目标识别模型，通过目标识别模型可以更准确地提取肢体的局部部位，快速而准确地获得目标识别模型。本技术方案可以达到即提高训练效率又能有效提升训练精度的效果。
下面将以具体实施例对本公开的技术方案以及本公开的技术方案如何解决上述技术问题进行详细说明。下面几个具体实施例可以相互结合，对于相同或相似的概念或过程可 能在某些实施例中不再赘述。下面将结合附图对本发明的实施例进行详细描述。
图1是根据本公开对象识别方法的一个应用网络架构图。根据本公开实施例的应用网络架构中可以包括一个电子设备以及一个与该电子设备通过局域网或者广域网进行网络连接的客户端，假设该电子设备可以为个人计算机、普通服务器，超级个人计算机，云服务器等类型的服务器，本公开中对电子设备的具体类型并不作出过多限定。客户端例如可以为手机、平板电脑、个人计算机、智能家电、可穿戴设备等终端设备，本公开中对客户端的具体类型并不作出过多限定。如图1所示，以电子设备为云服务器1，第一客户端2为手机21、平板电脑22为例，云服务器1中可以基于本公开的技术方案对目标识别模型进行训练。任一个第一客户端2可以向云服务器1请求获取训练结束的目标识别模型，例如客户端可以向云服务器发送识别请求，将获取训练结束的目标识别模型，当然，在实际应用中，云服务器还可以直接接收待识别图像，并基于本公开的对象识别方法识别待识别对象中的目标对象。
云服务器1采用本公开的技术方案进行目标识别模型构建时，可以将肢体识别模型中的肢体特征提取网络11和肢体识别网络12固定。在肢体特征提取网络11之后增加对象识别网络13，对象识别网络13例如可以为手框识别网络、手心识别网络中的一个或多个。对象识别网络13训练过程中，肢体特征提取网络11和肢体识别网络12固定，也即，肢体特征提取网络11、肢体识别网络12不参与训练，训练对象识别网络13即可。
参考图2，图2为本公开实施例提供的一种模型构建方法的一些实施例的流程图，该模型构建方法可以包括以下步骤201～步骤204。
在步骤201中，获取肢体识别模型，肢体识别模型包括参数已知的肢体特征提取网络，肢体识别网络用于识别肢体。
在一些实施例中，肢体识别模型可以用于识别图像中的肢体。图像中的肢体具体可以是指肢体的整体，可以包括肢体的整体区域和/或肢体关键点。肢体整体区域可以为能够包含整个人体的矩形框，肢体关键点可以包括肢体的各个局部部位所对应的位置点或坐标点。肢体的局部部位可以包括：肢体的手部、脚部、上下肢、头部等局部部位。
肢体识别模型可以预先训练获得。肢体识别模型可以包括参数已知的肢体特征提取网络和参数已知的肢体识别网络。肢体特征提取网络可以用于提取输入的图像的肢体特征。肢体识别网络可以对肢体特征进行回归计算，获得相应的人体框或者肢体关键点。肢体识别网络具体获得的信息可以根据任务需求设置。
在一些实施例中，肢体识别模型可以通过卷积神经网络、循环神经网络等类型的网络结构组建获得，本实施例中对肢体识别网络所采用的具体算法并不做出过多限定。
在步骤202中，将训练图像输入肢体特征提取网络，获得图像肢体特征。
步骤202中的肢体特征提取网络为已训练获得的肢体特征提取网络。
训练图像可以包括二维训练图像和/或三维训练图像。肢体特征提取网络可以直接用于训练图像的肢体特征的提取。图像肢体特征可以为训练图像中肢体的整体特征。
在步骤203中，基于图像肢体特征，通过对象识别网络，确定训练图像的对象识别结果，并基于对象识别结果更新对象识别网络，对象识别网络识别的对象为肢体中的局部部位。
在一些实施例中，对象识别网络可以对输入的图像肢体特征进行局部肢体部位的识别，获得对象识别结果。对象识别结果即为训练图像中肢体的某个局部部位的识别结果。肢体的局部部位，例如可以包括肢体的手部、手部中心点、眼部和/或眼部中心点等部位，本实施例中对肢体的局部部位的具体类型并不作出过多限定。
基于对象识别结果，更新对象识别网络可以是基于对象识别结果确定识别误差，根据识别误差判断对象识别网络是否满足收敛条件，响应于判断对象识别满足收敛条件，确定当前对象识别网络为完成更新的对象识别网络，响应于判断对象识别不满足收敛条件，基于识别误差更新对象识别网络，返回至步骤203继续执行。
在步骤204中，基于肢体特征提取网络和完成更新的对象识别网络，构建目标识别模型，目标识别模型用于识别目标图像中肢体的局部部位。
在一些实施例中，目标识别模型可以包括肢体特征提取网络和完成更新的对象识别网络。肢体特征提取网络和完成更新的对象识别网络可以用于执行肢体的局部部位的识别，例如肢体手部的识别、手部中心点等局部肢体部位的识别。
在一些实施例中，目标识别模型的应用步骤可以包括：通过肢体特征提取网络提取输入的图像的特征，将肢体特征提取网络输出的特征输入到完成更新的对象识别网络，通过对象识别网络识别肢体的局部部位，实现肢体特征提取网络和对象识别网络的综合应用。
在一些实施例中，目标识别模型还可以包括：在肢体特征提取网络和对象识别网络之间增加特征增强网络，通过特征增强网络对肢体特征提取网络的输出特征进行特征增强，将增强后的特征输入到对象识别网络进行肢体的局部部位的识别。
本公开实施例中，可以获取肢体识别模型，肢体识别模型可以包括参数已知的肢体特征提取网络。此外，肢体识别模型还可以包括参数已知的肢体识别网络，肢体识别网络可以对肢体特征提取网络提取的特征进行肢体整体的识别。肢体识别模型可以为已训练获得的模型，在肢体识别模型的基础上可以复用特征提取网络。通过将训练图像输入到肢体特征提取网络，可以获得图像肢体特征，图像肢体特征可以用于对象识别网络的对象识别， 并通过获得的对象识别结果对对象识别网络进行更新，使得可以重新利用肢体特征提取网络。对象识别网络识别的对象可以为待识别图像中局部肢体部位的表征，例如表征手部的手框、手心等。通过固定肢体特征提取网络的方式，可以直接使用肢体特征提取网络提取相应的特征，直接利用提取到的肢体特征进行对象识别网络的训练，可以极大减少目标识别模型的构建难度。同时，利用肢体特征提取网络和完成更新的对象识别网络可以构建目标识别模型，通过目标识别模型可以更准确地提取肢体的局部部位，快速而准确地获得目标识别模型。本技术方案可以达到即提高训练效率又能有效提升训练精度的效果。
此外，由于目标识别模型中肢体特征提取网络的参数是已知的，不需要再进行额外的特征网络训练。在肢体特征提取网络的基础上，对对象识别网络进行单独训练即可，减少了需要训练的网络的复杂度，因此，对于较为简单的网络训练，采用少量的具备局部肢体标签的训练图像即可。通过本公开的模型构建方法，通过少量的标注样本，也即训练图像即可以完成对象识别网络的高精度训练，在准确对对象识别网络的基础上，获得肢体的局部部位识别精度较高的目标识别模型，进而可以以较低的标注成本达到较高的模型训练效果。
如图3所示，为本公开实施例提供的一种模型构建方法的又一些实施例的流程图，与前述实施例的不同之处在于，基于图像肢体特征，通过对象识别网络，确定训练图像的对象识别结果，可以包括步骤301～步骤302。
在步骤301中，为对象识别网络确定候选参数；候选参数为对象识别网络中各网络函数的参数值。
在一些实施例中，候选参数可以为对象识别网络中各网络函数的参数值。对象识别网络可以包括多个网络函数，网络函数例如可以为激活函数、激励函数等。在一些实施例中，各个网络函数分别对应有权重和偏移量。网络函数一般可以表示为：y＝ω*f(x)+b,其中，f(x)为网络函数，ω为网络函数的权重，b为网络函数的偏移量或者偏置量。候选参数可以包括权重的取值和偏移量的取值。
候选参数可以通过参数初始化获得，参数初始化例如可以是随机确定各网络函数对应的参数值，或者还可以获取历史参数的方式作为候选参数。在一轮训练过程中，对象识别网络的候选参数可以维持不变，在一轮训练结束，可以更新对象识别网络的候选参数以进行网络的迭代训练。
在步骤302中，基于候选参数对应的对象识别网络，确定图像肢体特征对应的对象识别结果。
肢体特征提取网络可以为目标识别模型中的编码器，通过肢体特征提取网络可以准确 提取图像中的肢体特征。在对象识别模型构建过程中，肢体特征提取网络已知，可以通过肢体特征提取网络提取训练图像的图像肢体特征。图像肢体特征可以为表征训练图像中肢体特点的向量或者矩阵。
步骤302具体可以包括：将图像肢体特征输入候选参数对应的对象识别网络，获得对象识别结果。此外，还可以将图像肢体特征进行特征增强，将增强后的图像肢体特征输入到候选参数对应的对象识别网络，获得对象识别结果。还可以对图像肢体特征进行其他类型的处理后输入到候选参数对应的对象识别网络。
与前述实施例的不同之处在于，步骤203中基于对象识别结果更新对象识别网络，还可以包括步骤303～步骤305。
在步骤303中，根据对象识别结果，判断对象识别网络是否满足收敛条件。响应于判断对象识别网络满足收敛条件，执行步骤304，响应于判断对象识别网络不满足收敛条件，执行步骤305。
在步骤304中，确定候选参数为目标参数，确定目标参数对应的对象识别网络为完成更新的对象识别网络。
在步骤305中，更新候选参数，返回步骤302继续执行。
训练图像可以关联有识别结果标签，识别结果标签可以为图像中局部肢体的标注结果。例如，对象识别网络识别的局部部位为手时，识别结果标签可以为手部所在真实矩形框的标注结果，简称手框标签。对象识别网络识别的局部部位为手心时，识别结果标签可以为手心所对应的真实坐标点的标注结果。
根据对象识别结果，确定对象识别网络是否满足收敛条件可以包括：通过训练图像对应的对象识别结果和识别结果标签，确定识别误差，根据识别误差判断对象识别网络是否满足收敛条件，响应于根据识别误差判断对象识别网络满足收敛条，确定当前对象识别网络为完成更新的对象识别网络，响应于根据识别误差判断对象识别网络不满足收敛条，基于识别误差更新对象识别网络。
对象识别网络可以对图像肢体特征进行特征回归和分类，获得相应的对象识别结果。对象识别结果可以包括所在区域所对应的矩形框或者对象所在位置点。对象识别结果的结果种类具体可以根据识别对象的对象特性确定。例如，对象识别结果可以为手框。
对象识别结果可以用于对象识别网络的损失计算，获得识别误差。根据识别误差判断对象识别网络是否满足收敛条件，可以包括：判断识别误差是否小于损失阈值，响应于判断识别误差小于损失阈值，确定识别误差满足收敛条件，响应于判断识别误差大于或等于损失阈值，确定识别误差不满足收敛条件。
在一些实施例中，更新候选参数可以包括：基于梯度下降算法和识别误差，对候选参数进行更新，获得更新后的候选参数。
本实施例中，在训练对象识别网络时，可以利用肢体特征提取网络提取训练图像的图像肢体特征，利用候选参数对应的对象识别网络对图像肢体特征执行肢体的局部部位的识别，获得对象识别结果。利用对象识别结果对对象识别网络进行收敛条件检测。通过收敛条件可以对对象识别网络当前的候选参数进行使用效果评价，使得对象识别网络在训练过程中应用到肢体特征提取网络，减少需要训练的网络复杂度，提高对象识别网络的训练效率。此外，还通过收敛条件对对象识别网络是否终止训练进行判断，准确监控对象识别网络的训练过程，提高对象识别网络的训练精度。
在一些实施例中，与前述实施例的不同之处在于，对象识别网络包括手框识别网络；基于图像肢体特征，通过对象识别网络，确定训练图像的对象识别结果，可以包括：
基于手框参数对应的手框识别网络，确定图像肢体特征对应的手框识别结果，手框参数为手框识别网络的候选参数。
基于对象识别结果更新对象识别网络，包括：
根据手框识别结果，判断对象识别网络是否满足收敛条件；
响应于根据手框识别结果判断对象识别网络满足收敛条件，确定手框参数对应手框识别网络为完成更新的对象识别网络；
响应于根据手框识别结果判断对象识别网络不满足收敛条件，更新手框参数，返回基于手框参数对应的手框识别网络，确定图像肢体特征对应的手框识别结果继续执行。
在一些实施例中，基于手框参数对应的手框识别网络，确定图像肢体特征对应的手框识别结果，手框参数为手框识别网络的候选参数，可以包括：将图像肢体特征输入手框参数对应的手框识别网络，获得手框识别结果。还可以将图像肢体特征进行特征增强，将特征增强后的图像肢体特征输入到手框参数对应的手框识别网络，获得手框识别结果。
在一些实施例中，手框识别网络可以用于识别图像肢体特征中的手框。手框识别结果可以包括图像中的手部所在区域对应的矩形框，可以称为手框。
在一些实施例中，除手框识别网络之外，对象识别网络还可以包括其它人体部位的识别网络，例如，手心识别网络、人脸识别网络等，具体可以根据识别网络的识别目的或者识别需求确定。
本实施例中，在图像肢体特征的基础上，可以基于手框参数对应的手框识别网络进行手框识别，以获得手框识别结果。通过手框识别结果可以判断对象识别网络是否满足收敛条件，在对象识别网络不满足收敛条件时，可以更新手框参数，也即实现对象识别网络的 更新，在对象识别网络满足收敛条件时，可以确定手框参数对应的手框识别网络为完成更新的对象识别网络。通过手框识别结果对手框参数对应的手框识别网络是否满足收敛条件进行判断，进而在手框识别网络在不满足收敛条件时执行参数更新，可以实现手框识别网络的迭代更新，获得准确的手框识别网络。
在一些实施例中，对象识别网络还可以包括：手心识别网络和手框识别网络。
基于图像肢体特征，通过对象识别网络，确定训练图像的对象识别结果，包括：
基于手框参数对应的手框识别网络，确定图像肢体特征对应的手框识别结果，手框参数为手框识别网络的候选参数；
基于手心参数对应的手心识别网络，确定图像肢体特征对应的手心识别结果，手心参数为手心识别网络的候选参数。
基于对象识别结果更新对象识别网络，包括：
根据手框识别结果和手心识别结果，判断对象识别网络是否满足收敛条件；
响应于根据手框识别结果和手心识别结果判断对象识别网络满足收敛条件，确定手框参数对应的手框识别网络和手心参数对应的手心参数对应的手心识别网络为完成更新的对象识别网络；
响应于根据手框识别结果和手心识别结果判断对象识别网络不满足收敛条件，更新手框参数和手心参数，返回基于手框参数对应的手框识别网络，确定图像肢体特征对应的手框识别结果继续执行。
在一些实施例中，对象识别结果可以包括手框识别结果和手心识别结果。手心识别网络可以用于识别人体的手部中心点，手心识别结果可以包括手心位置坐标，位置坐标可以指手心在训练图像或者待识别图像中的坐标。
为了便于理解，如图4所示，为本实施例提供的一种目标识别模型的示例图。参考图4，目标识别模型可以包括肢体特征提取网络401。肢体特征提取网络401的输入可以为图像，输出可以分别连接手框识别网络402、手心识别网络403和肢体识别网络404。肢体特征提取网络401提取的特征可以分别输入到手框识别网络402、手心识别网络403和肢体识别网络404，以获得手框识别网络402输出的手框识别结果，手心识别网络403输出的手心识别结果，和肢体识别网络404输出的肢体识别结果，肢体识别结果例如可以包括肢体关键点或者肢体区域。
本实施例中，除手框识别网络之外，对象识别网络还可以包括手心识别网络。通过对手心识别网络的设置可以进一步实现对图像肢体特征的有效利用。同时，在手框识别网络的基础上，增加手心识别网络，可以联合手部的两个关键识别对象，也即手框和手心点两 者联合进行训练，使得手框识别网络和手心识别网络的训练过程互相影响，提高手框识别网络和手心识别网络的识别准确度。
在一些实施例中，根据手框识别结果和手心识别结果，判断对象识别网络是否满足收敛条件，包括：
根据手框识别结果和训练图像的手框标签，计算第一损失值；
根据手心识别结果和训练图像的手心标签，计算第二损失值；
将第一损失值和第二损失值进行加权求和，获得目标损失值；
响应于确定目标损失值满足损失阈值，确定对象识别网络满足收敛条件；
响应于确定目标损失值不满足损失阈值，确定对象识别网络不满足收敛条件。
在一些实施例中，第一损失值可以通过手框识别结果和手框标签进行差异计算获得。例如，可以计算手框识别结果和手框标签之间的距离，获得第一损失值。
在一些实施例中，第二损失值可以通过手心识别结果和手心标签进行差异计算获得。例如，可以计算手心识别结果和手心标签之间的距离，获得第二损失值。
目标损失值可以为第一损失值和第二损失值加权求和计算获得。可以确定手框识别网络的第一权重和手心识别网络的第二权重，通过第一权重和第一损失值相乘计算获得第一数值，通过第二权重和第二损失值相乘计算获得第二数值，计算第一数值和第二数值的和获得目标损失值。
在一些实施例中，第一权重和第二权重可以为预先设置的常数。在另一些实施例中，为了提高模型构建精度，可以将第一权重和第二权重作为待训练参数，参与手框识别网络和手心识别网络的训练，获得最后一次迭代获得的权重值为第一权重和第二权重的训练结果。
在一些实施例中，目标损失值满足损失阈值可以指目标损失值小于或等于损失阈值。目标损失值不满足损失阈值可以指目标损失值大于损失阈值。
本实施例中，可以通过分别计算手框识别网络的第一损失值和手心识别网络的第二损失值的方式，对手框识别网络和手心识别网络的训练效果分别进行评价。此外，还通过将第一损失值和第二损失值进行加权求和的方式进行最终的目标损失值的计算，实现将两个损失值进行综合损失计算，使得手框识别网络和手心识别网络两个网络能够联合进行收敛条件的判断，可以同时训练手框识别网络和手心识别网络。同时，手心识别网络和手框识别网络两个网络相互影响，提高网络识别精度。
在一些实施例中，在上述任一实施例的基础上，训练图像在对象识别网络对应的识别结果标签的确定步骤包括：
确定训练图像训练肢体识别模型时的肢体关键点标签；
根据肢体关键点标签，生成训练图像在对象识别网络对应的识别结果标签。
在一些实施例中，识别结果标签可以根据对象识别网络的识别对象确定。例如，对象识别网络的识别对象为手心时，识别结果标签可以为手心标签。对象识别网络的识别对象为手框时，识别结果标签可以为手框标签。
在一些实施例中，可以根据肢体关键点标签，生成训练图像在对象识别网络的手心标签。可以根据肢体关键点标签，生成训练图像在对象识别网络的手框标签。
本公开实施例中，通过获取已有的训练图像训练肢体识别模型时的肢体关键点标签，并根据肢体关键点标签生成训练图像的在对象识别网络对应的识别结果标签，例如依据肢体关键点与识别对象的位置约束关系，通过插值、拟合等方式可以利用已有的肢体关键点标签来实现肢体的局部部位对应的识别结果标签的标注，特别是当识别对象对应的识别结果标签数量有限时，可以通过肢体关键点标签的再次利用提高标注效率，丰富标签数量。
在一些实施例中，将第一损失值和第二损失值进行加权求和，获得目标损失值之后，还包括：
根据目标损失值，更新手框参数和手心参数。
在一些实施例中，可以采用目标损失值，结合梯度下降算法，更新手框参数和手心参数。当然，还可以采用其它参数更新算法，例如随机梯度下降算法等，本实施例中对参数更新所采用的具体算法并不做出过多限定。
本实施例中，通过此次迭代的目标损失值来更新手心参数和手框参数，可以使得各次迭代产生的损失参与到手框识别网络和手心识别网络的参数更新过程中，可以获得更准确的手心参数和手框参数。
在一些实施例中，根据手框识别结果和训练图像的手框标签，计算第一损失值，包括：
利用回归损失函数，计算手框识别结果和训练图像的手框标签对应的第一损失值；
根据手心识别结果和训练图像的手心标签，计算第二损失值，包括：
利用均方误差损失函数，计算手心识别结果和训练图像的手心标注对应的第二损失值。
回归损失函数可以包括平均绝对误差(也称，L1损失函数)、均方误差损失函数(也称，L2损失函数)、平滑平均绝对误差(Huber平滑L1损失函数)等损失函数中的任一种，本实施例中对回归损失函数所采用的函数类型并不做出过多限定。
均方误差损失函数(MSE，mean squared error)也称L2损失函数，可以对预测值与目标值之间差值的平方和进行误差计算。关于回归损失函数、均方误差损失函数的具体计算方式，可以参考相关技术中的内容，在此不再赘述。
本实施例中，可以利用回归损失函数计算第一损失值，利用均方误差函数计算第二损失值，通过不同损失函数计算不同网络的误差，实现对不同网络的针对性损失计算，使得选择的损失函数更适配相应的识别网络，提高损失计算的准确度。
在一些实施例中，在上述任一实施例的基础上，手框识别网络可以包括全局平均池化层和全连接层。手心识别网络可以基于热力图算法构造获得。
全局平均池化层可以通过全局平均池化算法构造获得，全局平均池化算法(global average pooling)可以对每个通道输入的原始数据进行均值计算，得到每个通道对应的均值，将每个通道对应的均值作为全连接层的输入。通过全局平均池化算法可以降低各通道的数据结构的复杂度，提高网络计算速度。
全连接层可以通过全连接算法(Fully Connected，FC)构造获得，可以用于手框的分类计算，通过全连接层可以实现手框的快速而准确的分类。热力图(heatmap)算法可以用于识别输入特征的手心点。上述算法具体实施时，可以实现相应对象的识别。
为了获得准确的肢体识别模型，本方案中，可以训练获得肢体识别模型，肢体识别模型的训练步骤可以包括：
基于二维肢体标注数据，训练获得肢体识别模型；
或者，基于二维肢体标注数据和三维肢体标注数据，训练获得肢体识别模型。
二维肢体标注数据可以包括训练图像和训练图像的二维标注数据。
三维肢体标注数据可以包括训练的图像和训练图像的三维标注数据。
二维标注数据可以指以二维数据方式表示的数据，例如二维坐标点、二维矩形框等。三维标注数据可以指以除二维数据之外，还可以包括图像的深度数据，例如，在二维坐标点的基础上增加深度数据，在二维矩形框的基础上，增加二维矩形框的深度数据。
肢体识别模型可以对图像进行肢体识别，获得肢体识别结果。本案中涉及的肢体包括但不限于肢体关键点、肢体区域等。当然，在实际应用中，肢体除人体外，还可以包括卡通形象、动物等作为待识别的主体。
本实施例中，可以使用二维肢体标注数据对肢体识别模型进行训练，或者还可以使用二维肢体标注数据和三维肢体标注数据对肢体识别模型进行训练。通过二维或者二维基础上结合三维的肢体标注数据对肢体识别模型进行训练，可以使得肢体识别模型能够对图像进行准确的肢体识别，提高肢体识别模型的训练效率和准确性。
为了实现完整的肢体识别过程。在某些实施例中，肢体识别模型还包括：参数已知的肢体识别网络；
基于肢体特征提取网络和完成更新的对象识别网络，构建目标识别模型，包括：
基于肢体特征提取网络、肢体识别网络和完成更新的对象识别网络，确定目标识别模型，目标识别模型用于执行目标图像中肢体和肢体的局部部位的识别。
在目标识别模型构建过程中，目标识别模型中的肢体特征提取网络和肢体识别网络的参数不变，肢体特征提取网络和肢体识别网络的参数未执行迭代更新，而对象识别网络的参数在训练时不断迭代更新的，直至满足收敛条件。
本实施例中，获得肢体识别模型即可以获得肢体特征提取网络和肢体识别网络，在构建目标识别模型时，可以将肢体识别网络作为目标识别模型的识别分支的一种，在肢体识别网络的基础上增加对象识别网络，在同一肢体特征提取网络的基础上，增加识别网络分支，可以实现多任务识别，且多任务可以共享肢体特征提取网络(backbone)。无需针对肢体和肢体的局部部位的识别单独训练特征提取网络，可以减少模型训练的复杂度，降低模型训练耗时，提高模型获取效率和精度。
如图5所示，为本公开实施例提供的一种对象识别方法的一些实施例的流程图，该对象识别方法可以包括以下几个步骤501～步骤504。
在步骤501中，响应于图像识别请求，接收待识别图像。
在步骤502中，获取训练获得的目标识别模型，目标识别模型包括参数已知的肢体特征提取网络和训练获得的对象识别网络。
在步骤503中，基于参数已知的肢体特征提取网络，提取获得待识别图像的图像肢体特征。
在步骤504中，基于训练获得的对象识别网络，确定待识别图像的图像肢体特征对应的对象识别结果，目标识别模型是采用如上述任意一些实施例的模型构建方法获得的。
对象识别结果包括肢体和肢体局部部位的识别结果。
在一些实施例中，对象识别网络可以包括：手心识别网络和/或手框识别网络。待识别图像的图像肢体特征对应的对象识别结果可以包括：手心识别网络识别待识别图像的图像肢体特征获得的手心识别结果，例如可以包括手心点坐标和/或手框识别网络识别待识别图像的图像肢体特征获得的手框识别结果，例如可以包括手框。
在一些实施例中，对象识别网络还可以包括：肢体识别网络。待识别图像的图像肢体特征对应的对象识别结果还可以包括：肢体识别网络识别待识别图像的图像肢体特征获得的肢体识别结果，例如包括肢体关键点或者肢体区域对应的矩形框。
本实施例中，可以响应于图像识别请求，接收待识别图像。通过获取训练得到的目标识别模型，可以利用目标识别模型中的肢体特征提取网络提取待识别图像的图像肢体特征。通过对象识别网络，可以获得待识别图像的图像肢体特征对应的对象识别结果，实现对肢 体对象的准确识别。
如图6所示，为本公开实施例提供的一种模型构建装置的一些实施例的结构示意图，该模型构建装置600可以包括第一获取单元601、特征提取单元602、模型训练单元603和模型构建单元604。
第一获取单元601用于获取肢体识别模型，肢体识别模型包括参数已知的肢体特征提取网络，肢体识别模型用于识别肢体。
特征提取单元602用于将训练图像输入肢体特征提取网络，获得图像肢体特征。
模型训练单元603用于基于图像肢体特征，通过对象识别网络，确定训练图像的对象识别结果，并基于对象识别结果更新对象识别网络，对象识别网络识别的对象为肢体中的局部部位。
模型构建单元604用于基于肢体特征提取网络和完成更新的对象识别网络，构建目标识别模型，目标识别模型用于执行目标图像中肢体的局部部位的识别。
在一些实施例中，模型训练单元可以包括：
第一确定模块，用于为对象识别网络确定候选参数，候选参数为对象识别网络中各网络函数的参数值；
第一识别模块，用于基于候选参数对应的对象识别网络，确定图像肢体特征对应的对象识别结果；
第一处理模块，用于响应于根据对象识别结果，确定对象识别网络满足收敛条件，确定候选参数为目标参数，确定目标参数对应的对象识别网络为完成更新的对象识别网络；
第二处理模块，用于响应于根据对象识别结果，确定对象识别网络不满足收敛条件，更新候选参数，返回至基于候选参数对应的对象识别网络，确定图像肢体特征对应的对象识别结果继续执行。
在一些实施例中，对象识别网络包括手框识别网络；模型训练单元还可以包括：
第二确定模块，用于基于手框参数对应的手框识别网络，确定图像肢体特征对应的手框识别结果，手框参数为手框识别网络的候选参数；
第一判断模块，用于根据手框识别结果，判断对象识别网络是否满足收敛条件；响应于根据手框识别结果判断对象识别网络满足收敛条件，确定手框参数对应手框识别网络为完成更新的对象识别网络；响应于根据手框识别结果判断对象识别网络不满足收敛条件，更新手框参数，返回基于手框参数对应的手框识别网络，确定图像肢体特征对应的手框识别结果继续执行。
在一些实施例中，对象识别网络包括：手心识别网络和手框识别网络。模型训练单元 还可以包括：
第三确定模块，用于基于手框参数对应的手框识别网络，确定图像肢体特征对应的手框识别结果，手框参数为手框识别网络的候选参数；
第四确定模块，用于基于手心参数对应的手心识别网络，确定图像肢体特征对应的手心识别结果，手心参数为手心识别网屏络的候选参数；
第二判断模块，用于根据手框识别结果和手心识别结果，判断对象识别网络是否满足收敛条件；响应于根据手框识别结果和手心识别结果判断对象识别网络满足收敛条件，确定手框参数对应的手框识别网络和手心参数对应的手心参数对应的手心识别网络为完成更新的对象识别网络；响应于根据手框识别结果和手心识别结果判断对象识别网络不满足收敛条件，更新手框参数和手心参数，返回基于手框参数对应的手框识别网络，确定图像肢体特征对应的手框识别结果继续执行。
在一些实施例中，第二判断模块包括：
第一计算子模块，用于根据手框识别结果和训练图像的手框标签，计算第一损失值；
第二计算子模块，用于根据手心识别结果和训练图像的手心标签，计算第二损失值；
目标计算子模块，用于将第一损失值和第二损失值进行加权求和，获得目标损失值；
第一判断子模块，用于响应于确定目标损失值满足损失阈值，确定对象识别网络满足收敛条件；
第二判断子模块，用于响应于确定目标损失值不满足损失阈值，确定对象识别网络不满足收敛条件。
在一些实施例中，第二判断模块还包括：
标签确定单元，用于确定训练图像训练肢体识别模型时的肢体关键点标签。
标签生成单元，用于根据肢体关键点标签，生成训练图像在对象识别网络对应的识别结果标签。
在一些实施例，第二判断模块还包括：
第一训练单元，用于基于二维肢体标注数据，训练获得肢体识别模型；
或者，第二训练单元，用于基于二维肢体标注数据和三维肢体标注数据，训练获得肢体识别模型。
在一些实施例，肢体识别模型还包括：参数已知的肢体识别网络；
模型构建单元，具体用于：
基于肢体特征提取网络、肢体识别网络和待训练的对象识别网络，构建目标识别模型。
在一些实施例中，模型构建单元，具体可以用于：
基于肢体特征提取网络、肢体识别网络和完成更新的对象识别网络，确定目标识别模型，目标识别模型用于执行目标图像中肢体和肢体的局部部位的识别。
如图7所示，为本公开实施例提供的一种对象识别装置的一些实施例的结构示意图，该对象识别装置700可以包括图像接收单元701、第二获取单元702、特征提取单元703和对象识别单元704。
图像接收单元701用于响应于图像识别请求，接收待识别图像。
第二获取单元702用于获取训练获得的目标识别模型，目标识别模型包括参数已知的肢体特征提取网络和完成更新的对象识别网络。
特征提取单元703用于基于参数已知的肢体特征提取网络，提取获得待识别图像的图像肢体特征。
对象识别单元704用于基于训练获得的对象识别网络，确定待识别图像的图像肢体特征对应的对象识别结果，目标识别模型是采用上述任一些实施例所提供的模型构建方法获得的。
本实施例提供的对象识别装置，可用于执行上述方法实施例的技术方案，其实现原理和技术效果类似，本实施例此处不再赘述。
为了实现上述实施例，本公开实施例还提供了一种电子设备。
参考图8，其示出了适于用来实现本公开实施例的电子设备800的结构示意图，该电子设备800可以为终端设备或服务器。终端设备可以包括但不限于诸如移动电话、笔记本电脑、数字广播接收器、个人数字助理(Personal Digital Assistant，简称PDA)、平板电脑(Portable Android Device，简称PAD)、便携式多媒体播放器(Portable Media Player，简称PMP)、车载终端(例如车载导航终端)等等的移动终端以及诸如数字TV、台式计算机等等的固定终端。图8示出的电子设备仅仅是一个示例，不应对本公开实施例的功能和使用范围带来任何限制。
如图8所示，电子设备800可以包括处理装置(例如中央处理器、图形处理器等)801，其可以根据存储在只读存储器(Read Only Memory，简称ROM)802中的程序或者从存储装置808加载到随机访问存储器(Random Access Memory，简称RAM)803中的程序而执行各种适当的动作和处理。在RAM 803中，还存储有电子设备800操作所需的各种程序和数据。处理装置801、ROM 802以及RAM 803通过总线804彼此相连。输入/输出(I/O)接口805也连接至总线804。
通常，以下装置可以连接至I/O接口805：包括例如触摸屏、触摸板、键盘、鼠标、摄像头、麦克风、加速度计、陀螺仪等的输入装置806；包括例如液晶显示器(Liquid Crystal  Display，简称LCD)、扬声器、振动器等的输出装置807；包括例如磁带、硬盘等的存储装置808；以及通信装置809。通信装置809可以允许电子设备800与其他设备进行无线或有线通信以交换数据。虽然图8示出了具有各种装置的电子设备800，但是应理解的是，并不要求实施或具备所有示出的装置。可以替代地实施或具备更多或更少的装置。
特别地，根据本公开的实施例，上文参考流程图描述的过程可以被实现为计算机软件程序。例如，本公开的实施例包括一种计算机程序产品，其包括承载在计算机可读介质上的计算机程序，该计算机程序包含用于执行流程图所示的方法的程序代码。在这样的实施例中，该计算机程序可以通过通信装置809从网络上被下载和安装，或者从存储装置808被安装，或者从ROM 802被安装。在该计算机程序被处理装置801执行时，执行本公开实施例的方法中限定的上述功能。
需要说明的是，本公开上述的计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质或者是上述两者的任意组合。计算机可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。计算机可读存储介质的更具体的例子可以包括但不限于：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机访问存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑磁盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本公开中，计算机可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。而在本公开中，计算机可读信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式，包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读信号介质还可以是计算机可读存储介质以外的任何计算机可读介质，该计算机可读信号介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。计算机可读介质上包含的程序代码可以用任何适当的介质传输，包括但不限于：电线、光缆、RF(射频)等等，或者上述的任意合适的组合。
上述计算机可读介质可以是上述电子设备中所包含的；也可以是单独存在，而未装配入该电子设备中。
上述计算机可读介质承载有一个或者多个程序，当上述一个或者多个程序被该电子设备执行时，使得该电子设备执行上述实施例所示的方法。
可以以一种或多种程序设计语言或其组合来编写用于执行本公开的操作的计算机程序代码，上述程序设计语言包括面向对象的程序设计语言—诸如Java、Smalltalk、C++， 还包括常规的过程式程序设计语言—诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络——包括局域网(Local Area Network，简称LAN)或广域网(Wide Area Network，简称WAN)—连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。
附图中的流程图和框图，图示了按照本公开各种实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分，该模块、程序段、或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意，在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个接连地表示的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或操作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
描述于本公开实施例中所涉及到的单元可以通过软件的方式实现，也可以通过硬件的方式来实现。其中，单元的名称在某种情况下并不构成对该单元本身的限定，例如，第一获取单元还可以被描述为“获取至少两个网际协议地址的单元”。
本文中以上描述的功能可以至少部分地由一个或多个硬件逻辑部件来执行。例如，非限制性地，可以使用的示范类型的硬件逻辑部件包括：现场可编程门阵列(FPGA)、专用集成电路(ASIC)、专用标准产品(ASSP)、片上系统(SOC)、复杂可编程逻辑设备(CPLD)等等。
在本公开的上下文中，机器可读介质可以是有形的介质，其可以包含或存储以供指令执行系统、装置或设备使用或与指令执行系统、装置或设备结合地使用的程序。机器可读介质可以是机器可读信号介质或机器可读储存介质。机器可读介质可以包括但不限于电子的、磁性的、光学的、电磁的、红外的、或半导体系统、装置或设备，或者上述内容的任何合适组合。机器可读存储介质的更具体示例会包括基于一个或多个线的电气连接、便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦除可编程只读存储器(EPROM或快闪存储器)、光纤、便捷式紧凑盘只读存储器(CD-ROM)、光学储存设备、磁储存设备、或上述内容的任何合适组合。
本公开实施例提供一种模型构建、对象识别方法、装置、设备、介质及产品，以克服 目前手框训练困难，识别准确率较低的问题。
第一方面，根据本公开的一个或多个实施例，提供了一种模型构建方法，包括：
获取肢体识别模型，肢体识别模型包括参数已知的肢体特征提取网络，肢体识别模型用于识别肢体；
将训练图像输入肢体特征提取网络，获得图像肢体特征；
基于图像肢体特征，通过对象识别网络，确定训练图像的对象识别结果，并基于对象识别结果更新对象识别网络，对象识别网络识别的对象为肢体中的局部部位；
基于肢体特征提取网络和完成更新的对象识别网络，构建目标识别模型，目标识别模型用于执行目标图像中肢体的局部部位的识别。
根据本公开的一个或多个实施例，基于图像肢体特征，通过对象识别网络，确定训练图像的对象识别结果，包括：
为对象识别网络确定候选参数，候选参数为对象识别网络中各网络函数的参数值；
基于候选参数对应的对象识别网络，确定图像肢体特征对应的对象识别结果；
基于对象识别结果更新对象识别网络，包括：
响应于根据对象识别结果，确定对象识别网络满足收敛条件，确定候选参数为目标参数，确定目标参数对应的对象识别网络为完成更新的对象识别网络；
响应于根据对象识别结果，确定对象识别网络不满足收敛条件，更新候选参数，返回至基于候选参数对应的对象识别网络，确定图像肢体特征对应的对象识别结果继续执行。
根据本公开的一个或多个实施例，对象识别网络包括手框识别网络；
基于图像肢体特征，通过对象识别网络，确定训练图像的对象识别结果，包括：
基于手框参数对应的手框识别网络，确定图像肢体特征对应的手框识别结果，手框参数为手框识别网络的候选参数；
基于对象识别结果更新对象识别网络，包括：
根据手框识别结果，判断对象识别网络是否满足收敛条件；
响应于根据手框识别结果，判断对象识别网络满足收敛条件，确定手框参数对应手框识别网络为完成更新的对象识别网络；
响应于根据手框识别结果，判断对象识别网络不满足收敛条件，更新手框参数，返回基于手框参数对应的手框识别网络，确定图像肢体特征对应的手框识别结果继续执行。
根据本公开的一个或多个实施例，对象识别网络包括：手心识别网络和手框识别网络；
基于图像肢体特征，通过对象识别网络，确定训练图像的对象识别结果，包括：
基于手框参数对应的手框识别网络，确定图像肢体特征对应的手框识别结果，手框参 数为手框识别网络的候选参数；
基于手心参数对应的手心识别网络，确定图像肢体特征对应的手心识别结果，手心参数为手心识别网屏络的候选参数；
基于对象识别结果更新对象识别网络，包括：
根据手框识别结果和手心识别结果，判断对象识别网络是否满足收敛条件；
响应于根据手框识别结果和手心识别结果，判断对象识别网络满足收敛条件，确定手框参数对应的手框识别网络和手心参数对应的手心参数对应的手心识别网络为完成更新的对象识别网络；
响应于根据手框识别结果和手心识别结果，判断对象识别网络不满足收敛条件，更新手框参数和手心参数，返回基于手框参数对应的手框识别网络，确定图像肢体特征对应的手框识别结果继续执行。
根据本公开的一个或多个实施例，根据手框识别结果和手心识别结果，判断对象识别网络是否满足收敛条件，包括：
根据手框识别结果和训练图像的手框标签，计算第一损失值；
根据手心识别结果和训练图像的手心标签，计算第二损失值；
将第一损失值和第二损失值进行加权求和，获得目标损失值；
响应于确定目标损失值满足损失阈值，确定对象识别网络满足收敛条件；
响应于确定目标损失值不满足损失阈值，确定对象识别网络不满足收敛条件。
根据本公开的一个或多个实施例，训练图像在对象识别网络对应的识别结果标签的确定步骤包括：
确定训练图像训练肢体识别模型时的肢体关键点标签；
根据肢体关键点标签，生成训练图像在对象识别网络对应的识别结果标签。
根据本公开的一个或多个实施例，肢体识别模型的训练步骤，包括：
基于二维肢体标注数据，训练获得肢体识别模型；
或者，基于二维肢体标注数据和三维肢体标注数据，训练获得肢体识别模型。
根据本公开的一个或多个实施例，肢体识别模型还包括：参数已知的肢体识别网络；
基于肢体特征提取网络和完成更新的对象识别网络，构建目标识别模型，包括：
基于肢体特征提取网络、肢体识别网络和完成更新的对象识别网络，确定目标识别模型，目标识别模型用于执行目标图像中肢体和肢体的局部部位的识别。
第二方面，根据本公开的一个或多个实施例，提供了一种对象识别方法，包括：
响应于图像识别请求，接收待识别图像；
获取训练获得的目标识别模型，目标识别模型包括参数已知的肢体特征提取网络和训练获得的对象识别网络；对象识别网络识别的对象为肢体中的局部部位；
基于参数已知的肢体特征提取网络，提取获得待识别图像的图像肢体特征；
基于训练获得的对象识别网络，确定待识别图像的图像肢体特征对应的对象识别结果，目标识别模型是采用上述任一些实施例提供的模型构建方法获得的。
第三方面，根据本公开的一个或多个实施例，提供了一种模型构建装置，包括：
第一获取单元，用于获取肢体识别模型，肢体识别模型包括参数已知的肢体特征提取网络，肢体识别模型用于识别肢体；
特征提取单元，用于将训练图像输入肢体特征提取网络，获得图像肢体特征；
模型训练单元，用于基于图像肢体特征，通过对象识别网络，确定训练图像的对象识别结果，并基于对象识别结果更新对象识别网络，对象识别网络识别的对象为肢体中的局部部位；
模型构建单元，用于基于肢体特征提取网络和完成更新的对象识别网络，构建目标识别模型，目标识别模型用于执行目标图像中肢体的局部部位的识别。
第四方面，根据本公开的一个或多个实施例，提供了一种对象识别装置，包括：
图像接收单元，用于响应于图像识别请求，接收待识别图像；
第二获取单元，用于获取训练获得的目标识别模型，目标识别模型包括参数已知的肢体特征提取网络和完成更新的对象识别网络；
特征提取单元，用于基于参数已知的肢体特征提取网络，提取获得待识别图像的图像肢体特征；
对象识别单元，用于基于训练获得的对象识别网络，确定待识别图像的图像肢体特征对应的对象识别结果，目标识别模型是采用上述任一些实施例提供的模型构建方法获得的。
第五方面，根据本公开的一个或多个实施例，提供了一种电子设备，包括：至少一个处理器和存储器；
存储器存储计算机执行指令；
至少一个处理器执行存储器存储的计算机执行指令，使得至少一个处理器执行前述实施例提供的模型构建方法或对象识别方法。
第四方面，根据本公开的一个或多个实施例，提供了一种计算机可读存储介质，计算机可读存储介质中存储有计算机执行指令，当处理器执行计算机执行指令时，实现前述实施例提供的模型构建方法或对象识别方法。
第五方面，根据本公开的一个或多个实施例，提供了一种计算机程序产品，包括计算 机程序，计算机程序被处理器执行时实现前述实施例提供的模型构建方法或对象识别方法。
第六方面，根据本公开的一个或多个实施例，提供了一种计算机程序，包括：指令，所述指令当由处理器执行时使所述处理器执行根据前述实施例所述的模型构建方法或者对象识别方法。
本实施例提供的技术方案，可以获得肢体识别模型，肢体识别模型中的肢体特征提取网络的参数已知，可以直接用于提取训练图像的图像肢体特征。通过对象识别网络可以对图像肢体特征进行对象识别，获得训练图像的对象识别结果，该对象识别结果可以用于更新对象识别网络，对象识别网络可以为对象肢体中的局部部位。通过直接使用肢体特征提取网络提取相应的特征，不需要再对特征提取网络进行训练，可以极大减少模型构建难度。同时，利用肢体特征提取网络可以学习到的图像肢体特征，可以应用于对象识别网络的训练，可以使得对象识别网络学习图像肢体特征并进行对象识别，对象识别网络识别的对象可以为肢体的局部部位，通过复用肢体识别模型中的肢体特征提取参数，可以利用对象识别结果对对象识别网络进行更新，减少更新的网络数量，进而可以实现对对象识别网络快速而准确的训练。本技术方案可以达到即提高训练效率又能有效提升训练精度的效果。
以上描述仅为本公开的较佳实施例以及对所运用技术原理的说明。本领域技术人员应当理解，本公开中所涉及的公开范围，并不限于上述技术特征的特定组合而成的技术方案，同时也应涵盖在不脱离上述公开构思的情况下，由上述技术特征或其等同特征进行任意组合而形成的其它技术方案。例如上述特征与本公开中公开的(但不限于)具有类似功能的技术特征进行互相替换而形成的技术方案。
此外，虽然采用特定次序描绘了各操作，但是这不应当理解为要求这些操作以所示出的特定次序或以顺序次序执行来执行。在一定环境下，多任务和并行处理可能是有利的。同样地，虽然在上面论述中包含了若干具体实现细节，但是这些不应当被解释为对本公开的范围的限制。在单独的实施例的上下文中描述的某些特征还可以组合地实现在单个实施例中。相反地，在单个实施例的上下文中描述的各种特征也可以单独地或以任何合适的子组合的方式实现在多个实施例中。
尽管已经采用特定于结构特征和/或方法逻辑动作的语言描述了本主题，但是应当理解所附权利要求书中所限定的主题未必局限于上面描述的特定特征或动作。相反，上面所描述的特定特征和动作仅仅是实现权利要求书的示例形式。

Claims (16)

  1. 一种模型构建方法，包括：
    获取肢体识别模型，所述肢体识别模型包括参数已知的肢体特征提取网络，所述肢体识别模型用于识别肢体；
    将训练图像输入所述肢体特征提取网络，获得图像肢体特征；
    基于所述图像肢体特征，通过对象识别网络，确定所述训练图像的对象识别结果，并基于所述对象识别结果更新所述对象识别网络，所述对象识别网络识别的对象为肢体中的局部部位；
    基于所述肢体特征提取网络和完成更新的对象识别网络，构建目标识别模型，所述目标识别模型用于执行目标图像中肢体的局部部位的识别。
  2. 根据权利要求1所述的模型构建方法，其中，所述基于所述图像肢体特征，通过对象识别网络，确定所述训练图像的对象识别结果，包括：
    为所述对象识别网络确定候选参数，所述候选参数为所述对象识别网络中各网络函数的参数值；
    基于所述候选参数对应的所述对象识别网络，确定所述图像肢体特征对应的对象识别结果；
    所述基于所述对象识别结果更新所述对象识别网络，包括：
    响应于根据所述对象识别结果，确定所述对象识别网络满足收敛条件，确定所述候选参数为目标参数，确定所述目标参数对应的对象识别网络为所述完成更新的对象识别网络；
    响应于根据所述对象识别结果，确定所述对象识别网络不满足收敛条件，更新所述候选参数，返回至所述基于所述图像肢体特征，通过对象识别网络，确定训练图像的对象识别结果继续执行。
  3. 根据权利要求1所述的模型构建方法，其中，所述对象识别网络包括手框识别网络；
    所述基于所述图像肢体特征，通过对象识别网络，确定训练图像的对象识别结果，包括：
    基于手框参数对应的所述手框识别网络，确定所述图像肢体特征对应的手框识别结果，所述手框参数为所述手框识别网络的候选参数；
    所述基于所述对象识别结果更新所述对象识别网络，包括：
    根据所述手框识别结果，判断所述对象识别网络是否满足收敛条件；
    响应于根据所述手框识别结果，判断所述对象识别网络满足所述收敛条件，确定所述手框参数对应手框识别网络为所述完成更新的对象识别网络；
    响应于根据所述手框识别结果，判断所述对象识别网络不满足所述收敛条件，更新所述手框参数，返回所述基于手框参数对应的所述手框识别网络，确定所述图像肢体特征对应的手框识别结果继续执行。
  4. 根据权利要求1所述的模型构建方法，其中，所述对象识别网络包括：手心识别网络和手框识别网络；
    所述基于所述图像肢体特征，通过对象识别网络，确定训练图像的对象识别结果，包括：
    基于手框参数对应的所述手框识别网络，确定所述图像肢体特征对应的手框识别结果，所述手框参数为所述手框识别网络的候选参数；
    基于手心参数对应的所述手心识别网络，确定所述图像肢体特征对应的手心识别结果，所述手心参数为所述手心识别网络的候选参数；
    所述基于所述对象识别结果更新所述对象识别网络，包括：
    根据所述手框识别结果和所述手心识别结果，判断所述对象识别网络是否满足收敛条件；
    响应于根据所述手框识别结果和所述手心识别结果，判断所述对象识别网络满足所述收敛条件，确定所述手框参数对应的手框识别网络和所述手心参数对应的手心参数对应的手心识别网络为所述完成更新的对象识别网络；
    响应于根据所述手框识别结果和所述手心识别结果，判断所述对象识别网络不满足所述收敛条件，更新所述手框参数和所述手心参数，返回所述基于手框参数对应的所述手框识别网络，确定所述图像肢体特征对应的手框识别结果继续执行。
  5. 根据权利要求4所述的模型构建方法，其中，所述根据所述手框识别结果和所述手心识别结果，判断所述对象识别网络是否满足收敛条件，包括：
    根据所述手框识别结果和所述训练图像的手框标签，计算第一损失值；
    根据所述手心识别结果和所述训练图像的手心标签，计算第二损失值；
    将所述第一损失值和所述第二损失值进行加权求和，获得目标损失值；
    响应于确定所述目标损失值满足损失阈值，确定所述对象识别网络满足收敛条件；
    响应于确定所述目标损失值不满足所述损失阈值，确定所述对象识别网络不满足收敛条件。
  6. 根据权利要求1-5任一项所述的模型构建方法，还包括：确定所述训练图像在所述 对象识别网络对应的识别结果标签，包括：
    确定所述训练图像训练所述肢体识别模型时的肢体关键点标签；
    根据所述肢体关键点标签，生成所述训练图像在所述对象识别网络对应的识别结果标签。
  7. 根据权利要求1-5任一项所述的模型构建方法，其中，获取肢体识别模型包括训练获得所述肢体识别模型，包括：
    基于二维肢体标注数据，训练获得所述肢体识别模型；
    或者，基于二维肢体标注数据和三维肢体标注数据，训练获得所述肢体识别模型。
  8. 根据权利要求1-7任一项所述的模型构建方法，其中，所述肢体识别模型还包括：参数已知的肢体识别网络；
    所述基于所述肢体特征提取网络和完成更新的对象识别网络，构建目标识别模型，包括：
    基于所述肢体特征提取网络、所述肢体识别网络和所述完成更新的对象识别网络，确定目标识别模型，所述目标识别模型用于执行目标图像中肢体和肢体的局部部位的识别。
  9. 根据权利要求1-8任一项所述的模型构建方法，其中，所述目标识别模型包括特征增强网络，其中，所述特征增强网络对所述肢体特征提取网络的输出的所述图像肢体特征进行特征增强。
  10. 一种对象识别方法，包括：
    响应于图像识别请求，接收待识别图像；
    获取训练获得的目标识别模型，所述目标识别模型包括参数已知的肢体特征提取网络和训练获得的对象识别网络；对象识别网络识别的对象为肢体中的局部部位；
    基于所述参数已知的肢体特征提取网络，提取获得所述待识别图像的图像肢体特征；
    基于所述训练获得的对象识别网络，确定所述待识别图像的图像肢体特征对应的对象识别结果，所述目标识别模型是采用如权利要求1-9任一项所述的模型构建方法获得的。
  11. 一种模型构建装置，包括：
    第一获取单元，用于获取肢体识别模型，所述肢体识别模型包括参数已知的肢体特征提取网络，所述肢体识别模型用于识别肢体；
    特征提取单元，用于将训练图像输入所述肢体特征提取网络，获得图像肢体特征；
    模型训练单元，用于基于所述图像肢体特征，通过对象识别网络，确定所述训练图像的对象识别结果，并基于所述对象识别结果更新所述对象识别网络，所述对象识别网络识别的对象为肢体中的局部部位；
    模型构建单元，用于基于所述肢体特征提取网络和完成更新的对象识别网络，构建目标识别模型，所述目标识别模型用于执行目标图像中肢体的局部部位的识别。
  12. 一种对象识别装置，包括：
    图像接收单元，用于响应于图像识别请求，接收待识别图像；
    第二获取单元，用于获取训练获得的目标识别模型，所述目标识别模型包括参数已知的肢体特征提取网络和完成更新的对象识别网络；
    特征提取单元，用于基于所述参数已知的肢体特征提取网络，提取获得所述待识别图像的图像肢体特征；
    对象识别单元，用于基于所述训练获得的对象识别网络，确定所述待识别图像的图像肢体特征对应的对象识别结果，所述目标识别模型是采用如权利要求1-9任一项所述的模型构建方法获得的。
  13. 一种电子设备，包括：处理器、存储器；
    所述存储器存储计算机执行指令；
    所述处理器执行所述存储器存储的计算机执行指令，使得所述处理器配置有如权利要求1至9任一项所述的模型构建方法或者如权利要求10所述的对象识别方法。
  14. 一种计算机可读存储介质，其中，所述计算机可读存储介质中存储有计算机执行指令，当处理器执行所述计算机执行指令时，实现如权利要求1至9任一项所述的模型构建方法或者如权利要求10所述的对象识别方法。
  15. 一种计算机程序产品，包括计算机程序，其中，所述计算机程序被处理器执行，以配置有如权利要求1至9任一项所述的模型构建方法或者如权利要求10所述的对象识别方法。
  16. 一种计算机程序，包括：指令，所述指令当由处理器执行时使所述处理器执行根据权利要求1至9任一项所述的模型构建方法或者如权利要求10所述的对象识别方法。
PCT/CN2024/081599 2023-03-14 2024-03-14 模型构建、对象识别方法、装置、设备、介质及产品 Ceased WO2024188293A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202310267677.0A CN118675221A (zh) 2023-03-14 2023-03-14 模型构建、对象识别方法、装置、设备、介质及产品
CN202310267677.0 2023-03-14

Publications (1)

Publication Number Publication Date
WO2024188293A1 true WO2024188293A1 (zh) 2024-09-19

Family

ID=92726810

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2024/081599 Ceased WO2024188293A1 (zh) 2023-03-14 2024-03-14 模型构建、对象识别方法、装置、设备、介质及产品

Country Status (2)

Country Link
CN (1) CN118675221A (zh)
WO (1) WO2024188293A1 (zh)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113449610A (zh) * 2021-06-08 2021-09-28 杭州格像科技有限公司 一种基于知识蒸馏和注意力机制的手势识别方法和系统
CN113673313A (zh) * 2021-07-07 2021-11-19 华南理工大学 一种基于分层卷积神经网络的手势姿态识别方法
CN113762304A (zh) * 2020-11-26 2021-12-07 北京京东乾石科技有限公司 图像处理方法、图像处理装置和电子设备
CN114627503A (zh) * 2022-03-17 2022-06-14 广州虎牙科技有限公司 一种人手识别方法、装置、电子设备及存储介质
CN114912540A (zh) * 2022-05-30 2022-08-16 上海商汤智能科技有限公司 迁移学习方法、装置、设备及存储介质
US20230070008A1 (en) * 2020-02-17 2023-03-09 Snap Inc. Generating three-dimensional object models from two-dimensional images

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20230070008A1 (en) * 2020-02-17 2023-03-09 Snap Inc. Generating three-dimensional object models from two-dimensional images
CN113762304A (zh) * 2020-11-26 2021-12-07 北京京东乾石科技有限公司 图像处理方法、图像处理装置和电子设备
CN113449610A (zh) * 2021-06-08 2021-09-28 杭州格像科技有限公司 一种基于知识蒸馏和注意力机制的手势识别方法和系统
CN113673313A (zh) * 2021-07-07 2021-11-19 华南理工大学 一种基于分层卷积神经网络的手势姿态识别方法
CN114627503A (zh) * 2022-03-17 2022-06-14 广州虎牙科技有限公司 一种人手识别方法、装置、电子设备及存储介质
CN114912540A (zh) * 2022-05-30 2022-08-16 上海商汤智能科技有限公司 迁移学习方法、装置、设备及存储介质

Also Published As

Publication number Publication date
CN118675221A (zh) 2024-09-20

Similar Documents

Publication Publication Date Title
CN111860300B (zh) 一种关键点检测方法、装置、终端设备及存储介质
CN110322500B (zh) 即时定位与地图构建的优化方法及装置、介质和电子设备
CN110532981B (zh) 人体关键点提取方法、装置、可读存储介质及设备
CN113297973B (zh) 关键点检测方法、装置、设备和计算机可读介质
CN111783626B (zh) 图像识别方法、装置、电子设备及存储介质
CN110020620A (zh) 一种大姿态下的人脸识别方法、装置及设备
CN111368668B (zh) 三维手部识别方法、装置、电子设备及存储介质
CN109754464B (zh) 用于生成信息的方法和装置
CN113378773A (zh) 手势识别方法、装置、设备、存储介质以及程序产品
WO2024104239A1 (zh) 视频标注方法、装置、设备、介质及产品
CN113537309A (zh) 一种对象识别方法、装置及电子设备
CN118243134A (zh) 基于单目视觉惯性里程计的数据处理方法、装置、电子设备及存储介质
WO2024152723A1 (zh) 表情信息识别方法、装置、设备、可读存储介质及产品
CN114495173A (zh) 一种姿态识别方法、装置、电子设备及计算机可读介质
CN113706606B (zh) 确定隔空手势位置坐标的方法及装置
CN111601129B (zh) 控制方法、装置、终端及存储介质
WO2024188293A1 (zh) 模型构建、对象识别方法、装置、设备、介质及产品
WO2025167333A1 (zh) 一种图像生成方法、装置、设备、介质、产品
CN115880719B (zh) 手势深度信息生成方法、装置、设备和计算机可读介质
WO2024169825A1 (zh) 动作识别方法、装置、电子设备、介质及计算机程序产品
CN116912314A (zh) 一种姿态估计方法及其相关设备
CN115482279A (zh) 物体位姿估计方法、装置、介质和设备
CN115482275B (zh) 位置参数获取方法、装置、设备及介质
CN117275017A (zh) 文字提取方法、装置、设备及存储介质
CN112880675B (zh) 用于视觉定位的位姿平滑方法、装置、终端和移动机器人

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24769983

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 24769983

Country of ref document: EP

Kind code of ref document: A1