WO2025103191A1 - 信息获取方法、装置、设备及介质 - Google Patents

信息获取方法、装置、设备及介质 Download PDF

Info

Publication number
WO2025103191A1
WO2025103191A1 PCT/CN2024/130274 CN2024130274W WO2025103191A1 WO 2025103191 A1 WO2025103191 A1 WO 2025103191A1 CN 2024130274 W CN2024130274 W CN 2024130274W WO 2025103191 A1 WO2025103191 A1 WO 2025103191A1
Authority
WO
WIPO (PCT)
Prior art keywords
information
prediction
target object
target
network
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
PCT/CN2024/130274
Other languages
English (en)
French (fr)
Inventor
胡青文
郑珊如
许有疆
陈思颖
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Zitiao Network Technology Co Ltd
Original Assignee
Beijing Zitiao Network Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Zitiao Network Technology Co Ltd filed Critical Beijing Zitiao Network Technology Co Ltd
Publication of WO2025103191A1 publication Critical patent/WO2025103191A1/zh
Anticipated expiration legal-status Critical
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
    • G06V10/449—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters
    • G06V10/451—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters with interaction between the filter responses, e.g. cortical complex cells
    • G06V10/454—Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/52—Scale-space analysis, e.g. wavelet analysis
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Definitions

  • the present disclosure relates to the field of computer technology, and in particular to an information acquisition method, device, equipment and medium.
  • the present disclosure provides an information acquisition method, device, equipment and medium.
  • an embodiment of the present disclosure provides an information acquisition method, the method comprising: acquiring a target image to be processed; wherein the target image contains a target object; inputting the target image into a preset information prediction model; wherein the information prediction model comprises a feature extraction network, a key point prediction network and a part information prediction network; performing feature extraction on the target image through the feature extraction network to obtain image features; based on the image features, acquiring position information of key points corresponding to multiple body parts of the target object through the key point prediction network; based on the image features, acquiring body information of the target object through the part information prediction network; the body information comprises length information of a first body part and/or rotation angle information of a second body part.
  • the disclosed embodiment further provides an information acquisition device, comprising: an image acquisition module, for acquiring a target image to be processed; wherein the target image contains a target object; a model input module, for inputting the target image into a preset information prediction model; wherein the information prediction model includes a feature extraction network, a key point prediction network and a part information prediction network; a feature extraction module, for extracting features of the target image through the feature extraction network to obtain image features; a key point position acquisition module, for acquiring the position information of key points corresponding to multiple body parts of the target object through the key point prediction network based on the image features; a body information acquisition module, for acquiring the position information of key points corresponding to multiple body parts of the target object through the part information prediction network based on the image features
  • the body information of the target object; the body information includes length information of a first body part and/or rotation angle information of a second body part.
  • an embodiment of the present disclosure further provides an electronic device, comprising: a storage device on which a computer program is stored; and a processing device for executing the computer program in the storage device to implement the steps of the information acquisition method provided in the embodiment of the present disclosure.
  • an embodiment of the present disclosure further provides a computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program is used to execute the information acquisition method provided by the embodiment of the present disclosure.
  • FIG1 is a schematic diagram of a flow chart of an information acquisition method provided by an embodiment of the present disclosure
  • FIG2 is a schematic diagram of the structure of an information prediction model provided by an embodiment of the present disclosure.
  • FIG3 is a schematic diagram of the structure of an information prediction model provided by an embodiment of the present disclosure.
  • FIG4 is a schematic diagram of the structure of an information prediction model provided by an embodiment of the present disclosure.
  • FIG5 is a schematic diagram of the structure of an information prediction model provided by an embodiment of the present disclosure.
  • FIG6 is a schematic diagram of the structure of a prediction unit provided by an embodiment of the present disclosure.
  • FIG7 is a schematic diagram of the structure of an information prediction model provided by an embodiment of the present disclosure.
  • FIG8 is a schematic diagram of the structure of an information acquisition device provided by an embodiment of the present disclosure.
  • FIG. 9 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure.
  • the above technical solution provided by the embodiment of the present disclosure can directly use the feature extraction network in the information prediction model to extract features of the target image, and based on the extracted image features, use the key point prediction network in the information prediction model to extract features of the target image.
  • the method can directly use the information prediction model to obtain relatively rich information such as the position of key points corresponding to multiple body parts of the target object and the length information and/or rotation angle information of the body parts at one time. Such information is also more conducive to subsequent flexible application processing and can better meet image processing requirements.
  • FIG1 is a flow chart of an information acquisition method provided by an embodiment of the present disclosure, which can be executed by an information acquisition device, wherein the device can be implemented by software and/or hardware, and can generally be integrated in an electronic device.
  • the method mainly includes the following steps S102 to S110:
  • Step S102 obtaining a target image to be processed; wherein the target image contains a target object.
  • the target object may be a person, an animal, etc., which is not limited here.
  • the disclosed embodiment does not limit the method for obtaining the target image.
  • the target image may be an image collected by the user, or an image of the user taken with the authorization of the user (in which case the user is the target object), or an image selected by the user from an image library, etc.
  • Step S104 input the target image into a preset information prediction model; wherein the information prediction model includes a feature extraction network, a key point prediction network, and a part information prediction network.
  • the information prediction model is a neural network model, and the embodiments of the present disclosure do not limit the specific structures of the feature extraction network, the key point prediction network, and the part information prediction network included in the information prediction model.
  • Step S106 extracting features from the target image through a feature extraction network to obtain image features.
  • the feature extraction network can extract features from the target image by downsampling layer by layer to obtain image features of a desired scale.
  • Step S108 based on the image features, obtaining the position information of the key points corresponding to the multiple body parts of the target object through the key point prediction network.
  • the multiple body parts include arms and/or legs, and also include one or more of the head, neck, chest, abdomen and waist.
  • the embodiment of the present disclosure sets and obtains corresponding key points based on multiple body parts of the target object.
  • the multiple body parts not only include arms and/or legs, but can also further include the head, neck, chest, abdomen and waist, etc.
  • the target person in the image is subjected to special effects such as breast enhancement, waist slimming, and swan neck, if it is based on the existing human 2D key point protocol designed according to the skeleton structure (only 17 limb key points are set), the limb key points cannot be directly applied, and it is also necessary to estimate through other points on this basis, which seriously affects the accuracy and efficiency of subsequent image processing.
  • Step S110 based on the image features, obtaining the body information of the target object through a body information prediction network; the body information includes the length information of the first body part and/or the rotation angle information of the second body part.
  • the first body part includes the part between the shoulder and waist of the target object, that is, the first body part may be the upper body of the target object, and the second body part includes the shoulder and/or waist. That is, the information prediction model provided by the embodiment of the present disclosure can not only output key points corresponding to multiple body parts, but also can output key points corresponding to multiple body parts at the same time. Outputting the length information of the first body part and/or the rotation angle information of the second body part is convenient for subsequent direct application, such as processing the upper body that the user is more concerned about, or processing rotatable parts such as shoulders and waist, which is more convenient and flexible.
  • the above method can directly use the information prediction model to obtain relatively rich information such as the key point positions corresponding to the body parts and the length information and/or rotation angle information of the body parts at one time. Such information is also more conducive to subsequent flexible application processing and can better meet image processing needs.
  • the feature extraction of the target image through the feature extraction network to obtain image features includes: extracting multi-scale features of the target image through the feature extraction network to obtain image features of multiple scales.
  • the feature extraction network includes multiple downsampling layers, and the target image can be downsampled layer by layer to obtain multiple features with scales from large to small. For example, the target image is downsampled by 2 times, 4 times, 8 times, and 16 times to obtain the corresponding scale features.
  • the above-mentioned method of obtaining the position information of key points corresponding to multiple body parts of the target object through a key point prediction network based on image features includes: obtaining the position information of key points corresponding to multiple body parts of the target object through a key point prediction network based on one or more features of image features of multiple scales.
  • one or more features of features of multiple scales can be selected according to needs. It can be understood that the receptive fields of features of different scales are different, and the position information of key points can be predicted more accurately and reliably through features of multiple scales.
  • the key point prediction network when the key point prediction network obtains the position information of the key points corresponding to the multiple body parts of the target object, it can refer to the following steps 1 to 3:
  • Step 1 fuse image features of at least two target scales among multiple scales to obtain fused features.
  • the target scale may include features of the smallest scale among features of multiple scales, and one or more features of the intermediate scale among features of multiple scales.
  • features obtained by 16-fold downsampling and features obtained by 8-fold downsampling may be selected for fusion.
  • features of different scales may be unified into the same scale, such as upsampling the features obtained by 16-fold downsampling to obtain a size consistent with that of the features obtained by 8-fold downsampling. Fusion operations such as concatenation, dot multiplication, addition, and convolution may then be performed.
  • the specific feature fusion method adopted in the embodiments of the present disclosure is not limited.
  • the above-mentioned fused features finally obtained fuse feature information of different scales, and the information carried is richer and more comprehensive.
  • Step 2 Obtain the heat map corresponding to the key points of the target part in the target object based on the fusion features.
  • a convolution operation can be performed on the fused features to obtain heat maps corresponding to the key points of multiple body parts in the target object.
  • each key point can correspond to a heat map. Assuming there are 35 key points in total, 35 heat maps can be obtained.
  • the heat map corresponding to each key point can be obtained more accurately and reliably, which helps to further ensure the accuracy of the position prediction of the key points.
  • a key point detection algorithm based on a heat map can be used to determine the position information of the key points of the target part of the target object. For details, please refer to the relevant technology and will not be repeated here.
  • the method of obtaining the target object's body information through a part information prediction network based on image features includes the following method 1 and/or method 2:
  • Method 1 Based on the smallest scale feature among the image features of multiple scales, the body information of the target object is obtained through the part information prediction network.
  • the part information prediction network is directly related to the feature extraction network.
  • Method 2 Obtain fusion features generated by the key point prediction network, and based on the fusion features, obtain the body information of the target object through the part information prediction network; wherein the fusion features are features obtained by the key point prediction network performing a first fusion process on image features of at least two target scales in multiple scales.
  • the part information prediction network is directly related to the key point prediction network and indirectly related to the feature extraction network.
  • the embodiments of the present disclosure provide structural schematic diagrams of three information prediction models as shown in Figures 2 to 4.
  • the input of the feature extraction network is the target image
  • the feature extraction network is directly connected to the key point prediction network and the part information prediction network, respectively
  • one or more scales of image features output by the feature extraction network are used as the input of the key point prediction network
  • the minimum scale of image features output by the feature extraction network is used as the output of the part information prediction network
  • the output of the key point prediction network is the position information of key points of multiple body parts
  • the output of the part information prediction network is the length information of the first body part and/or the rotation angle information of the second body part.
  • the input of the part information prediction network includes both the minimum-scale image features output by the feature extraction network and the fusion features obtained from the key point prediction network.
  • the part information prediction network is directly associated with the feature extraction network and the key point prediction network.
  • a structural diagram of an information prediction model shown in Figure 5 can be referred to, which further illustrates that the specific structure of the part information prediction network includes a first prediction unit, a second prediction unit and a result fusion unit; the first prediction unit is used to obtain a first prediction result corresponding to the body information of the target object based on the smallest scale feature among image features of multiple scales; the second prediction unit is used to obtain a second prediction result corresponding to the body information of the target object based on the fused feature; the result fusion unit is used to fuse the first prediction result and the second prediction result to determine the body information of the target object according to the fusion result.
  • a weighted fusion process can be performed based on the first prediction result and the second prediction result to obtain a weighted fusion result; and the physical information of the target object is determined according to the weighted fusion result.
  • the disclosed embodiments do not restrict the respective weights of the first prediction result and the second prediction result.
  • the weight corresponding to the first prediction result is not lower than the weight corresponding to the second prediction result. According to research, this weight setting method can better ensure the reliability of the final information obtained.
  • the network structure of the first prediction unit and the second prediction unit is the same, and the network structure includes a global processing layer, a length information prediction layer, and a rotation angle prediction layer; wherein the global processing layer is used to perform global processing based on the input information of the network structure to obtain global information; the length information prediction layer is used to perform a first decoupling processing based on the global information to predict the length information of the first body part; the rotation angle prediction layer is used to perform a second decoupling processing based on the global information to predict the rotation angle information of the second body part.
  • a structural diagram of a prediction unit shown in FIG6 can be referred to, which not only illustrates the global processing layer, the length information prediction layer and the rotation angle prediction layer, but also illustrates the specific implementation method of each layer, wherein the global processing layer includes a GAP (Global average pooling) network layer and two sequentially connected FC (Full Connection) network layers, and the length information prediction layer and the rotation angle prediction layer are respectively implemented using FC network layers.
  • GAP Global average pooling
  • FC Fel Connection
  • the disclosed embodiment further provides a structural schematic diagram of an information prediction model as shown in FIG7.
  • the feature extraction network also referred to as the backbone network
  • the feature fusion layer can output fusion features, which are input to the heat map processing layer, and the heat map processing layer is used to obtain the heat map corresponding to the key points based on the fusion features, and output the position information of the key points of multiple body parts based on the heat map.
  • the part information prediction network in FIG7 includes a first prediction unit, a second prediction unit, and a result fusion unit, wherein the input of the first prediction unit is the 16-fold down-sampling feature, and the output is the first prediction result corresponding to the body information of the target object, the input of the second prediction unit is the above-mentioned fusion feature, and the output is the second prediction result corresponding to the body information of the target object, and the result fusion unit is used to fuse the first prediction result and the second prediction result, thereby obtaining the length information of the first body part and/or the rotation angle information of the second body part.
  • the above is only an exemplary description. In actual applications, more or fewer networks may be included, and the network layers within the network can be flexibly adjusted according to needs, which is not limited here.
  • the embodiment of the present disclosure also provides a method for obtaining an information prediction model.
  • the information prediction model is obtained according to the following steps A to C:
  • Step A obtaining a sample image carrying label information; wherein the sample image includes a target object, and the label information includes position labels of key points corresponding to multiple body parts of the target object and a body information label of the target object.
  • step A can be performed with reference to the following steps A1 to A5:
  • Step A1 obtaining a sample image containing a target object.
  • Step A2 using a preset network model to obtain multiple body key points of the three-dimensional structure corresponding to the target object in the sample image.
  • the preset network model can be a 3D mesh model, which can mark 6890 3D key points of the target object (such as a human body) throughout the body.
  • Step A3 determining the position information of the key points corresponding to the multiple body parts of the target object in the sample image according to the multiple body key points and the mapping relationship between the multiple body key points and the key points corresponding to the multiple body parts of the target object.
  • the disclosed embodiment does not directly annotate key points for 2D sample images, but uses a preset network model to first obtain multiple body key points of the three-dimensional structure corresponding to the target object in the sample image, and then regresses 6890 3D key points to the key points corresponding to multiple body parts required by the disclosed embodiment (such as 35 key points) through the mapping relationship between the multiple body key points and the key points corresponding to the multiple body parts of the target object (which can be represented by a regression matrix).
  • the mapping relationship can be obtained based on the following method: for each key point corresponding to multiple body parts of the target object, the association weight of the key point with each body key point is obtained, and then the mapping relationship between the key point and the multiple body key points is obtained based on the association weight of the key point with each body key point.
  • the key points corresponding to the multiple body parts of the target object in the sample image can be efficiently and accurately determined directly through the mapping relationship, which greatly saves the cost of manual annotation.
  • Step A4 determining the body information of the target object in the sample image according to the position information of the key points corresponding to the multiple body parts of the target object in the sample image.
  • the required key points can be further obtained therefrom, and the body information of the target object can be determined according to the position relationship between the key points.
  • the following steps (1) to (3) can be referred to:
  • Step (1) determines at least two first target points corresponding to a first body part and at least two second target points corresponding to a second body part based on the position information of key points corresponding to multiple body parts of the target object in the sample image.
  • the first body part includes the part between the shoulder and waist of the target object, which can also be called the upper body part of the target object;
  • the at least two first target points include: a first target point determined based on the center point between the key point corresponding to the left shoulder part and the key point corresponding to the right shoulder part (which can be referred to as the first center point), and a first target point determined based on the center point between the key point corresponding to the left waist part and the key point corresponding to the right waist part (which can be referred to as the second center point).
  • the first center point is also the midpoint of the shoulder
  • the second center point is also the midpoint of the waist.
  • both the first center point and the second center point can be used as the first target point.
  • the second body part includes a shoulder, and at least two second target points include: a key point corresponding to a left shoulder and a key point corresponding to a right shoulder; and/or, the second body part includes a waist, and at least two second target points include: a key point corresponding to a left waist and a key point corresponding to a right waist.
  • Step (2) determines the length information of the first body part according to at least two first target points.
  • the length information of the first body part i.e., the length information of the upper body, can be determined according to the distance between the aforementioned first center point and the aforementioned second center point.
  • Step (3) determines the rotation angle information of the second body part according to at least two second target points.
  • the rotation angle information of the shoulder can be represented by the angles between the connecting lines between the key points corresponding to the left shoulder part and the key points corresponding to the right shoulder part and the XYZ axes in the spatial coordinate system, and the angles can be the Euler rotation angles (x1, y1, z1).
  • the rotation angle information of the waist can be represented by the angles between the connecting lines between the key points corresponding to the left waist part and the key points corresponding to the right waist part and the XYZ axes in the spatial coordinate system, and the angles can be the Euler rotation angles (x2, y2, z2).
  • Step A5 Based on the position information of key points corresponding to multiple body parts of the target object in the sample image and the body information of the target object in the sample image, label information is added to the sample image.
  • the label information of the sample image can be determined and the sample image can be associated with the label information, that is, the sample image is attached with label information.
  • sample images with label information can be obtained without manual labeling, so the cost of obtaining sample images is low, and a large number of sample images can be obtained for model training as needed.
  • the label information obtained by the above method is relatively more accurate than the label information obtained by manual labeling, and can better avoid common problems such as manual labeling errors. Therefore, the obtained label information is more accurate and reliable, which helps to train a more reliable information prediction model in both the reliability of sample images and the number of sample images.
  • Step B obtaining the information prediction results output by the preset neural network model for the sample image; the information prediction results include the position information prediction results of the key points corresponding to the multiple body parts of the target object in the sample image and the body information prediction results.
  • the structure of the neural network model is consistent with the structure of the aforementioned information prediction model, and the image processing method is also consistent. By adjusting the parameters of the neural network model, an information prediction model that can accurately output the position information of the key points corresponding to the multiple body parts and the body information is finally obtained.
  • Step C Based on the label information and the information prediction result, the neural network model is trained to obtain the information prediction model based on the trained neural network model. Specifically, the parameters of the neural network model can be adjusted in the direction of reducing the difference between the label information and the information prediction result until the information prediction result of the neural network model meets the requirements, thereby obtaining the information prediction model.
  • step C can be performed with reference to the following steps C1 to C3:
  • Step C1 based on the difference between the location tag and the location information prediction result, determine the first loss.
  • the first loss can be determined based on the difference between the location tag corresponding to the sample image and the location information prediction result using a preset first loss function.
  • the embodiment of the present disclosure does not limit the first loss function, such as an MSE (Mean-Square Error) loss function.
  • Step C2 based on the difference between the body information label and the body information prediction result, determine the second loss.
  • the second loss can be determined based on the difference between the body information label corresponding to the sample image and the body information prediction result using a preset second loss function.
  • the disclosed embodiment does not limit the second loss function, such as an L2 loss function.
  • Step C3 training the neural network model based on the first loss and the second loss to obtain an information prediction model based on the trained neural network model.
  • the total loss can be determined based on the first loss and the second loss, and the network parameters in the neural network model can be adjusted based on the total loss.
  • the training is stopped when the total loss converges to a preset threshold, and the trained neural network model is used as an information prediction model.
  • the above method is to train the feature extraction network, key point prediction network and part information prediction network in the neural network model at the same time.
  • the feature extraction network and the key point prediction network can also be trained first, and then the parameters of the feature extraction network and the key point prediction network are fixed, and then the part information prediction network is trained.
  • the specific training method can be flexibly selected and is not limited here.
  • the above-mentioned method provided by the embodiment of the present disclosure compared with the related art that only outputs the corresponding key points based on the skeleton structure, can directly use the information prediction model to obtain the key point positions corresponding to multiple body parts and the length information and/or rotation angle information of the body parts at one time, which is richer information. Such information is also more conducive to subsequent flexible application processing and can better meet the image processing requirements.
  • the training data of the information prediction model provided by the embodiment of the present disclosure does not need to be manually labeled, and the reliability of the information prediction model is fully guaranteed in terms of the two dimensions of training sample quality and training sample quantity.
  • FIG8 is a schematic diagram of the structure of an information acquisition device provided by an embodiment of the present disclosure.
  • the device can be implemented by software and/or hardware, and can generally be integrated in an electronic device.
  • the information acquisition device includes:
  • the image acquisition module 802 is used to acquire a target image to be processed; wherein the target image contains a target object;
  • the model input module 804 is used to input the target image into a preset information prediction model; wherein the information prediction model includes a feature extraction network, a key point prediction network and a part information prediction network;
  • a feature extraction module 806 is used to extract features of a target image through a feature extraction network to obtain image features
  • a key point position acquisition module 808 is used to acquire the position information of key points corresponding to multiple body parts of the target object through a key point prediction network based on image features;
  • the body information acquisition module 810 is used to acquire the body information of the target object through a part information prediction network based on image features; the body information includes length information of a first body part and/or rotation angle information of a second body part.
  • the above-mentioned device can directly use the information prediction model to obtain relatively rich information such as the key point positions corresponding to the body parts and the length information and/or rotation angle information of the body parts at one time. Such information is also more conducive to subsequent flexible application processing and can better meet image processing needs.
  • the feature extraction module 806 is specifically used to: perform multi-scale feature extraction on the target image through the feature extraction network to obtain image features of multiple scales.
  • the key point position acquisition module 808 is specifically used to: based on one or more features of the image features of the multiple scales, obtain the position information of the key points corresponding to multiple body parts of the target object through the key point prediction network.
  • the key point position acquisition module 808 is specifically used to: fuse image features of at least two target scales among the multiple scales to obtain fused features; obtain a thermal map corresponding to the key points of the target part in the target object based on the fused features; and predict the position information of the key points of the target part in the target object based on the thermal map.
  • the body information acquisition module 810 is specifically used to: acquire the body information of the target object through the part information prediction network based on the smallest scale feature among the image features of the multiple scales; and/or acquire the fusion feature generated by the key point prediction network, and acquire the body information of the target object through the part information prediction network based on the fusion feature; wherein the fusion feature is a feature obtained by the key point prediction network by fusing image features of at least two target scales among the multiple scales.
  • the part information prediction network includes a first prediction unit, a second prediction unit and a result fusion unit; the first prediction unit is used to obtain a first prediction result corresponding to the body information of the target object based on the smallest scale feature among the multiple scales of image features; the second prediction unit is used to obtain a second prediction result corresponding to the body information of the target object based on the fused feature; the result fusion unit is used to fuse the first prediction result and the second prediction result to determine the body information of the target object according to the fusion result.
  • the result fusion unit is specifically used to: perform weighted fusion processing based on the first prediction result and the second prediction result to obtain a weighted fusion result; and determine the body information of the target object according to the weighted fusion result.
  • the weight corresponding to the first prediction result is not less than the weight corresponding to the second prediction result.
  • the network structure of the first prediction unit and the second prediction unit is the same, and the network structure includes a global processing layer, a length information prediction layer, and a rotation angle prediction layer; wherein the global processing layer is used to perform global processing based on the input information of the network structure to obtain global information; the length information prediction layer is used to perform a first decoupling processing based on the global information to predict the length information of the first body part; the rotation angle prediction layer is used to perform a second decoupling processing based on the global information to predict the rotation angle information of the second body part.
  • the device also includes a model acquisition module, which is used to obtain the information prediction model according to the following steps: obtaining a sample image carrying label information; wherein the sample image contains a target object, and the label information includes position labels of key points corresponding to multiple body parts of the target object and body information labels of the target object; obtaining information prediction results output by a preset neural network model for the sample image; the information prediction results include position information prediction results and body information prediction results of key points corresponding to multiple body parts of the target object in the sample image; based on the label information and the information prediction results, training the neural network model to obtain an information prediction model based on the trained neural network model.
  • a model acquisition module which is used to obtain the information prediction model according to the following steps: obtaining a sample image carrying label information; wherein the sample image contains a target object, and the label information includes position labels of key points corresponding to multiple body parts of the target object and body information labels of the target object; obtaining information prediction results output by a preset neural network model for the sample image; the information prediction results include
  • the model acquisition module is specifically used to: acquire a sample image containing a target object; use a preset network model to acquire multiple body key points of the three-dimensional structure corresponding to the target object in the sample image; determine the position information of the key points corresponding to the multiple body parts of the target object in the sample image based on the multiple body key points and the mapping relationship between the multiple body key points and the key points corresponding to the multiple body parts of the target object; determine the body information of the target object in the sample image based on the position information of the key points corresponding to the multiple body parts of the target object in the sample image; and attach label information to the sample image based on the position information of the key points corresponding to the multiple body parts of the target object in the sample image and the body information of the target object in the sample image.
  • the model acquisition module is specifically used to: determine at least two first target points corresponding to the first body part and at least two second target points corresponding to the second body part based on the position information of key points corresponding to multiple body parts of the target object in the sample image; determine the length information of the first body part based on the at least two first target points; determine the rotation angle information of the second body part based on the at least two second target points.
  • the first body part includes the area between the shoulder and waist of the target object;
  • the at least two first target points include: a first target point determined based on the center point between the key point corresponding to the left shoulder part and the key point corresponding to the right shoulder part, and a first target point determined based on the center point between the key point corresponding to the left waist part and the key point corresponding to the right waist part.
  • the second body part includes a shoulder
  • the at least two second target points include: a key point corresponding to a left shoulder and a key point corresponding to a right shoulder
  • the second body part includes a waist
  • the at least two second target points include: a key point corresponding to a left waist and a key point corresponding to a right waist.
  • the model acquisition module is specifically used to: determine a first loss based on the difference between the location tag and the location information prediction result; determine a second loss based on the difference between the body information tag and the body information prediction result; and train the neural network model based on the first loss and the second loss to obtain an information prediction model based on the trained neural network model.
  • the plurality of body parts include arms and/or legs, and further include one or more of the head, neck, chest, abdomen, and waist.
  • the information acquisition device provided in the embodiments of the present disclosure can execute the information acquisition method provided in any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method.
  • An embodiment of the present disclosure provides an electronic device, the electronic device comprising: a storage device on which a computer program is stored; a processing device for executing the computer program in the storage device to implement the steps of any method in the present disclosure.
  • An embodiment of the present disclosure provides an electronic device, the electronic device comprising: a storage device on which a computer program is stored; a processing device for executing the computer program in the storage device to implement the steps of any method in the present disclosure.
  • the terminal device in the embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc.
  • mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc.
  • PDAs personal digital assistants
  • PADs tablet computers
  • PMPs portable multimedia players
  • vehicle-mounted terminals such as vehicle-mounted navigation terminals
  • fixed terminals such as digital TVs, desktop computers, etc.
  • the electronic device shown in FIG9 is only an example and should not bring any limitation to the functions and scope of use of
  • the electronic device 900 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage device 908 into a random access memory (RAM) 903.
  • a processing device e.g., a central processing unit, a graphics processing unit, etc.
  • RAM random access memory
  • various programs and data required for the operation of the electronic device 900 are also stored.
  • the processing device 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904.
  • An input/output (I/O) interface 905 is also connected to the bus 904.
  • the following devices may be connected to the I/O interface 905: input devices 906 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 907 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 908 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 909.
  • the communication device 909 may allow the electronic device 900 to communicate wirelessly or wired with other devices to exchange data.
  • FIG. 9 shows an electronic device 900 with various devices, it should be understood that it is not required to implement or have all the devices shown. More or fewer devices may be implemented or have alternatively.
  • an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart.
  • the computer program can be downloaded and installed from a network through a communication device 909, or installed from a storage device 908, or installed from a ROM 902.
  • the processing device 901 the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.
  • the embodiment of the present disclosure may also be a computer-readable storage medium on which computer program instructions are stored.
  • the processor executes the information acquisition method provided by the embodiment of the present disclosure.
  • the computer readable storage medium can adopt any combination of one or more readable media.
  • the readable medium can be a readable signal medium or a readable storage medium.
  • the readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above.
  • readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
  • RAM random access memory
  • ROM read-only memory
  • EPROM or flash memory erasable programmable read-only memory
  • CD-ROM compact disk read-only memory
  • magnetic storage device or any suitable combination of the above.
  • the embodiments of the present disclosure also provide a computer program product, including a computer program/instruction, which implements the information acquisition method in the embodiments of the present disclosure when the computer program/instruction is executed by a processor.
  • a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information.
  • the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.
  • the prompt information in response to receiving an active request from the user, may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form.
  • the pop-up window may also carry a selection control for the user to choose "agree” or “disagree” to provide personal information to the electronic device.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Software Systems (AREA)
  • Medical Informatics (AREA)
  • Computing Systems (AREA)
  • Geometry (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biodiversity & Conservation Biology (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Image Analysis (AREA)

Abstract

本公开实施例涉及一种信息获取方法、装置、设备及介质，该方法包括：获取待处理的目标图像；目标图像中包含有目标对象；将目标图像输入至预设的信息预测模型；信息预测模型包括特征提取网络、关键点预测网络和部位信息预测网络；通过特征提取网络对目标图像进行特征提取，得到图像特征；基于图像特征，通过关键点预测网络获取目标对象的多个身体部位对应的关键点的位置信息；基于图像特征，通过部位信息预测网络获取目标对象的身体信息；身体信息包括第一身体部位的长度信息和/或第二身体部位的旋转角度信息。本公开实施例可以直接借助信息预测模型一次性得到较为丰富的信息，此类信息更有利于后续灵活应用处理，可以较好地满足图像处理需求。

Description

信息获取方法、装置、设备及介质
本申请要求2023年11月17日递交的、标题为“信息获取方法、装置、设备及介质”、申请号为202311541140.5的中国发明专利申请的优先权，该申请的全部内容通过引用结合在本申请中。
技术领域
本公开涉及计算机技术领域，尤其涉及一种信息获取方法、装置、设备及介质。
背景技术
在一些图像处理场合中，需要对图像中所包含的人物等目标对象进行美体等处理。相关技术中，进行此类处理通常需要利用现有的关键点检测模型检测目标对象的关键点，且现有的关键点检测模型可检测到的关键点主要是根据整个骨架结构设计的点，不便于后续应用处理，难以较好地满足图像处理需求。
发明内容
为了解决上述技术问题或者至少部分地解决上述技术问题，本公开提供了一种信息获取方法、装置、设备及介质。
第一方面，本公开实施例提供了一种信息获取方法，所述方法包括：获取待处理的目标图像；其中，所述目标图像中包含有目标对象；将所述目标图像输入至预设的信息预测模型；其中，所述信息预测模型包括特征提取网络、关键点预测网络和部位信息预测网络；通过所述特征提取网络对所述目标图像进行特征提取，得到图像特征；基于所述图像特征，通过所述关键点预测网络获取所述目标对象的多个身体部位对应的关键点的位置信息；基于所述图像特征，通过所述部位信息预测网络获取所述目标对象的身体信息；所述身体信息包括第一身体部位的长度信息和/或第二身体部位的旋转角度信息。
第二方面，本公开实施例还提供了一种信息获取装置，包括：图像获取模块，用于获取待处理的目标图像；其中，所述目标图像中包含有目标对象；模型输入模块，用于将所述目标图像输入至预设的信息预测模型；其中，所述信息预测模型包括特征提取网络、关键点预测网络和部位信息预测网络；特征提取模块，用于通过所述特征提取网络对所述目标图像进行特征提取，得到图像特征；关键点位置获取模块，用于基于所述图像特征，通过所述关键点预测网络获取所述目标对象的多个身体部位对应的关键点的位置信息；身体信息获取模块，用于基于所述图像特征，通过所述部位信息预测网络获取 所述目标对象的身体信息；所述身体信息包括第一身体部位的长度信息和/或第二身体部位的旋转角度信息。
第三方面，本公开实施例还提供了一种电子设备，所述电子设备包括：存储装置，其上存储有计算机程序；处理装置，用于执行所述存储装置中的所述计算机程序，以实现本公开实施例提供的信息获取方法的步骤。
第四方面，本公开实施例还提供了一种计算机可读存储介质，所述存储介质存储有计算机程序，所述计算机程序用于执行本公开实施例提供的信息获取方法。
应当理解，本部分所描述的内容并非旨在标识本公开的实施例的关键或重要特征，也不用于限制本公开的范围。本公开的其它特征将通过以下的说明书而变得容易理解。
附图说明
此处的附图被并入说明书中并构成本说明书的一部分，示出了符合本公开的实施例，并与说明书一起用于解释本公开的原理。
为了更清楚地说明本公开实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，对于本领域普通技术人员而言，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。
图1为本公开实施例提供的一种信息获取方法的流程示意图；
图2为本公开实施例提供的一种信息预测模型的结构示意图；
图3为本公开实施例提供的一种信息预测模型的结构示意图；
图4为本公开实施例提供的一种信息预测模型的结构示意图；
图5为本公开实施例提供的一种信息预测模型的结构示意图；
图6为本公开实施例提供的一种预测单元的结构示意图；
图7为本公开实施例提供的一种信息预测模型的结构示意图；
图8为本公开实施例提供的一种信息获取装置的结构示意图；
图9为本公开实施例提供的一种电子设备的结构示意图。
具体实施方式
为了能够更清楚地理解本公开的上述目的、特征和优点，下面将对本公开的方案进行进一步描述。需要说明的是，在不冲突的情况下，本公开的实施例及实施例中的特征可以相互组合。
在下面的描述中阐述了很多具体细节以便于充分理解本公开，但本公开还可以采用其他不同于在此描述的方式来实施；显然，说明书中的实施例只是本公开的一部分实施例，而不是全部的实施例。
本公开实施例提供的上述技术方案，可以直接利用信息预测模型中的特征提取网络对目标图像进行特征提取，并基于提取出的图像特征，利用信息预测模型中的关键点预 测网络获取目标对象的多个身体部位对应的关键点的位置信息，以及利用信息预测模型中的部位信息预测网络获取目标对象的身体信息，且身体信息包括第一身体部位的长度信息和/或第二身体部位的旋转角度信息。上述方式可以直接借助信息预测模型一次性得到身体部位对应的关键点位置以及身体部位的长度信息和/或旋转角度信息等较为丰富的信息，此类信息也更有利于后续灵活应用处理，可以较好地满足图像处理需求。
图1为本公开实施例提供的一种信息获取方法的流程示意图，该方法可以由信息获取装置执行，其中该装置可以采用软件和/或硬件实现，一般可集成在电子设备中。如图1所示，该方法主要包括如下步骤S102～步骤S110：
步骤S102，获取待处理的目标图像；其中，目标图像中包含有目标对象。目标对象可以为人物、动物等，在此不进行限制。此外，本公开实施例对目标图像的获取方式也不进行限制，诸如，目标图像可以是用户采集的图像，也可以是在用户授权的情况下拍摄得到的用户图像(此时用户为目标对象)，还可以是用户从图像库中选择的图像等。
步骤S104，将目标图像输入至预设的信息预测模型；其中，信息预测模型包括特征提取网络、关键点预测网络和部位信息预测网络。信息预测模型为神经网络模型，本公开实施例对信息预测模型所包含的特征提取网络、关键点预测网络和部位信息预测网络的具体结构不进行限制。
步骤S106，通过特征提取网络对目标图像进行特征提取，得到图像特征。诸如，可以通过特征提取网络采用逐层下采样的方式对目标图像进行特征提取，得到所需尺度的图像特征。
步骤S108，基于图像特征，通过关键点预测网络获取目标对象的多个身体部位对应的关键点的位置信息。
示例性地，多个身体部位包括臂部和/或腿部，还包括头部、颈部、胸部、腹部和腰部中的一种或多种。相比于相关技术中只基于骨架结构获取关键点，本公开实施例基于目标对象的多个身体部位来设置并获取相应的关键点，多个身体部位不仅包含臂部和/或腿部，还可以进一步包含头部、颈部、胸部、腹部和腰部等，通过预测多个身体部位的关键点，更便于后续直接应用，保障后续应用效果。诸如，若对图像中的目标人物进行丰胸、瘦腰、天鹅颈等特效处理时，倘若基于现有根据骨架结构设计的人体2D关键点协议(只设置了17个肢体关键点)，则无法直接应用肢体关键点，还需要在此基础上通过其它点位进行估算，严重影响后续的图像处理准确性以及图像处理效率。
步骤S110，基于图像特征，通过部位信息预测网络获取目标对象的身体信息；身体信息包括第一身体部位的长度信息和/或第二身体部位的旋转角度信息。
在一些实施示例中，第一身体部位包括目标对象的肩部与腰部之间的部位，也即第一身体部位可以为目标对象的上半身，第二身体部位包括肩部和/或腰部。也即，本公开实施例提供的信息预测模型不仅可以输出多个身体部位对应的关键点，而且还可以同时 输出第一身体部位的长度信息和/或第二身体部位的旋转角度信息，便于后续直接应用，诸如，针对用户更关注的上半身进行处理，或者针对肩部、腰部等可旋转的部位进行处理，更方便灵活。
上述方式可以直接借助信息预测模型一次性得到身体部位对应的关键点位置以及身体部位的长度信息和/或旋转角度信息等较为丰富的信息，此类信息也更有利于后续灵活应用处理，可以较好地满足图像处理需求。
在一些实施方式中，上述通过特征提取网络对目标图像进行特征提取，得到图像特征，包括：通过特征提取网络对目标图像进行多尺度特征提取，得到多种尺度的图像特征。示例性地，特征提取网络包括多个下采样层，可以对目标图像进行逐层下采样，从而得到尺度由大至小的多个特征。诸如，对目标图像进行2倍下采样、4倍下采样、8倍下采样和16倍下采样，以此得到各自对应的尺度特征。
在前述基础上，上述基于图像特征，通过关键点预测网络获取目标对象的多个身体部位对应的关键点的位置信息，包括：基于多种尺度的图像特征中的一种或多种特征，通过关键点预测网络获取目标对象的多个身体部位对应的关键点的位置信息。在实际应用中，可以根据需求选择多种尺度的特征中的一种或多种特征，可以理解的是，不同尺度的特征的感受野不同，通过多种尺度的特征能够更为准确可靠地预测关键点的位置信息。
在一些具体的实施示例中，关键点预测网络在获取目标对象的多个身体部位对应的关键点的位置信息时，可以参照如下步骤1～步骤3执行：
步骤1，将多种尺度中至少两个目标尺度的图像特征进行融合，得到融合特征。
示例性地，目标尺度可以包括多种尺度的特征中最小尺度的特征，以及多种尺度的特征中的中间尺度的一种或多种特征，诸如，可以选择16倍下采样所得的特征以及8倍下采样所得的特征进行融合，在具体融合时，可以将不同尺度的特征统一为相同尺度，诸如，将16倍下采样所得的特征进行上采样处理，从而得到与8倍下采样所得的特征的尺寸一致，进而便可执行诸如拼接、点乘、相加、卷积等融合操作，本公开实施例所采用的具体的特征融合方式不进行限制，最终所得的上述融合特征融合了不同尺度的特征信息，所携带的信息更为丰富全面。
步骤2，根据融合特征获取目标对象中目标部位的关键点对应的热力图。
示例性地，可以针对融合特征执行卷积操作，从而得到目标对象中多个身体部位的关键点对应的热力图。在实际应用中，每个关键点均可对应一张热力图，假设一共35个关键点，则可以得到35张热力图。利用携带信息更为丰富全面的融合特征，可以更为准确可靠地获取到每个关键点对应的热力图，有助于进一步保障关键点的位置预测准确性。
步骤3，基于热力图预测目标对象中目标部位的关键点的位置信息。
在实际应用中，可以采用基于热力图的关键点检测算法确定目标对象中目标部位的关键点的位置信息，具体可参照相关技术，在此不再赘述。
在一些实施方式中，上述基于图像特征，通过部位信息预测网络获取目标对象的身体信息，包括如下方式一和/或方式二：
方式一，基于多种尺度的图像特征中的最小尺度的特征，通过部位信息预测网络获取目标对象的身体信息。在该方式中，部位信息预测网络与特征提取网络直接相关。
方式二，获取关键点预测网络生成的融合特征，并基于融合特征，通过部位信息预测网络获取目标对象的身体信息；其中，融合特征是关键点预测网络将多种尺度中至少两个目标尺度的图像特征进行第一融合处理所得的特征。在该方式中，部位信息预测网络与关键点预测网络直接相关，与特征提取网络间接相关。
为便于理解，本公开实施例提供了如图2～图4所示的三种信息预测模型的结构示意图，具体的，对应于前述方式一，参见图2所示的一种信息预测模型的结构示意图，示意出特征提取网络的输入为目标图像，特征提取网络分别与关键点预测网络和部位信息预测网络直接相连接，特征提取网络输出的一种或多种尺度的图像特征作为关键点预测网络的输入，特征提取网络输出的最小尺度的图像特征作为部位信息预测网络的输出，关键点预测网络的输出为多个身体部位的关键点的位置信息，部位信息预测网络的输出为第一身体部位的长度信息和/或第二身体部位的旋转角度信息。
对应于前述方式二，参见图3所示的一种信息预测模型的结构示意图，与图2的区别在于，特征提取网络输出的多种尺度的图像特征作为关键点预测网络的输入，部位信息预测网络的输入为从关键点预测网络获取到的融合特征，此时部位信息预测网络与关键点预测网络并非直接相关联。
对应于前述方式一和方式二，参见图4所示的一种信息预测模型的结构示意图，与图2和图3的区别在于部位信息预测网络的输入既包含特征提取网络输出的最小尺度的图像特征，又包含从关键点预测网络获取到的融合特征，此时部位信息预测网络与特征提取网络以及关键点预测网络均直接相关联。
一些具体的实施示例中，在图4的基础上，可以参照图5所示的一种信息预测模型的结构示意图，进一步示意出部位信息预测网络的具体结构包括第一预测单元、第二预测单元和结果融合单元；第一预测单元用于基于多种尺度的图像特征中的最小尺度的特征，获取目标对象的身体信息对应的第一预测结果；第二预测单元用于基于融合特征，获取目标对象的身体信息对应的第二预测结果；结果融合单元用于根据第一预测结果和第二预测结果进行融合，以根据融合结果确定目标对象的身体信息。
结果融合单元在根据第一预测结果和第二预测结果进行融合，以根据融合结果确定目标对象的身体信息的一种具体实施示例中，可以基于第一预测结果和第二预测结果进行加权融合处理，得到加权融合结果；根据加权融合结果，确定目标对象的身体信息。 本公开实施例对第一预测结果和第二预测结果各自的权重不进行限制，在一些具体实施方式中，第一预测结果对应的权重不低于第二预测结果对应的权重，经研究，这种权重设置方式可以较好地保障最终所得信息的可靠性。
在一些具体的实施示例中，第一预测单元和第二预测单元的网络结构相同，且网络结构包括全局处理层、长度信息预测层以及旋转角度预测层；其中，全局处理层用于基于网络结构的输入信息进行全局处理，得到全局信息；长度信息预测层用于基于全局信息进行第一解耦处理，以预测得到第一身体部位的长度信息；旋转角度预测层用于基于全局信息进行第二解耦处理，以预测得到第二身体部位的旋转角度信息。
为便于理解，可以参照图6所示的一种预测单元的结构示意图，不仅示意出全局处理层、长度信息预测层以及旋转角度预测层，还示意出各层的具体实现方式，其中，全局处理层包括GAP(Global average pooling，全局平均池化)网络层以及2个顺次连接的FC(Full Connection，全连接)网络层，长度信息预测层以及旋转角度预测层分别采用FC网络层实现。应当说明的是，全局处理层、长度信息预测层以及旋转角度预测层中虽然均包含FC网络层，但是具体参数可以不同，在此不进行限制。上述采用两个不同分支对全局信息进行解耦处理，从而分别进行信息预测的方式，相比于相关技术中只利用一条网络支路去预测所有信息的方式而言，最终所得的结果更为准确。
为便于理解，本公开实施例进一步提供了一种如图7所示的信息预测模型的结构示意图。示意出了特征提取网络(也可称为骨干网络)可输出16倍下采样特征以及8倍下采样特征，其中，16倍下采样特征经过上采样层后与8倍下采样特征共同输入至特征融合层，特征融合层可输出融合特征，融合特征输入至热力图处理层，热力图处理层用于基于融合特征获取关键点对应的热力图，并基于热力图输出多个身体部位的关键点的位置信息。图7中的部位信息预测网络包括第一预测单元、第二预测单元和结果融合单元，第一预测单元的输入为16倍下采样特征，输出为目标对象的身体信息对应的第一预测结果，第二预测单元的输入为上述融合特征，输出为目标对象的身体信息对应的第二预测结果，结果融合单元用于根据第一预测结果和第二预测结果进行融合，从而得到第一身体部位的长度信息和/或第二身体部位的旋转角度信息。以上仅为示例性说明，在实际应用中可以包含更多或更少的网络，网络内部的网络层可以根据需求灵活调整，在此不进行限制。
进一步，本公开实施例还提供了信息预测模型的获得方式，示例性地，信息预测模型是按照如下步骤A～步骤C得到的：
步骤A，获取携带有标签信息的样本图像；其中，样本图像包含目标对象，且标签信息包括目标对象中多个身体部位对应的关键点的位置标签以及目标对象的身体信息标签。为便于理解，在一些具体的实施示例中，步骤A可以参照如下步骤A1～步骤A5执行：
步骤A1，获取包含有目标对象的样本图像。
步骤A2，采用预设网络模型获取样本图像中目标对象对应的三维立体结构的多个身体关键点。示例性地，预设网络模型可以为3D mesh模型，其可标记目标对象(诸如人体)全身的6890个3D关键点。
步骤A3，根据多个身体关键点，以及多个身体关键点与目标对象的多个身体部位对应的关键点之间的映射关系，确定样本图像中目标对象的多个身体部位对应的关键点的位置信息。
本公开实施例并非直接针对2D样本图像进行关键点标注，而是利用预设网络模型先获取样本图像中目标对象对应的三维立体结构的多个身体关键点，再通过多个身体关键点与目标对象的多个身体部位对应的关键点之间的映射关系(可通过回归矩阵表征)，即可将6890个3D关键点回归至本公开实施例所需的多个身体部位对应的关键点(诸如35个关键点)。在实际应用中，映射关系可以基于如下方式得到：对于目标对象的多个身体部位对应的每个关键点，获取该关键点分别与每个身体关键点的关联权重，然后再基于该关键点分别与每个身体关键点的关联权重，得到该关键点与多个身体关键点的映射关系。在通过预设网络模型已知目标对象对应的三维立体结构的多个身体关键点的基础上，直接通过映射关系便可高效且准确地确定样本图像中目标对象的多个身体部位对应的关键点，极大节约了人工标注成本。
步骤A4，根据样本图像中目标对象的多个身体部位对应的关键点的位置信息，确定样本图像中目标对象的身体信息。在已知多个身体部位对应的关键点的位置信息的基础上，还可进一步从中获取所需的关键点，并根据关键点之间的位置关系确定目标对象的身体信息。示例性地，可以参照如下步骤(1)～步骤(3)：
步骤(1)根据样本图像中目标对象的多个身体部位对应的关键点的位置信息，确定第一身体部位对应的至少两个第一目标点以及第二身体部位对应的至少两个第二目标点。
在一些具体的实施示例中，第一身体部位包括目标对象的肩部与腰部之间的部位，也可称为目标对象的上半身部位；至少两个第一目标点包括：基于左肩部位对应的关键点与右肩部位对应的关键点之间的中心点(可简称第一中心点)确定的第一目标点，以及基于左腰部位对应的关键点与右腰部位对应的关键点之间的中心点(可简称第二中心点)确定的第一目标点。其中，第一中心点也即肩部中点，第二中心点也即腰部中点，本公开实施例可将第一中心点和第二中心点均作为第一目标点。
在一些具体的实施示例中，第二身体部位包括肩部，至少两个第二目标点包括：左肩部位对应的关键点与右肩部位对应的关键点；和/或，第二身体部位包括腰部，至少两个第二目标点包括：左腰部位对应的关键点与右腰部位对应的关键点。
步骤(2)根据至少两个第一目标点，确定第一身体部位的长度信息。在第一身体部位包括目标对象的肩部与腰部之间的部位的情况下，可以根据前述第一中心点和前述第二中心点之间的距离，确定第一身体部位的长度信息，也即上半身长度信息。
步骤(3)根据至少两个第二目标点，确定第二身体部位的旋转角度信息。示例性地，肩部的旋转角度信息可通过左肩部位对应的关键点与右肩部位对应的关键点之间的连线分别与空间坐标系中XYZ轴各自的夹角表征，该夹角可以为欧拉旋转角度(x1、y1、z1)，腰部的旋转角度信息可通过左腰部位对应的关键点与右腰部位对应的关键点之间的连线分别与空间坐标系中XYZ轴各自的夹角表征，该夹角可以为欧拉旋转角度(x2、y2、z2)。
步骤A5，基于样本图像中目标对象的多个身体部位对应的关键点的位置信息以及样本图像中目标对象的身体信息，为样本图像附带标签信息。
在已知样本图像中目标对象的多个身体部位对应的关键点的位置信息以及样本图像中目标对象的身体信息的基础上，即可确定样本图像的标签信息，并将样本图像与标签信息建立关联，也即为样本图像附带标签信息。
通过前述步骤A1～步骤A5，无需人工标注便可得到附带有标签信息的样本图像，因此获取样本图像的成本较低，可以根据需求获得大量样本图像进行模型训练，而且上述方式获取到的标签信息相比于人工标注所得的标签信息而言，精度相对更高，可以较好的避免常见的人工标注误差等问题，因此所得的标签信息更为准确可靠，在样本图像的可靠性以及样本图像的数量两个维度都有助于训练得到更为可靠的信息预测模型。
步骤B，获取预设的神经网络模型针对样本图像输出的信息预测结果；信息预测结果包括样本图像中目标对象的多个身体部位对应的关键点的位置信息预测结果以及身体信息预测结果。神经网络模型的结构与前述信息预测模型的结构一致，对图像的处理方式也一致，通过对神经网络模型的参数进行调整，最终得到能够准确输出多个身体部位对应的关键点的位置信息以及身体信息的信息预测模型。
步骤C，基于标签信息和信息预测结果，对神经网络模型进行训练，以基于训练后的神经网络模型得到信息预测模型。具体的，可以朝着缩小标签信息和信息预测结果之间的差异的方向来调整神经网络模型的参数，直至神经网络模型的信息预测结果符合需求，得到信息预测模型。
在一些具体的实施示例中，步骤C可以参照如下步骤C1～步骤C3执行：
步骤C1，基于位置标签和位置信息预测结果之间的差异，确定第一损失。示例性地，可以基于样本图像对应的位置标签和位置信息预测结果之间的差异，利用预设的第一损失函数，确定第一损失。本公开实施例对第一损失函数不进行限制，诸如可以为MSE(Mean-Square Error，均方误差)损失函数。
步骤C2，基于身体信息标签和身体信息预测结果之间的差异，确定第二损失。示例性地，可以基于样本图像对应的身体信息标签和身体信息预测结果之间的差异，利用预设的第二损失函数，确定第二损失。本公开实施例对第二损失函数不进行限制，诸如可以为L2损失函数。
步骤C3，基于第一损失和第二损失，对神经网络模型进行训练，以基于训练后的神经网络模型得到信息预测模型。
实际应用中，可以基于第一损失和第二损失确定总损失，基于总损失调整神经网络模型中的网络参数，直至总损失收敛至预设阈值时停止训练，将训练后的神经网络模型作为信息预测模型。以上方式即为对神经网络模型中的特征提取网络、关键点预测网络和部位信息预测网络同时训练，此外，若神经网络模型的结构为图3或图4的结构，由于神经网络模型中的部位信息预测网络需要依赖关键点预测网络输出的信息，在此情况下也可以优先训练特征提取网络以及关键点预测网络，之后固定特征提取网络以及关键点预测网络的参数，再训练部位信息预测网络。具体训练方式可灵活选择，在此不进行限制。
综上，本公开实施例提供的上述方式，与相关技术中只基于骨架结构输出相应的关键点相比，可以直接借助信息预测模型一次性得到多个身体部位对应的关键点位置以及身体部位的长度信息和/或旋转角度信息等较为丰富的信息，此类信息也更有利于后续灵活应用处理，可以较好地满足图像处理需求。而且，本公开实施例提供的信息预测模型的训练数据无需人工标注，在训练样本质量以及训练样本数量两个维度充分保障了信息预测模型的可靠性。
对应于前述信息获取方法，本公开实施例提供了一种信息获取装置，图8为本公开实施例提供的一种信息获取装置的结构示意图，该装置可由软件和/或硬件实现，一般可集成在电子设备中，可通过执行信息获取方法，如图8所示，信息获取装置包括：
图像获取模块802，用于获取待处理的目标图像；其中，目标图像中包含有目标对象；
模型输入模块804，用于将目标图像输入至预设的信息预测模型；其中，信息预测模型包括特征提取网络、关键点预测网络和部位信息预测网络；
特征提取模块806，用于通过特征提取网络对目标图像进行特征提取，得到图像特征；
关键点位置获取模块808，用于基于图像特征，通过关键点预测网络获取目标对象的多个身体部位对应的关键点的位置信息；
身体信息获取模块810，用于基于图像特征，通过部位信息预测网络获取目标对象的身体信息；身体信息包括第一身体部位的长度信息和/或第二身体部位的旋转角度信息。
上述装置可以直接借助信息预测模型一次性得到身体部位对应的关键点位置以及身体部位的长度信息和/或旋转角度信息等较为丰富的信息，此类信息也更有利于后续灵活应用处理，可以较好地满足图像处理需求。
在一些实施方式中，所述特征提取模块806具体用于：通过所述特征提取网络对所述目标图像进行多尺度特征提取，得到多种尺度的图像特征。
在一些实施方式中，所述关键点位置获取模块808具体用于：基于所述多种尺度的图像特征中的一种或多种特征，通过所述关键点预测网络获取所述目标对象的多个身体部位对应的关键点的位置信息。
在一些实施方式中，所述关键点位置获取模块808具体用于：将所述多种尺度中至少两个目标尺度的图像特征进行融合，得到融合特征；根据所述融合特征获取所述目标对象中目标部位的关键点对应的热力图；基于所述热力图预测所述目标对象中目标部位的关键点的位置信息。
在一些实施方式中，所述身体信息获取模块810具体用于：基于所述多种尺度的图像特征中的最小尺度的特征，通过所述部位信息预测网络获取所述目标对象的身体信息；和/或，获取所述关键点预测网络生成的融合特征，并基于所述融合特征，通过所述部位信息预测网络获取所述目标对象的身体信息；其中，所述融合特征是所述关键点预测网络将所述多种尺度中至少两个目标尺度的图像特征进行融合所得的特征。
在一些实施方式中，所述部位信息预测网络包括第一预测单元、第二预测单元和结果融合单元；所述第一预测单元用于基于所述多种尺度的图像特征中的最小尺度的特征，获取所述目标对象的身体信息对应的第一预测结果；所述第二预测单元用于基于所述融合特征，获取所述目标对象的身体信息对应的第二预测结果；所述结果融合单元用于根据所述第一预测结果和所述第二预测结果进行融合，以根据融合结果确定所述目标对象的身体信息。
在一些实施方式中，所述结果融合单元具体用于：基于所述第一预测结果和所述第二预测结果进行加权融合处理，得到加权融合结果；根据所述加权融合结果，确定所述目标对象的身体信息。
在一些实施方式中，所述第一预测结果对应的权重不低于所述第二预测结果对应的权重。
在一些实施方式中，所述第一预测单元和所述第二预测单元的网络结构相同，且所述网络结构包括全局处理层、长度信息预测层以及旋转角度预测层；其中，所述全局处理层用于基于所述网络结构的输入信息进行全局处理，得到全局信息；所述长度信息预测层用于基于所述全局信息进行第一解耦处理，以预测得到第一身体部位的长度信息；所述旋转角度预测层用于基于所述全局信息进行第二解耦处理，以预测得到第二身体部位的旋转角度信息。
在一些实施方式中，所述装置还包括模型获取模块，用于按照如下步骤得到所述信息预测模型：获取携带有标签信息的样本图像；其中，所述样本图像包含目标对象，且所述标签信息包括所述目标对象中多个身体部位对应的关键点的位置标签以及所述目标对象的身体信息标签；获取预设的神经网络模型针对所述样本图像输出的信息预测结果；所述信息预测结果包括所述样本图像中目标对象的多个身体部位对应的关键点的位置信息预测结果以及身体信息预测结果；基于所述标签信息和所述信息预测结果，对所述神经网络模型进行训练，以基于训练后的神经网络模型得到信息预测模型。
在一些实施方式中，所述模型获取模块具体用于：获取包含有目标对象的样本图像；采用预设网络模型获取所述样本图像中目标对象对应的三维立体结构的多个身体关键点；根据所述多个身体关键点，以及所述多个身体关键点与所述目标对象的多个身体部位对应的关键点之间的映射关系，确定所述样本图像中目标对象的多个身体部位对应的关键点的位置信息；根据所述样本图像中目标对象的多个身体部位对应的关键点的位置信息，确定所述样本图像中目标对象的身体信息；基于所述样本图像中目标对象的多个身体部位对应的关键点的位置信息以及所述样本图像中目标对象的身体信息，为所述样本图像附带标签信息。
在一些实施方式中，所述模型获取模块具体用于：根据所述样本图像中目标对象的多个身体部位对应的关键点的位置信息，确定第一身体部位对应的至少两个第一目标点以及第二身体部位对应的至少两个第二目标点；根据所述至少两个第一目标点，确定所述第一身体部位的长度信息；根据所述至少两个第二目标点，确定所述第二身体部位的旋转角度信息。
在一些实施方式中，所述第一身体部位包括所述目标对象的肩部与腰部之间的部位；所述至少两个第一目标点包括：基于左肩部位对应的关键点与右肩部位对应的关键点之间的中心点确定的第一目标点，以及基于左腰部位对应的关键点与右腰部位对应的关键点之间的中心点确定的第一目标点。
在一些实施方式中，所述第二身体部位包括肩部，所述至少两个第二目标点包括：左肩部位对应的关键点与右肩部位对应的关键点；和/或，所述第二身体部位包括腰部，所述至少两个第二目标点包括：左腰部位对应的关键点与右腰部位对应的关键点。
在一些实施方式中，所述模型获取模块具体用于：基于所述位置标签和所述位置信息预测结果之间的差异，确定第一损失；基于所述身体信息标签和所述身体信息预测结果之间的差异，确定第二损失；基于所述第一损失和所述第二损失，对所述神经网络模型进行训练，以基于训练后的神经网络模型得到信息预测模型。
在一些实施方式中，所述多个身体部位包括臂部和/或腿部，还包括头部、颈部、胸部、腹部和腰部中的一种或多种。
本公开实施例所提供的信息获取装置可执行本公开任意实施例所提供的信息获取方法，具备执行方法相应的功能模块和有益效果。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的装置实施例的具体工作过程，可以参考方法实施例中的对应过程，在此不再赘述。
本公开实施例提供了一种电子设备，电子设备包括：存储装置，其上存储有计算机程序；处理装置，用于执行所述存储装置中的所述计算机程序，以实现本公开中任一项方法的步骤。本公开实施例提供了一种电子设备，电子设备包括：存储装置，其上存储有计算机程序；处理装置，用于执行所述存储装置中的所述计算机程序，以实现本公开中任一项方法的步骤。
下面参考图9，其示出了适于用来实现本公开实施例的电子设备900的结构示意图。本公开实施例中的终端设备可以包括但不限于诸如移动电话、笔记本电脑、数字广播接收器、PDA(个人数字助理)、PAD(平板电脑)、PMP(便携式多媒体播放器)、车载终端(例如车载导航终端)等等的移动终端以及诸如数字TV、台式计算机等等的固定终端。图9示出的电子设备仅仅是一个示例，不应对本公开实施例的功能和使用范围带来任何限制。
如图9所示，电子设备900可以包括处理装置(例如中央处理器、图形处理器等)901，其可以根据存储在只读存储器(ROM)902中的程序或者从存储装置908加载到随机访问存储器(RAM)903中的程序而执行各种适当的动作和处理。在RAM 903中，还存储有电子设备900操作所需的各种程序和数据。处理装置901、ROM 902以及RAM 903通过总线904彼此相连。输入/输出(I/O)接口905也连接至总线904。
通常，以下装置可以连接至I/O接口905：包括例如触摸屏、触摸板、键盘、鼠标、摄像头、麦克风、加速度计、陀螺仪等的输入装置906；包括例如液晶显示器(LCD)、扬声器、振动器等的输出装置907；包括例如磁带、硬盘等的存储装置908；以及通信装置909。通信装置909可以允许电子设备900与其他设备进行无线或有线通信以交换数据。虽然图9示出了具有各种装置的电子设备900，但是应理解的是，并不要求实施或具备所有示出的装置。可以替代地实施或具备更多或更少的装置。
特别地，根据本公开的实施例，上文参考流程图描述的过程可以被实现为计算机软件程序。例如，本公开的实施例包括一种计算机程序产品，其包括承载在非暂态计算机可读介质上的计算机程序，该计算机程序包含用于执行流程图所示的方法的程序代码。在这样的实施例中，该计算机程序可以通过通信装置909从网络上被下载和安装，或者从存储装置908被安装，或者从ROM 902被安装。在该计算机程序被处理装置901执行时，执行本公开实施例的方法中限定的上述功能。
除了上述方法和设备以外，本公开的实施例还可以是计算机程序产品，其包括计算机程序指令，所述计算机程序指令在被处理器运行时使得所述处理器执行本公开实施例 所提供的图像处理方法。所述计算机程序产品可以以一种或多种程序设计语言的任意组合来编写用于执行本公开实施例操作的程序代码，所述程序设计语言包括面向对象的程序设计语言，诸如Java、C++等，还包括常规的过程式程序设计语言，诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算设备上执行、部分地在用户设备上执行、作为一个独立的软件包执行、部分在用户计算设备上部分在远程计算设备上执行、或者完全在远程计算设备或服务器上执行。
此外，本公开的实施例还可以是计算机可读存储介质，其上存储有计算机程序指令，所述计算机程序指令在被处理器运行时使得所述处理器执行本公开实施例所提供的信息获取方法。
所述计算机可读存储介质可以采用一个或多个可读介质的任意组合。可读介质可以是可读信号介质或者可读存储介质。可读存储介质例如可以包括但不限于电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。可读存储介质的更具体的例子(非穷举的列表)包括：具有一个或多个导线的电连接、便携式盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。
本公开实施例还提供了一种计算机程序产品，包括计算机程序/指令，该计算机程序/指令被处理器执行时实现本公开实施例中的信息获取方法。
可以理解的是，在使用本公开各施例公开的技术方案之前，均应当依据相关法律法规通过恰当的方式对本公开所涉及个人信息的类型、使用范围、使用场景等告知用户并获得用户的授权。
例如，在响应于接收到用户的主动请求时，向用户发送提示信息，以明确地提示用户，其请求执行的操作将需要获取和使用到用户的个人信息。从而，使得用户可以根据提示信息来自主地选择是否向执行本公开技术方案的操作的电子设备、应用程序、服务器或存储介质等软件或硬件提供个人信息。
作为一种可选的但非限定性的实现方式，响应于接收到用户的主动请求，向用户发送提示信息的方式例如可以是弹窗的方式，弹窗中可以以文字的方式呈现提示信息。此外，弹窗中还可以承载供用户选择“同意”或者“不同意”向电子设备提供个人信息的选择控件。
可以理解的是，上述通知和获取用户授权过程仅是示意性的，不对本公开的实现方式构成限定，其他满足相关法律法规的方式也可应用于本公开的实现方式中。
需要说明的是，在本文中，诸如“第一”和“第二”等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且，术语“包括”、“包含”或者其任何其 他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。
以上所述仅是本公开的具体实施方式，使本领域技术人员能够理解或实现本公开。对这些实施例的多种修改对本领域的技术人员来说将是显而易见的，本文中所定义的一般原理可以在不脱离本公开的精神或范围的情况下，在其它实施例中实现。因此，本公开将不会被限制于本文所述的这些实施例，而是要符合与本文所公开的原理和新颖特点相一致的最宽的范围。

Claims (19)

  1. 一种信息获取方法，包括：
    获取待处理的目标图像；其中，所述目标图像中包含有目标对象；
    将所述目标图像输入至预设的信息预测模型；其中，所述信息预测模型包括特征提取网络、关键点预测网络和部位信息预测网络；
    通过所述特征提取网络对所述目标图像进行特征提取，得到图像特征；
    基于所述图像特征，通过所述关键点预测网络获取所述目标对象的多个身体部位对应的关键点的位置信息；
    基于所述图像特征，通过所述部位信息预测网络获取所述目标对象的身体信息；所述身体信息包括第一身体部位的长度信息和/或第二身体部位的旋转角度信息。
  2. 根据权利要求1所述的方法，其中，所述通过所述特征提取网络对所述目标图像进行特征提取，得到图像特征，包括：
    通过所述特征提取网络对所述目标图像进行多尺度特征提取，得到多种尺度的图像特征。
  3. 根据权利要求2所述的方法，其中，所述基于所述图像特征，通过所述关键点预测网络获取所述目标对象的多个身体部位对应的关键点的位置信息，包括：
    基于所述多种尺度的图像特征中的一种或多种特征，通过所述关键点预测网络获取所述目标对象的多个身体部位对应的关键点的位置信息。
  4. 根据权利要求3所述的方法，其中，所述获取所述目标对象的多个身体部位对应的关键点的位置信息，包括：
    将所述多种尺度中至少两个目标尺度的图像特征进行融合，得到融合特征；
    根据所述融合特征获取所述目标对象中目标部位的关键点对应的热力图；
    基于所述热力图预测所述目标对象中目标部位的关键点的位置信息。
  5. 根据权利要求2所述的方法，其中，所述基于所述图像特征，通过所述部位信息预测网络获取所述目标对象的身体信息，包括：
    基于所述多种尺度的图像特征中的最小尺度的特征，通过所述部位信息预测网络获取所述目标对象的身体信息；和/或，
    获取所述关键点预测网络生成的融合特征，并基于所述融合特征，通过所述部位信息预测网络获取所述目标对象的身体信息；其中，所述融合特征是所述关键点预测网络将所述多种尺度中至少两个目标尺度的图像特征进行融合所得的特征。
  6. 根据权利要求5所述的方法，其中，所述部位信息预测网络包括第一预测单元、第二预测单元和结果融合单元；
    所述第一预测单元用于基于所述多种尺度的图像特征中的最小尺度的特征，获取所述目标对象的身体信息对应的第一预测结果；
    所述第二预测单元用于基于所述融合特征，获取所述目标对象的身体信息对应的第二预测结果；
    所述结果融合单元用于根据所述第一预测结果和所述第二预测结果进行融合，以根据融合结果确定所述目标对象的身体信息。
  7. 根据权利要求6所述的方法，其中，所述根据所述第一预测结果和所述第二预测结果进行融合，以根据融合结果确定所述目标对象的身体信息，包括：
    基于所述第一预测结果和所述第二预测结果进行加权融合处理，得到加权融合结果；
    根据所述加权融合结果，确定所述目标对象的身体信息。
  8. 根据权利要求7所述的方法，其中，所述第一预测结果对应的权重不低于所述第二预测结果对应的权重。
  9. 根据权利要求7所述的方法，其中，所述第一预测单元和所述第二预测单元的网络结构相同，且所述网络结构包括全局处理层、长度信息预测层以及旋转角度预测层；其中，
    所述全局处理层用于基于所述网络结构的输入信息进行全局处理，得到全局信息；
    所述长度信息预测层用于基于所述全局信息进行第一解耦处理，以预测得到第一身体部位的长度信息；
    所述旋转角度预测层用于基于所述全局信息进行第二解耦处理，以预测得到第二身体部位的旋转角度信息。
  10. 根据权利要求1至9任一项所述的方法，其中，所述信息预测模型是按照如下步骤得到的：
    获取携带有标签信息的样本图像；其中，所述样本图像包含目标对象，且所述标签信息包括所述目标对象中多个身体部位对应的关键点的位置标签以及所述目标对象的身体信息标签；
    获取预设的神经网络模型针对所述样本图像输出的信息预测结果；所述信息预测结果包括所述样本图像中目标对象的多个身体部位对应的关键点的位置信息预测结果以及身体信息预测结果；
    基于所述标签信息和所述信息预测结果，对所述神经网络模型进行训练，以基于训练后的神经网络模型得到信息预测模型。
  11. 根据权利要求10所述的方法，其中，所述获取携带有标签信息的样本图像，包括：
    获取包含有目标对象的样本图像；
    采用预设网络模型获取所述样本图像中目标对象对应的三维立体结构的多个身体关键点；
    根据所述多个身体关键点，以及所述多个身体关键点与所述目标对象的多个身体部位对应的关键点之间的映射关系，确定所述样本图像中目标对象的多个身体部位对应的关键点的位置信息；
    根据所述样本图像中目标对象的多个身体部位对应的关键点的位置信息，确定所述样本图像中目标对象的身体信息；
    基于所述样本图像中目标对象的多个身体部位对应的关键点的位置信息以及所述样本图像中目标对象的身体信息，为所述样本图像附带标签信息。
  12. 根据权利要求11所述的方法，其中，所述根据所述样本图像中目标对象的多个身体部位对应的关键点的位置信息，确定所述样本图像中目标对象的身体信息，包括：
    根据所述样本图像中目标对象的多个身体部位对应的关键点的位置信息，确定第一身体部位对应的至少两个第一目标点以及第二身体部位对应的至少两个第二目标点；
    根据所述至少两个第一目标点，确定所述第一身体部位的长度信息；
    根据所述至少两个第二目标点，确定所述第二身体部位的旋转角度信息。
  13. 根据权利要求12所述的方法，其中，所述第一身体部位包括所述目标对象的肩部与腰部之间的部位；
    所述至少两个第一目标点包括：基于左肩部位对应的关键点与右肩部位对应的关键点之间的中心点确定的第一目标点，以及基于左腰部位对应的关键点与右腰部位对应的关键点之间的中心点确定的第一目标点。
  14. 根据权利要求12所述的方法，其中，所述第二身体部位包括肩部，所述至少两个第二目标点包括：左肩部位对应的关键点与右肩部位对应的关键点；
    和/或，
    所述第二身体部位包括腰部，所述至少两个第二目标点包括：左腰部位对应的关键点与右腰部位对应的关键点。
  15. 根据权利要求10所述的方法，其中，所述基于所述标签信息和所述信息预测结果，对所述神经网络模型进行训练，以基于训练后的神经网络模型得到信息预测模型，包括：
    基于所述位置标签和所述位置信息预测结果之间的差异，确定第一损失；
    基于所述身体信息标签和所述身体信息预测结果之间的差异，确定第二损失；
    基于所述第一损失和所述第二损失，对所述神经网络模型进行训练，以基于训练后的神经网络模型得到信息预测模型。
  16. 根据权利要求1所述的方法，其中，所述多个身体部位包括臂部和/或腿部，还包括头部、颈部、胸部、腹部和腰部中的一种或多种。
  17. 一种信息获取装置，包括：
    图像获取模块，用于获取待处理的目标图像；其中，所述目标图像中包含有目标对象；
    模型输入模块，用于将所述目标图像输入至预设的信息预测模型；其中，所述信息预测模型包括特征提取网络、关键点预测网络和部位信息预测网络；
    特征提取模块，用于通过所述特征提取网络对所述目标图像进行特征提取，得到图像特征；
    关键点位置获取模块，用于基于所述图像特征，通过所述关键点预测网络获取所述目标对象的多个身体部位对应的关键点的位置信息；
    身体信息获取模块，用于基于所述图像特征，通过所述部位信息预测网络获取所述目标对象的身体信息；所述身体信息包括第一身体部位的长度信息和/或第二身体部位的旋转角度信息。
  18. 一种电子设备，所述电子设备包括：
    存储装置，其上存储有计算机程序；
    处理装置，用于执行所述存储装置中的所述计算机程序，以实现权利要求1-16中任一项所述的信息获取方法的步骤。
  19. 一种计算机可读存储介质，所述存储介质存储有计算机程序，所述计算机程序用于执行上述权利要求1-16中任一所述的信息获取方法。
PCT/CN2024/130274 2023-11-17 2024-11-06 信息获取方法、装置、设备及介质 Pending WO2025103191A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202311541140.5 2023-11-17
CN202311541140.5A CN120020878A (zh) 2023-11-17 2023-11-17 信息获取方法、装置、设备及介质

Publications (1)

Publication Number Publication Date
WO2025103191A1 true WO2025103191A1 (zh) 2025-05-22

Family

ID=95704064

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2024/130274 Pending WO2025103191A1 (zh) 2023-11-17 2024-11-06 信息获取方法、装置、设备及介质

Country Status (2)

Country Link
CN (1) CN120020878A (zh)
WO (1) WO2025103191A1 (zh)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108229343A (zh) * 2017-12-18 2018-06-29 北京市商汤科技开发有限公司 目标对象关键点检测方法、深度学习神经网络及装置
CN115482588A (zh) * 2022-10-20 2022-12-16 聚好看科技股份有限公司 一种预测三维模型的姿态的方法及电子设备
WO2023094377A1 (en) * 2021-11-24 2023-06-01 Koninklijke Philips N.V. A method and system for body part measurement for skin treatment
CN116612495A (zh) * 2023-05-05 2023-08-18 阿里巴巴（中国）有限公司 图像处理方法及装置

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108229343A (zh) * 2017-12-18 2018-06-29 北京市商汤科技开发有限公司 目标对象关键点检测方法、深度学习神经网络及装置
WO2023094377A1 (en) * 2021-11-24 2023-06-01 Koninklijke Philips N.V. A method and system for body part measurement for skin treatment
CN115482588A (zh) * 2022-10-20 2022-12-16 聚好看科技股份有限公司 一种预测三维模型的姿态的方法及电子设备
CN116612495A (zh) * 2023-05-05 2023-08-18 阿里巴巴（中国）有限公司 图像处理方法及装置

Also Published As

Publication number Publication date
CN120020878A (zh) 2025-05-20

Similar Documents

Publication Publication Date Title
CN111260774B (zh) 生成3d关节点回归模型的方法和装置
JP2023547917A (ja) 画像分割方法、装置、機器および記憶媒体
CN111985240A (zh) 命名实体识别模型的训练方法、命名实体识别方法及装置
CN115578570A (zh) 图像处理方法、装置、可读介质及电子设备
CN113468330B (zh) 信息获取方法、装置、设备及介质
US20240311285A1 (en) Testing control method and apparatus for application, and electronic device and storage medium
CN111597465A (zh) 展示方法、装置和电子设备
WO2021217340A1 (zh) 基于ai的通用智能家居方案自动化设计方法及装置
CN117633228A (zh) 模型训练方法和装置
CN113255539A (zh) 多任务融合的人脸定位方法、装置、设备及存储介质
CN111597466A (zh) 展示方法、装置和电子设备
CN116453221B (zh) 目标对象姿态确定方法、训练方法、装置以及存储介质
CN116028820B (zh) 一种模型训练的方法、装置、存储介质及电子设备
CN111652432A (zh) 用户属性信息的确定方法、装置、电子设备及存储介质
CN116071527A (zh) 一种对象处理方法、装置、存储介质及电子设备
CN112037305B (zh) 对图像中的树状组织进行重建的方法、设备及存储介质
WO2025103191A1 (zh) 信息获取方法、装置、设备及介质
CN115187510A (zh) 回环检测方法、装置、电子设备及介质
WO2025045144A1 (zh) 图像检测方法、装置、电子设备和存储介质
CN114022570B (zh) 相机间外参的标定方法及电子设备
CN118535144A (zh) 页面生成方法和装置
WO2025103195A1 (zh) 信息获取方法、装置、设备及介质
CN117523249A (zh) 一种目标检测方法、装置、电子设备及存储介质
CN114120352A (zh) 一种人体轮廓点检测方法、装置、电子设备和存储介质
CN121352936B (zh) 试衣图像生成方法及相关设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24890548

Country of ref document: EP

Kind code of ref document: A1