CN115690843A

CN115690843A - Human body key point prediction model training and detecting method, device, equipment and medium

Info

Publication number: CN115690843A
Application number: CN202211347587.4A
Authority: CN
Inventors: 任高生; 陈增海
Original assignee: Guangzhou Cubesili Information Technology Co Ltd
Current assignee: Guangzhou Cubesili Information Technology Co Ltd
Priority date: 2022-10-31
Filing date: 2022-10-31
Publication date: 2023-02-03

Abstract

The application relates to the field of artificial intelligence and live broadcast, and provides a method, a device, equipment and a medium for human body key point prediction model training and human body key point detection. The method and the device can realize efficient and accurate prediction of the key points of the human body. The method comprises the following steps: adding a one-dimensional thermodynamic diagram prediction branch to a trained human body key point prediction model containing a two-dimensional thermodynamic diagram prediction branch to obtain a human body key point prediction model to be trained, inputting an image sample into the model to obtain one-dimensional and two-dimensional thermodynamic diagram prediction information respectively output by the one-dimensional and two-dimensional branches, respectively obtaining first and second human body key point position prediction information according to the one-dimensional and two-dimensional thermodynamic diagram prediction information, obtaining model loss according to the one-dimensional and two-dimensional thermodynamic diagram prediction information, the first and second human body key point position prediction information and the one-dimensional and two-dimensional thermodynamic diagram marking information, and training the model according to the loss to obtain the human body key point prediction model containing the one-dimensional thermodynamic diagram prediction branch applied to human body key point prediction.

Description

Human body key point prediction model training and detecting method, device, equipment and medium

Technical Field

The present application relates to the field of artificial intelligence and live broadcast technologies, and in particular, to a training method for a human key point prediction model, a human key point detection method, an apparatus, an electronic device, and a computer-readable storage medium.

Background

With the development of artificial intelligence technology, the functions of deformation, beautification and the like of the human body in the collected image are provided in various applications such as short video, live broadcast and the like, wherein the functions relate to the detection of key points of the human body.

The human body key point prediction model provided in the prior art has large learning difficulty and calculation amount, and is difficult to consider the detection accuracy and efficiency of the human body key points.

Disclosure of Invention

In view of the above, it is necessary to provide a training method for a human body key point prediction model, a human body key point detection method, an apparatus, an electronic device, and a computer-readable storage medium.

In a first aspect, the present application provides a training method for a human body key point prediction model. The method comprises the following steps:

obtaining a human body key point prediction model after first training; the first trained human keypoint prediction model includes a two-dimensional thermodynamic diagram prediction branch;

adding a one-dimensional thermodynamic diagram prediction branch based on the first trained human body key point prediction model to obtain a human body key point prediction model to be trained for the second time, wherein the human body key point prediction model comprises the one-dimensional thermodynamic diagram prediction branch and a two-dimensional thermodynamic diagram prediction branch;

inputting an image sample containing a human body into the human body key point prediction model to be trained secondly, and acquiring one-dimensional thermodynamic diagram prediction information output by the one-dimensional thermodynamic diagram prediction branch and two-dimensional thermodynamic diagram prediction information output by the two-dimensional thermodynamic diagram prediction branch;

obtaining first human body key point position prediction information corresponding to the one-dimensional thermodynamic diagram prediction branch according to the one-dimensional thermodynamic diagram prediction information, and obtaining second human body key point position prediction information corresponding to the two-dimensional thermodynamic diagram prediction branch according to the two-dimensional thermodynamic diagram prediction information;

obtaining model loss according to the one-dimensional thermodynamic diagram prediction information, the two-dimensional thermodynamic diagram prediction information, the first human body key point position prediction information, the second human body key point position prediction information, and the one-dimensional thermodynamic diagram marking information and the two-dimensional thermodynamic diagram marking information corresponding to the image sample;

and performing second training on the human body key point prediction model to be subjected to second training according to the model loss, and obtaining the human body key point prediction model containing the one-dimensional thermodynamic diagram prediction branch applied to human body key point prediction when a training completion condition is met.

In a second aspect, the present application provides a method for detecting key points of a human body. The method comprises the following steps:

acquiring a trained human body key point prediction model containing a one-dimensional thermodynamic diagram prediction branch; the human body key point prediction model is obtained by training according to the training method of the human body key point prediction model;

inputting an image to be detected containing a human body into the trained human body key point prediction model containing the one-dimensional thermodynamic diagram prediction branch, and obtaining corresponding human body key point position prediction information according to the one-dimensional thermodynamic diagram prediction information output by the one-dimensional thermodynamic diagram prediction branch;

and obtaining a human body key point position detection result of the image to be detected according to the human body key point position prediction information.

In a third aspect, the present application provides a training apparatus for a human body key point prediction model. The device comprises:

the model acquisition module is used for acquiring a human body key point prediction model after first training; the first trained human keypoint prediction model includes a two-dimensional thermodynamic diagram prediction branch;

the model obtaining module is used for adding a one-dimensional thermodynamic diagram prediction branch based on the first trained human key point prediction model to obtain a human key point prediction model to be trained for the second time, wherein the human key point prediction model comprises the one-dimensional thermodynamic diagram prediction branch and a two-dimensional thermodynamic diagram prediction branch;

the image input module is used for inputting an image sample containing a human body into the human body key point prediction model to be trained secondly, and acquiring one-dimensional thermodynamic diagram prediction information output by the one-dimensional thermodynamic diagram prediction branch and two-dimensional thermodynamic diagram prediction information output by the two-dimensional thermodynamic diagram prediction branch;

the information obtaining module is used for obtaining first human body key point position prediction information corresponding to the one-dimensional thermodynamic diagram prediction branch according to the one-dimensional thermodynamic diagram prediction information and obtaining second human body key point position prediction information corresponding to the two-dimensional thermodynamic diagram prediction branch according to the two-dimensional thermodynamic diagram prediction information;

the loss obtaining module is used for obtaining model loss according to the one-dimensional thermodynamic diagram prediction information, the two-dimensional thermodynamic diagram prediction information, the first human body key point position prediction information, the second human body key point position prediction information, and the one-dimensional thermodynamic diagram marking information and the two-dimensional thermodynamic diagram marking information corresponding to the image sample;

and the model training module is used for carrying out second training on the human body key point prediction model to be subjected to second training according to the model loss, and obtaining the human body key point prediction model containing the one-dimensional thermodynamic diagram prediction branch applied to human body key point prediction when a training completion condition is met.

In a fourth aspect, the present application provides a human keypoint detection apparatus. The device comprises:

the model obtaining module is used for obtaining a trained human body key point prediction model containing one-dimensional thermodynamic diagram prediction branches; the human body key point prediction model is obtained by training according to the training device of the human body key point prediction model;

the model processing module is used for inputting an image to be detected containing a human body into the trained human body key point prediction model containing the one-dimensional thermodynamic diagram prediction branch, and obtaining corresponding human body key point position prediction information according to the one-dimensional thermodynamic diagram prediction information output by the one-dimensional thermodynamic diagram prediction branch;

and the result acquisition module is used for acquiring a human key point position detection result of the image to be detected according to the human key point position prediction information.

In a fifth aspect, the present application provides an electronic device. The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor realizes the following steps when executing the computer program:

obtaining a human body key point prediction model after first training; the first trained human keypoint prediction model includes a two-dimensional thermodynamic diagram prediction branch; adding a one-dimensional thermodynamic diagram prediction branch based on the first trained human body key point prediction model to obtain a human body key point prediction model to be trained for the second time, wherein the human body key point prediction model comprises the one-dimensional thermodynamic diagram prediction branch and a two-dimensional thermodynamic diagram prediction branch; inputting image samples containing human bodies into the human body key point prediction model to be trained secondly, and acquiring one-dimensional thermodynamic diagram prediction information output by the one-dimensional thermodynamic diagram prediction branch and two-dimensional thermodynamic diagram prediction information output by the two-dimensional thermodynamic diagram prediction branch; obtaining first human body key point position prediction information corresponding to the one-dimensional thermodynamic diagram prediction branch according to the one-dimensional thermodynamic diagram prediction information, and obtaining second human body key point position prediction information corresponding to the two-dimensional thermodynamic diagram prediction branch according to the two-dimensional thermodynamic diagram prediction information; obtaining model loss according to the one-dimensional thermodynamic diagram prediction information, the two-dimensional thermodynamic diagram prediction information, the first human body key point position prediction information, the second human body key point position prediction information, and the one-dimensional thermodynamic diagram marking information and the two-dimensional thermodynamic diagram marking information corresponding to the image sample; and performing second training on the human key point prediction model to be subjected to second training according to the model loss, and obtaining the human key point prediction model containing the one-dimensional thermodynamic diagram prediction branch applied to human key point prediction when training completion conditions are met.

In a sixth aspect, the present application provides an electronic device. The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor realizes the following steps when executing the computer program:

acquiring a trained human body key point prediction model containing a one-dimensional thermodynamic diagram prediction branch; the human body key point prediction model is obtained by training according to the training method of the human body key point prediction model; inputting an image to be detected containing a human body into the trained human body key point prediction model containing the one-dimensional thermodynamic diagram prediction branch, and obtaining corresponding human body key point position prediction information according to the one-dimensional thermodynamic diagram prediction information output by the one-dimensional thermodynamic diagram prediction branch; and obtaining a human body key point position detection result of the image to be detected according to the human body key point position prediction information.

In a seventh aspect, the present application provides a computer-readable storage medium. The computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of:

acquiring a human body key point prediction model after first training; the first trained human keypoint prediction model includes a two-dimensional thermodynamic diagram prediction branch; adding a one-dimensional thermodynamic diagram prediction branch based on the first trained human body key point prediction model to obtain a human body key point prediction model to be trained secondly, wherein the human body key point prediction model comprises the one-dimensional thermodynamic diagram prediction branch and a two-dimensional thermodynamic diagram prediction branch; inputting an image sample containing a human body into the human body key point prediction model to be trained secondly, and acquiring one-dimensional thermodynamic diagram prediction information output by the one-dimensional thermodynamic diagram prediction branch and two-dimensional thermodynamic diagram prediction information output by the two-dimensional thermodynamic diagram prediction branch; obtaining first human body key point position prediction information corresponding to the one-dimensional thermodynamic diagram prediction branch according to the one-dimensional thermodynamic diagram prediction information, and obtaining second human body key point position prediction information corresponding to the two-dimensional thermodynamic diagram prediction branch according to the two-dimensional thermodynamic diagram prediction information; obtaining model loss according to the one-dimensional thermodynamic diagram prediction information, the two-dimensional thermodynamic diagram prediction information, the first human body key point position prediction information, the second human body key point position prediction information, and the one-dimensional thermodynamic diagram marking information and the two-dimensional thermodynamic diagram marking information corresponding to the image sample; and performing second training on the human key point prediction model to be subjected to second training according to the model loss, and obtaining the human key point prediction model containing the one-dimensional thermodynamic diagram prediction branch applied to human key point prediction when training completion conditions are met.

In an eighth aspect, the present application provides a computer-readable storage medium. The computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of:

The training method of the human body key point prediction model, the human body key point detection method, the device, the electronic equipment and the storage medium are used for obtaining the human body key point prediction model containing the two-dimensional thermodynamic diagram prediction branch after first training and adding the one-dimensional thermodynamic diagram prediction branch to obtain the human body key point prediction model to be trained secondly, wherein the human body key point prediction model contains the one-dimensional thermodynamic diagram prediction branch and the two-dimensional thermodynamic diagram prediction branch, an image sample containing a human body is input into the model, one-dimensional thermodynamic diagram prediction information output by the one-dimensional thermodynamic diagram prediction branch and two-dimensional thermodynamic diagram prediction information output by the two-dimensional thermodynamic diagram prediction branch are obtained, first human body key point position prediction information and second human body key point position prediction information are obtained according to the one-dimensional thermodynamic diagram prediction information, model losses are obtained according to the one-dimensional thermodynamic diagram prediction information, the first human body key point position prediction information, the second human body key point prediction information and the one-dimensional thermodynamic diagram prediction information and the two-dimensional thermodynamic diagram labeling information corresponding to the image sample, the model losses are obtained according to train the model, and the human body key point prediction model containing the one-dimensional thermodynamic diagram prediction branch applied to human body key point prediction is obtained when the training completion condition is met. According to the scheme, the one-dimensional thermodynamic diagram prediction branch is added on the basis of a trained human body key point prediction model containing the two-dimensional thermodynamic diagram prediction branch, then the human body key point prediction model containing the one-dimensional thermodynamic diagram prediction branch and the two-dimensional thermodynamic diagram prediction branch is trained, the model learning difficulty and the calculated amount can be reduced, the one-dimensional thermodynamic diagram prediction branch and the two-dimensional thermodynamic diagram prediction branch can accurately predict the human body key point, the lighter one-dimensional thermodynamic diagram prediction branch in the model is used for prediction during application, efficient and accurate prediction of the human body key point is achieved, and the detection accuracy and efficiency of the human body key point are considered.

Drawings

FIG. 1 is a diagram of an application environment of a related method in an embodiment of the present application;

FIG. 2 is a schematic flowchart of a training method of a human body key point prediction model in an embodiment of the present application;

FIG. 3 is a schematic flowchart of the step of obtaining model loss in the embodiment of the present application;

FIG. 4 is a diagram of a human body keypoint prediction model in an embodiment of the present application;

FIG. 5 is a schematic flow chart of a method for detecting key points of a human body according to an embodiment of the present application;

FIG. 6 is a schematic diagram illustrating a process of detecting key points of a human body in an application example of the present application;

FIG. 7 is a block diagram of a training apparatus for a human keypoint prediction model in an embodiment of the present application;

FIG. 8 is a block diagram of a human body key point detection device according to an embodiment of the present disclosure;

fig. 9 is an internal structural view of an electronic device in an embodiment of the present application;

fig. 10 is an internal structural diagram of an electronic device in another embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of and not restrictive on the broad application.

The training method of the human body key point prediction model and the human body key point detection method provided by the embodiment of the application can be applied to an application environment shown in fig. 1, the application environment can comprise a terminal and a server, and the terminal is communicated with the server through a network. Specifically, the training method of the human key point prediction model provided by the embodiment of the present application may be executed by a server, and the human key point detection method provided by the embodiment of the present application may be applied to a terminal, that is, the server may train the human key point prediction model to obtain the trained human key point prediction model including the one-dimensional thermodynamic diagram prediction branch, and then the server may send the human key point prediction model to the terminal, and the terminal may perform human key point detection on an image to be detected by using the human key point prediction model including the one-dimensional thermodynamic diagram prediction branch. In the application environment, the terminal can be but is not limited to various personal computers, notebook computers, smart phones and tablet computers; the server may be implemented as a stand-alone server or as a server cluster consisting of a plurality of servers.

The following describes a training method and a human body key point detection method of a human body key point prediction model provided by the present application in sequence based on an application environment shown in fig. 1 with reference to various embodiments and corresponding drawings.

In one embodiment, as shown in fig. 2, a method for training a human body key point prediction model is provided, which may include:

step S201, a human body key point prediction model after first training is obtained.

And S202, adding a one-dimensional thermodynamic diagram prediction branch based on the human body key point prediction model subjected to the first training to obtain a human body key point prediction model to be subjected to second training, which comprises the one-dimensional thermodynamic diagram prediction branch and the two-dimensional thermodynamic diagram prediction branch.

The above steps S201 and S202 are related steps of forming a human body key point prediction model to be trained secondly. Specifically, in step S201, a first trained human key point prediction model is obtained, where the first trained human key point prediction model is a human key point prediction model including a two-dimensional thermodynamic diagram prediction branch, that is, the first trained human key point prediction model has a human key point prediction capability and has a two-dimensional thermodynamic diagram prediction branch, the first trained human key point prediction model can perform human key point prediction based on a two-dimensional thermodynamic diagram, and can obtain corresponding two-dimensional thermodynamic diagram prediction information from the two-dimensional thermodynamic diagram prediction branch according to an input image, where the two-dimensional thermodynamic diagram prediction information is a two-dimensional thermodynamic diagram corresponding to a human key point predicted by the two-dimensional thermodynamic diagram prediction branch, and a human key point detection result can be obtained according to the two-dimensional thermodynamic diagram prediction information, where the human key point detection result is specifically a position coordinate of a human key point on an image. In step S202, a one-dimensional thermodynamic diagram prediction branch is added to the first trained human keypoint prediction model, so as to form a human keypoint prediction model to be trained for the second time, which includes the one-dimensional thermodynamic diagram prediction branch and the two-dimensional thermodynamic diagram prediction branch. Correspondingly, the one-dimensional thermodynamic diagram prediction branch is used for outputting one-dimensional thermodynamic diagram prediction information, and the one-dimensional thermodynamic diagram prediction information refers to one-dimensional thermodynamic diagrams corresponding to predicted human body key points output by the one-dimensional thermodynamic diagram prediction branch. Therefore, after the one-dimensional thermodynamic prediction branch is added, a human body key point prediction model comprising the one-dimensional thermodynamic prediction branch and the two-dimensional thermodynamic prediction branch can be obtained, the human body key point prediction model comprising the two branches needs to be trained in subsequent related steps to be distinguished from the first trained human body key point prediction model comprising the two-dimensional thermodynamic prediction branch, and after the one-dimensional thermodynamic prediction branch is added, the human body key point prediction model is marked as a human body key point prediction model to be trained for the second time, wherein the human body key point prediction model comprises the one-dimensional thermodynamic prediction branch and the two-dimensional thermodynamic prediction branch. In summary, the steps S201 and S202 are to add a one-dimensional thermodynamic diagram prediction branch to train on the basis of the converged human critical point prediction model including the two-dimensional thermodynamic diagram prediction branch.

Step S203, inputting the image sample containing the human body into a human body key point prediction model to be trained for the second time, and acquiring one-dimensional thermodynamic diagram prediction information output by the one-dimensional thermodynamic diagram prediction branch and two-dimensional thermodynamic diagram prediction information output by the two-dimensional thermodynamic diagram prediction branch.

And step S204, obtaining first human body key point position prediction information corresponding to the one-dimensional thermodynamic diagram prediction branch according to the one-dimensional thermodynamic diagram prediction information, and obtaining second human body key point position prediction information corresponding to the two-dimensional thermodynamic diagram prediction branch according to the two-dimensional thermodynamic diagram prediction information.

Step S203 and step S204 are processes of inputting an image sample including a human body into a human body key point prediction model to be trained for the second time and then forming related data information. Specifically, in step S203, an image sample including a human body is input into a human body key point prediction model to be trained for the second time, where the human body key point prediction model includes a one-dimensional thermodynamic prediction branch and a two-dimensional thermodynamic prediction branch, the one-dimensional thermodynamic prediction branch outputs one-dimensional thermodynamic prediction information corresponding to the image sample, and the two-dimensional thermodynamic prediction branch outputs two-dimensional thermodynamic prediction information corresponding to the image sample. Next, in step S204, further, corresponding human key position prediction information (denoted as first human key position prediction information) is obtained according to the one-dimensional thermodynamic diagram prediction information output by the one-dimensional thermodynamic diagram prediction branch, and corresponding human key position prediction information (denoted as second human key position prediction information) is obtained according to the two-dimensional thermodynamic diagram prediction information output by the two-dimensional thermodynamic diagram prediction branch, where the first and second human key position prediction information may be position coordinates of a human key on an image.

As an embodiment, the step S204 specifically includes: aiming at the one-dimensional thermodynamic diagram prediction information corresponding to each human body key point, obtaining first human body key point position prediction information according to the position information enabling the heat data value in the one-dimensional thermodynamic diagram prediction information to be maximum; and aiming at the two-dimensional thermodynamic diagram prediction information corresponding to each human body key point, obtaining second human body key point position prediction information according to the position information which enables the heat data value in the two-dimensional thermodynamic diagram prediction information to be maximum.

In this embodiment, each human body key point to be predicted may correspond to one piece of one-dimensional thermodynamic diagram prediction information and one piece of two-dimensional thermodynamic diagram prediction information, and thus, for each human body key point, the first human body key point position prediction information may be calculated according to the one-dimensional thermodynamic diagram prediction information corresponding to the human body key point, and the second human body key point position prediction information may be calculated according to the two-dimensional thermodynamic diagram prediction information corresponding to the human body key point. Specifically, for each human body key point, first human body key point position prediction information is obtained according to position information enabling the heat data value in the one-dimensional heat diagram prediction information to be maximum, second human body key point position prediction information is obtained according to position information enabling the heat data value in the two-dimensional heat diagram prediction information to be maximum, and the one-dimensional heat diagram prediction information and the two-dimensional heat diagram prediction information can be processed through an argmax function to obtain corresponding first human body key point position prediction information and second human body key point position prediction information, so that the first human body key point position prediction information and the second human body key point position prediction information can be obtained through calculation according to the one-dimensional heat diagram prediction information and the two-dimensional heat diagram prediction information in an efficient and gradual changing mode.

And S205, obtaining model loss according to the one-dimensional thermodynamic diagram prediction information, the two-dimensional thermodynamic diagram prediction information, the first human body key point position prediction information, the second human body key point position prediction information, and the one-dimensional thermodynamic diagram marking information and the two-dimensional thermodynamic diagram marking information corresponding to the image sample.

This step is a step of acquiring the model loss. Specifically, the one-dimensional thermodynamic diagram prediction information and the first human body key point position prediction information from the one-dimensional thermodynamic diagram prediction branch, and the two-dimensional thermodynamic diagram prediction information and the second human body key point position prediction information from the two-dimensional thermodynamic diagram prediction branch can be obtained through the foregoing steps S203 and S204. In addition, one-dimensional thermodynamic diagram labeling information and two-dimensional thermodynamic diagram labeling information corresponding to the image sample are also acquired, wherein the one-dimensional thermodynamic diagram labeling information is a one-dimensional thermodynamic diagram generated according to position labeling of key points of a human body in the image sample and is used for monitoring output of a one-dimensional thermodynamic diagram prediction branch, and the two-dimensional thermodynamic diagram labeling information is a two-dimensional thermodynamic diagram generated according to position labeling of key points of the human body in the image sample and is used for monitoring output of a two-dimensional thermodynamic diagram prediction branch. One-dimensional thermodynamic diagram prediction information, first human body key point position prediction information, two-dimensional thermodynamic diagram prediction information and second human body key point position prediction information from the one-dimensional thermodynamic diagram prediction branch, and one-dimensional thermodynamic diagram marking information and two-dimensional thermodynamic diagram marking information generated by marking the positions of human body key points in the image sample are obtained, and then model loss for representing and supervising the output of the one-dimensional thermodynamic diagram prediction branch and the output of the two-dimensional thermodynamic diagram prediction branch in the model is obtained according to the information.

Specifically, in an embodiment, as shown in fig. 3, step S205 further includes:

and S301, acquiring a first model loss according to the consistency of the one-dimensional thermodynamic diagram prediction information and the one-dimensional thermodynamic diagram marking information.

In the step, for supervision of the one-dimensional thermodynamic diagram predicted branch, the first model loss is obtained according to the consistency of the output one-dimensional thermodynamic diagram predicted information and the one-dimensional thermodynamic diagram labeling information. Specifically, for the calculation of the first model loss in step S301, the first model loss may be obtained according to the mean square error between the one-dimensional thermodynamic diagram prediction information and the one-dimensional thermodynamic diagram labeling information, that is, the mean square error mselos is used to calculate the first model loss, and the specific calculation formula may be represented as:

wherein mse _1D Representing the loss of the first model, n representing the number of key points of the human body, i representing the serial number of the key points of the human body, y1 _i Representing one-dimensional thermodynamic diagram annotation information, y _i ' represents one-dimensional thermodynamic diagram prediction information.

And step S302, acquiring a second model loss according to the consistency of the two-dimensional thermodynamic diagram prediction information and the two-dimensional thermodynamic diagram marking information.

In the step, for supervision of the two-dimensional thermodynamic diagram prediction branch, a second model loss is obtained according to consistency of the two-dimensional thermodynamic diagram prediction information and the two-dimensional thermodynamic diagram marking information which are output by the two-dimensional thermodynamic diagram prediction branch. Specifically, for the calculation of the second model loss in step S302, the second model loss may also be obtained according to a mean square error of the two-dimensional thermodynamic prediction information and the two-dimensional thermodynamic annotation information, that is, the second model loss may also be calculated by using a mean square error mselos, where a specific calculation formula may be represented as:

wherein mse _2D Representing the second model loss, y2 _i Representing two-dimensional thermodynamic diagram annotation information,

representing two-dimensional thermodynamic diagram prediction information.

And step S303, acquiring a third model loss according to the consistency of the first human body key point position prediction information and the second human body key point position prediction information.

In this step, the one-dimensional thermodynamic diagram is used for predicting branches, the two-dimensional thermodynamic diagram is used for predicting the branches and monitoring the branches, and specifically, a third model loss is obtained according to consistency of first human body key point position prediction information from the one-dimensional thermodynamic diagram prediction branches and second human body key point position prediction information from the two-dimensional thermodynamic diagram prediction branches. Specifically, for the calculation of the third model loss in step S303, the third model loss may be obtained according to the absolute error between the first human body key point position prediction information and the second human body key point position prediction information, that is, the third model loss may be calculated by using the absolute error L1loss, and the specific calculation formula may be represented as:

wherein L1 represents the third model loss,

and z' represents the first human body key point position prediction information.

And step S304, obtaining model loss according to the first model loss, the second model loss and the third model loss.

The step integrates the first model loss, the second model loss and the third model loss to obtain the total model loss. Specifically, the first model loss, the second model loss, and the third model loss may be weighted to obtain the total model loss, such as loss = mse _2D +mse _1D +L1。

According to the scheme of the embodiment, the one-dimensional thermodynamic diagram prediction branch and the two-dimensional thermodynamic diagram prediction branch can be supervised by using the one-dimensional thermodynamic diagram labeling information and the two-dimensional thermodynamic diagram labeling information respectively, consistency constraint is also performed on output results of the one-dimensional thermodynamic diagram prediction branch and the two-dimensional thermodynamic diagram prediction branch, the output results of the one-dimensional thermodynamic diagram prediction branch are supervised by using the output results of the two-dimensional thermodynamic diagram prediction branch, the learning difficulty and the calculation amount of the model are reduced, and the human body key points can be accurately detected by using both the one-dimensional thermodynamic diagram prediction branch and the two-dimensional thermodynamic diagram prediction branch.

And S206, performing second training on the human key point prediction model to be subjected to second training according to the model loss, and obtaining the human key point prediction model containing the one-dimensional thermodynamic diagram prediction branch applied to human key point prediction when the training completion condition is met.

In this step, a second training is performed on the human key point prediction model to be subjected to the second training according to the model loss obtained in step S205, the model parameters are continuously updated, and the trained human key point prediction model including the one-dimensional thermodynamic diagram prediction branch and the two-dimensional thermodynamic diagram prediction branch can be obtained when the training completion condition is satisfied. The training completion condition may be, for example, that the model loss is less than or equal to a preset model loss value. The method comprises the steps of simultaneously training a one-dimensional thermodynamic diagram prediction branch and a two-dimensional thermodynamic diagram prediction branch in a model training process, obtaining a human key point prediction model comprising the one-dimensional thermodynamic diagram prediction branch and the two-dimensional thermodynamic diagram prediction branch when the model training is finished, and applying the one-dimensional thermodynamic diagram prediction branch to human key point prediction, namely applying the one-dimensional thermodynamic diagram prediction branch and the two-dimensional thermodynamic diagram prediction branch to carry out human key point prediction during the training process, namely obtaining the human key point prediction model comprising the one-dimensional thermodynamic diagram prediction branch applied to human key point prediction.

The method for training the human key point prediction model comprises the steps of obtaining a first trained human key point prediction model containing a two-dimensional thermodynamic diagram prediction branch, adding the one-dimensional thermodynamic diagram prediction branch to obtain a human key point prediction model to be trained for the second time, inputting an image sample containing a human body into the model, obtaining one-dimensional thermodynamic diagram prediction information output by the one-dimensional thermodynamic diagram prediction branch and two-dimensional thermodynamic diagram prediction information output by the two-dimensional thermodynamic diagram prediction branch, obtaining first human key point position prediction information according to the one-dimensional thermodynamic diagram prediction information and second human key point position prediction information according to the two-dimensional thermodynamic diagram prediction information, obtaining model loss according to the one-dimensional thermodynamic diagram prediction information, the first human key point position prediction information, the second human key point position prediction information and one-dimensional and two-dimensional thermodynamic diagram labeling information corresponding to the image sample, conducting second training on the model according to the model loss, and obtaining the human key point prediction model containing the one-dimensional thermodynamic diagram prediction branch applied to human key point prediction when training completion conditions are met. According to the scheme, the one-dimensional thermodynamic diagram prediction branch is added on the basis of a trained human body key point prediction model containing the two-dimensional thermodynamic diagram prediction branch, then the human body key point prediction model containing the one-dimensional thermodynamic diagram prediction branch and the two-dimensional thermodynamic diagram prediction branch is trained, the model learning difficulty and the calculated amount can be reduced, the one-dimensional thermodynamic diagram prediction branch and the two-dimensional thermodynamic diagram prediction branch can accurately predict the human body key point, the lighter one-dimensional thermodynamic diagram prediction branch in the model is used for prediction during application, efficient and accurate prediction of the human body key point is achieved, and the detection accuracy and efficiency of the human body key point are considered.

In one embodiment, the first trained human key point prediction model further includes a feature extraction network, and the step S203 of inputting the image sample including the human body into the human key point prediction model to be trained for the second time, and obtaining the one-dimensional thermodynamic diagram prediction information output by the one-dimensional thermodynamic diagram prediction branch and the two-dimensional thermodynamic diagram prediction information output by the two-dimensional thermodynamic diagram prediction branch specifically includes:

inputting an image sample containing a human body into a human body key point prediction model to be trained secondly, obtaining a shared feature map by a feature extraction network in the human body key point prediction model to be trained secondly according to the image sample, respectively transmitting the shared feature map to a one-dimensional thermodynamic diagram prediction branch and a two-dimensional thermodynamic diagram prediction branch, outputting one-dimensional thermodynamic diagram prediction information by the one-dimensional thermodynamic diagram prediction branch according to the shared feature map, and outputting two-dimensional thermodynamic diagram prediction information by the two-dimensional thermodynamic diagram prediction branch according to the shared feature map.

Specifically, as shown in fig. 4, the first trained human keypoint prediction model may specifically include a feature extraction network and a two-dimensional thermodynamic diagram prediction branch, and the human keypoint prediction model to be trained secondly may be obtained by adding a one-dimensional thermodynamic diagram prediction branch to the first trained human keypoint prediction model, that is, the human keypoint prediction model to be trained secondly may specifically include a feature extraction network, a one-dimensional thermodynamic diagram prediction branch, and a two-dimensional thermodynamic diagram prediction branch. Therefore, when a human body key point prediction model to be subjected to second training is subjected to second training, image samples containing a human body are input into the human body key point prediction model to be subjected to second training, a shared feature map is obtained by feature extraction of the image samples through a feature extraction network, the shared feature map is shared by a one-dimensional thermodynamic prediction branch and a two-dimensional thermodynamic prediction branch respectively, the shared feature map can be transferred to the one-dimensional thermodynamic prediction branch and the two-dimensional thermodynamic prediction branch respectively through the feature extraction network, the one-dimensional thermodynamic prediction branch can predict n human body key points respectively according to the shared feature map, specifically, a resurpe operation and a cony operation of v1 × 1 can be performed on the shared feature map, so that corresponding one-dimensional thermodynamic prediction information is obtained and output, the one-dimensional thermodynamic prediction information can obtain corresponding coordinates (such as x1, y1 and the like) through an argmax function, namely first human body key point position prediction information, correspondingly, the two-dimensional thermodynamic prediction branch can obtain corresponding two-dimensional key point position prediction information, the two-dimensional thermodynamic prediction branch can obtain n human body key point position prediction information according to the shared feature map prediction branch, the two-dimensional thermodynamic prediction information and the two-dimensional thermodynamic prediction information can be obtained, the two-dimensional thermodynamic prediction branch prediction information and the two-dimensional key point prediction information obtained through the argmax function, and the second-dimensional thermodynamic prediction information obtained, and the two-dimensional thermodynamic prediction information obtained through the second-dimensional thermodynamic prediction point prediction branch can be obtained, namely, the second high-dimensional thermodynamic prediction information, and the second high-dimensional thermodynamic prediction information obtained.

As for the Feature extraction network, as shown in fig. 4, in some embodiments, an FPN (Feature Pyramid network) structure may be specifically adopted, and the FPN effectively fuses Feature maps at different depths through two processes, namely, from top to bottom and from bottom to top, so that accuracy of detecting a keypoint can be improved. In a specific implementation, the input size of an image sample is (H, W) for a one-dimensional thermodynamic prediction branch, the output size of the one-dimensional thermodynamic prediction branch may be (kH, kW), k = [1,2, \8230;), and the output size of the two-dimensional thermodynamic prediction branch is (H/m, W/m), m is a multiple of the downsampling of a feature pyramid network, such as 4, 8, and the like, and the error of the output mode adopted in the present application is [0,1/2k ], which may reach a sub-pixel level.

Further, in an embodiment, the feature extraction network may include a first feature extraction network and a second feature extraction network that are cascaded, in the embodiment, the feature extraction network in the human body keypoint prediction model to be trained secondly obtains a shared feature map according to the image sample, and respectively transfers the shared feature map to the one-dimensional thermodynamic diagram prediction branch and the two-dimensional thermodynamic diagram prediction branch, which may specifically include:

and obtaining an initial shared feature map by a first feature extraction network in the human body key point prediction model to be trained secondly according to the image sample, transmitting the initial shared feature map to a second feature extraction network, obtaining a shared feature map by the second feature extraction network according to the initial shared feature map, and transmitting the shared feature map to the one-dimensional thermodynamic diagram prediction branch and the two-dimensional thermodynamic diagram prediction branch respectively.

In this embodiment, the feature extraction network may include a first feature extraction network and a second feature extraction network that are cascaded, where the first feature extraction network and the second feature extraction network may both adopt a feature pyramid network, that is, the feature extraction network may include two feature pyramid networks that are cascaded, so that, first, an initial shared feature map may be obtained from a first feature extraction network in a model, that is, the first feature pyramid network, according to an image sample, and then the first feature pyramid network transfers the initial shared feature map to a second feature extraction network, that is, the second feature pyramid network, obtains a shared feature map from the second feature pyramid network, and the second feature pyramid transfers the shared feature map to the one-dimensional thermal prediction branch and the two-dimensional thermal prediction branch, respectively, so as to complete output of corresponding prediction information. That is, in this embodiment, the input of the second feature extraction network is the output of the first feature extraction network, which can implement more refined prediction based on the feature map output by the first feature extraction network, and further improve the accuracy and stability of the key point prediction, while based on the cascaded feature pyramid network structure, the accuracy and stability of the human key point prediction in the large-posture scenes such as the lateral body and the like can be improved.

In one embodiment, as shown in fig. 5, a human body key point detection method is provided, which may be applied to the terminal in fig. 1, and the method may include the following steps:

step S501, a trained human body key point prediction model containing one-dimensional thermodynamic diagram prediction branches is obtained.

The human key point prediction model is obtained by training according to the training method of the human key point prediction model provided by the embodiment of the application.

Step S502, inputting the image to be detected containing the human body into a trained human body key point prediction model containing a one-dimensional thermodynamic diagram prediction branch, and obtaining corresponding human body key point position prediction information according to the one-dimensional thermodynamic diagram prediction information output by the one-dimensional thermodynamic diagram prediction branch.

And S503, obtaining a human body key point position detection result of the image to be detected according to the human body key point position prediction information.

In this embodiment, after the server trains the human key point prediction model including the one-dimensional thermodynamic diagram prediction branch applied to human key point prediction according to the training method of the human key point prediction model provided in the embodiment of the present application, the server may send the human key point prediction model to the terminal, and the terminal obtains the trained human key point prediction model including the one-dimensional thermodynamic diagram prediction branch. Then the terminal can obtain an image to be detected containing a human body and input the image to the human body key point prediction model, obtain one-dimensional thermodynamic diagram prediction information output by a one-dimensional thermodynamic diagram prediction branch in the model, then obtain corresponding human body key point position prediction information such as position coordinates corresponding to each human body key point through argmax function of the one-dimensional thermodynamic diagram prediction information, and then obtain a human body key point position detection result of the image to be detected according to the human body key point position prediction information, wherein the human body key point position detection result can be specifically a mark of the predicted position coordinates of the human body key points on the image to be detected.

More specifically, as shown in fig. 6, after an original image is collected, a terminal may first detect whether a human body is included in the original image by using a human body detector, if no human body is present, the process is ended, if a human body is present, the human body detector extracts a human body region, cuts the detected human body region from the original image, obtains an image to be detected including a human body, and then performs human body keypoint detection based on the image to be detected including a human body, that is, the image to be detected including a human body is input into the trained human body keypoint prediction model including the one-dimensional thermodynamic diagram prediction branch, obtains corresponding human body keypoint position prediction information according to the one-dimensional thermodynamic diagram prediction information output by the one-dimensional thermodynamic diagram prediction branch, obtains a human body keypoint position detection result of the image to be detected according to the human body keypoint position prediction information, for example, marks position coordinates of human keypoints on the image to be detected, thereby may also perform inverse transformation of human body region cutting in a preamble step on the human body keypoint position detection result of the image to be detected, and map the corresponding human body position detection result back to the original image, thereby obtaining a human body keypoint position detection result of the human body in the original image.

The scheme of the application example can realize real-time prediction of the human key points, can be particularly applied to human key point detection of images shot by the camera in a live webcast scene, can further process the human body in the images in aspects of long legs, body beauty and the like based on the human key point detection result, and is favorable for improving the efficiency, the precision and the stability of image processing in aspects of long legs, body beauty and the like.

It should be understood that, although the steps in the flowcharts related to the embodiments as described above are sequentially displayed as indicated by arrows, the steps are not necessarily performed sequentially as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a part of the steps in the flowcharts related to the embodiments described above may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, and the execution order of the steps or stages is not necessarily sequential, but may be performed alternately or alternately with other steps or at least a part of the steps or stages in other steps.

Based on the same inventive concept, the embodiment of the present application further provides a related apparatus for implementing the related method. The implementation scheme for solving the problem provided by the apparatus is similar to the implementation scheme described in the above method, so specific limitations in one or more embodiments of the related apparatus provided below can be referred to the limitations of the related method above, and are not described herein again.

In one embodiment, as shown in fig. 7, there is provided an apparatus for training a human body keypoint prediction model, where the apparatus 700 may include:

a model obtaining module 701, configured to obtain a first trained human key point prediction model; the first trained human keypoint prediction model includes a two-dimensional thermodynamic diagram prediction branch;

a model obtaining module 702, configured to add a one-dimensional thermodynamic diagram prediction branch based on the first trained human key point prediction model to obtain a human key point prediction model to be trained for the second time, where the human key point prediction model includes the one-dimensional thermodynamic diagram prediction branch and a two-dimensional thermodynamic diagram prediction branch;

an image input module 703, configured to input an image sample including a human body into the human body key point prediction model to be trained for the second time, and obtain one-dimensional thermodynamic diagram prediction information output by the one-dimensional thermodynamic diagram prediction branch and two-dimensional thermodynamic diagram prediction information output by the two-dimensional thermodynamic diagram prediction branch;

an information obtaining module 704, configured to obtain, according to the one-dimensional thermodynamic diagram prediction information, first human body key point position prediction information corresponding to the one-dimensional thermodynamic diagram prediction branch, and obtain, according to the two-dimensional thermodynamic diagram prediction information, second human body key point position prediction information corresponding to the two-dimensional thermodynamic diagram prediction branch;

a loss obtaining module 705, configured to obtain a model loss according to the one-dimensional thermodynamic diagram prediction information, the two-dimensional thermodynamic diagram prediction information, the first human body key point position prediction information, the second human body key point position prediction information, and the one-dimensional thermodynamic diagram annotation information and the two-dimensional thermodynamic diagram annotation information corresponding to the image sample;

and the model training module 706 is configured to perform second training on the human key point prediction model to be subjected to second training according to the model loss, and obtain a human key point prediction model including the one-dimensional thermodynamic diagram prediction branch applied to human key point prediction when a training completion condition is met.

In one embodiment, the loss obtaining module 705 is configured to obtain a first model loss according to consistency between the one-dimensional thermodynamic diagram prediction information and the one-dimensional thermodynamic diagram labeling information; acquiring a second model loss according to the consistency of the two-dimensional thermodynamic diagram prediction information and the two-dimensional thermodynamic diagram marking information; acquiring a third model loss according to the consistency of the first human body key point position prediction information and the second human body key point position prediction information; and obtaining the model loss according to the first model loss, the second model loss and the third model loss.

In one embodiment, the loss obtaining module 705 is configured to obtain the first model loss according to a mean square error of the one-dimensional thermodynamic diagram prediction information and the one-dimensional thermodynamic diagram labeling information; obtaining the loss of the second model according to the mean square error of the two-dimensional thermodynamic diagram prediction information and the two-dimensional thermodynamic diagram marking information; and acquiring the loss of the third model according to the absolute error of the first human body key point position prediction information and the second human body key point position prediction information.

In one embodiment, the first trained human keypoint prediction model further comprises a feature extraction network; an image input module 703, configured to input the image sample including the human body into the human body keypoint prediction model to be trained secondly, obtain a shared feature map according to the image sample by the feature extraction network in the human body keypoint prediction model to be trained secondly, transfer the shared feature map to the one-dimensional thermodynamic prediction branch and the two-dimensional thermodynamic prediction branch respectively, output the one-dimensional thermodynamic prediction information according to the shared feature map by the one-dimensional thermodynamic prediction branch, and output the two-dimensional thermodynamic prediction information according to the shared feature map by the two-dimensional thermodynamic prediction branch.

In one embodiment, the feature extraction network comprises a first feature extraction network and a second feature extraction network in cascade; an image input module 703 is configured to obtain, by the first feature extraction network in the human key point prediction model to be trained secondly, an initial shared feature map according to the image sample, transfer the initial shared feature map to the second feature extraction network, obtain, by the second feature extraction network, the shared feature map according to the initial shared feature map, and transfer the shared feature map to the one-dimensional thermodynamic prediction branch and the two-dimensional thermodynamic prediction branch, respectively.

In an embodiment, the information obtaining module 704 is configured to, for the one-dimensional thermodynamic diagram prediction information corresponding to each human body keypoint, obtain the first human body keypoint location prediction information according to the location information that maximizes the thermal data value in the one-dimensional thermodynamic diagram prediction information; and aiming at the two-dimensional thermodynamic diagram prediction information corresponding to each human body key point, obtaining the second human body key point position prediction information according to the position information which enables the heat data value in the two-dimensional thermodynamic diagram prediction information to be maximum.

In one embodiment, as shown in fig. 8, a human body keypoint detection apparatus is provided, the apparatus 800 may comprise:

a model obtaining module 801, configured to obtain a trained human body key point prediction model including a one-dimensional thermodynamic diagram prediction branch; the human body key point prediction model is obtained by training according to the training device of the human body key point prediction model;

the model processing module 802 is configured to input an image to be detected including a human body into the trained human body key point prediction model including the one-dimensional thermodynamic diagram prediction branch, and obtain corresponding human body key point position prediction information according to the one-dimensional thermodynamic diagram prediction information output by the one-dimensional thermodynamic diagram prediction branch;

and the result obtaining module 803 is configured to obtain a human body key point position detection result of the image to be detected according to the human body key point position prediction information.

The various modules in the related devices described above may be implemented in whole or in part by software, hardware, and combinations thereof. The modules can be embedded in a hardware form or independent of a processor in the electronic device, or can be stored in a memory in the electronic device in a software form, so that the processor can call and execute operations corresponding to the modules.

In one embodiment, an electronic device is provided, which may be a server, and its internal structure diagram may be as shown in fig. 9. The electronic device includes a processor, a memory, and a network interface connected by a system bus. Wherein the processor of the electronic device is configured to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the electronic device may be used to store data such as image samples. The network interface of the electronic device is used for connecting and communicating with an external terminal through a network. The computer program is executed by a processor to implement a method of training a human keypoint prediction model.

In one embodiment, an electronic device is provided, which may be a terminal, and an internal structure thereof may be as shown in fig. 10. The electronic device comprises a processor, a memory, a communication interface, a display screen and an input device which are connected through a system bus. Wherein the processor of the electronic device is configured to provide computing and control capabilities. The memory of the electronic equipment comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The communication interface of the electronic device is used for carrying out wired or wireless communication with an external terminal, and the wireless communication can be realized through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. The computer program is executed by a processor to implement a method of human keypoint detection. The display screen of the electronic equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on the shell of the electronic equipment, an external keyboard, a touch pad or a mouse and the like.

Those skilled in the art will appreciate that the configurations shown in fig. 9 and 10 are block diagrams of only some of the configurations relevant to the present disclosure, and do not constitute a limitation on the electronic devices to which the present disclosure may be applied, and that a particular electronic device may include more or less components than those shown in the drawings, or may combine certain components, or have a different arrangement of components.

In one embodiment, an electronic device is further provided, which includes a memory and a processor, where the memory stores a computer program, and the processor implements the steps of the above method embodiments when executing the computer program.

In an embodiment, a computer-readable storage medium is provided, on which a computer program is stored which, when being executed by a processor, carries out the steps of the above-mentioned method embodiments.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, database, or other medium used in the embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high-density embedded nonvolatile Memory, resistive Random Access Memory (ReRAM), magnetic Random Access Memory (MRAM), ferroelectric Random Access Memory (FRAM), phase Change Memory (PCM), graphene Memory, and the like. Volatile Memory can include Random Access Memory (RAM), external cache Memory, and the like. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), for example. The databases involved in the embodiments provided herein may include at least one of relational and non-relational databases. The non-relational database may include, but is not limited to, a block chain based distributed database, and the like. The processors referred to in the embodiments provided herein may be general purpose processors, central processing units, graphics processors, digital signal processors, programmable logic devices, quantum computing based data processing logic devices, etc., without limitation.

It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) referred to in the present application are information and data authorized by the user or sufficiently authorized by each party.

All possible combinations of the technical features in the above embodiments may not be described for the sake of brevity, but should be considered as being within the scope of the present disclosure as long as there is no contradiction between the combinations of the technical features.

The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present application. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. A training method of a human body key point prediction model is characterized by comprising the following steps:

2. The method of claim 1, wherein obtaining model losses from the one-dimensional thermodynamic prediction information, the two-dimensional thermodynamic prediction information, the first human keypoint location prediction information, the second human keypoint location prediction information, and the one-dimensional thermodynamic annotation information and the two-dimensional thermodynamic annotation information corresponding to the image sample comprises:

acquiring a first model loss according to the consistency of the one-dimensional thermodynamic diagram prediction information and the one-dimensional thermodynamic diagram marking information;

acquiring a second model loss according to the consistency of the two-dimensional thermodynamic diagram prediction information and the two-dimensional thermodynamic diagram marking information;

acquiring a third model loss according to the consistency of the first human body key point position prediction information and the second human body key point position prediction information;

and obtaining the model loss according to the first model loss, the second model loss and the third model loss.

3. The method of claim 2,

the obtaining a first model loss according to the consistency of the one-dimensional thermodynamic diagram prediction information and the one-dimensional thermodynamic diagram marking information comprises:

acquiring the loss of the first model according to the mean square error of the one-dimensional thermodynamic diagram prediction information and the one-dimensional thermodynamic diagram marking information;

the obtaining of the second model loss according to the consistency of the two-dimensional thermodynamic diagram prediction information and the two-dimensional thermodynamic diagram labeling information comprises:

acquiring the loss of the second model according to the mean square error of the two-dimensional thermodynamic diagram prediction information and the two-dimensional thermodynamic diagram marking information;

the obtaining a third model loss according to the consistency of the first human body key point position prediction information and the second human body key point position prediction information includes:

and acquiring the loss of the third model according to the absolute error of the first human body key point position prediction information and the second human body key point position prediction information.

4. The method of any one of claims 1 to 3, wherein the first trained human keypoint prediction model further comprises a feature extraction network; the step of inputting image samples containing human bodies into the human body key point prediction model to be trained for the second time, and obtaining one-dimensional thermodynamic diagram prediction information output by the one-dimensional thermodynamic diagram prediction branch and two-dimensional thermodynamic diagram prediction information output by the two-dimensional thermodynamic diagram prediction branch comprises the following steps:

inputting the image sample containing the human body into the human body key point prediction model to be trained secondly, obtaining a shared feature map by the feature extraction network in the human body key point prediction model to be trained secondly according to the image sample, respectively transmitting the shared feature map to the one-dimensional thermodynamic prediction branch and the two-dimensional thermodynamic prediction branch, outputting the one-dimensional thermodynamic prediction information by the one-dimensional thermodynamic prediction branch according to the shared feature map, and outputting the two-dimensional thermodynamic prediction information by the two-dimensional thermodynamic prediction branch according to the shared feature map.

5. The method of claim 4, wherein the feature extraction network comprises a first feature extraction network and a second feature extraction network in cascade; the obtaining, by the feature extraction network in the human body key point prediction model to be trained secondly, a shared feature map according to the image sample, and respectively transferring the shared feature map to the one-dimensional thermodynamic diagram prediction branch and the two-dimensional thermodynamic diagram prediction branch includes:

and obtaining an initial shared feature map by the first feature extraction network in the human body key point prediction model to be trained secondly according to the image sample, transmitting the initial shared feature map to the second feature extraction network, obtaining the shared feature map by the second feature extraction network according to the initial shared feature map, and transmitting the shared feature map to the one-dimensional thermodynamic diagram prediction branch and the two-dimensional thermodynamic diagram prediction branch respectively.

6. The method according to any one of claims 1 to 3, wherein the obtaining of the first human body key point position prediction information corresponding to the one-dimensional thermodynamic diagram prediction branch according to the one-dimensional thermodynamic diagram prediction information and the obtaining of the second human body key point position prediction information corresponding to the two-dimensional thermodynamic diagram prediction branch according to the two-dimensional thermodynamic diagram prediction information comprises:

aiming at one-dimensional thermodynamic diagram prediction information corresponding to each human key point, obtaining first human key point position prediction information according to position information enabling the heat data value in the one-dimensional thermodynamic diagram prediction information to be maximum;

and aiming at the two-dimensional thermodynamic diagram prediction information corresponding to each human body key point, obtaining the second human body key point position prediction information according to the position information which enables the heat data value in the two-dimensional thermodynamic diagram prediction information to be maximum.

7. A method for detecting key points of a human body is characterized by comprising the following steps:

acquiring a trained human body key point prediction model containing a one-dimensional thermodynamic diagram prediction branch; the human body key point prediction model is obtained by training according to the training method of the human body key point prediction model of any one of claims 1 to 6;

8. An apparatus for training a human key point prediction model, the apparatus comprising:

the image input module is used for inputting image samples containing human bodies into the human body key point prediction model to be trained secondly, and acquiring one-dimensional thermodynamic diagram prediction information output by the one-dimensional thermodynamic diagram prediction branch and two-dimensional thermodynamic diagram prediction information output by the two-dimensional thermodynamic diagram prediction branch;

9. A human keypoint detection device, characterized in that it comprises:

the model obtaining module is used for obtaining a trained human body key point prediction model containing one-dimensional thermodynamic diagram prediction branches; wherein the human body key point prediction model is obtained by training according to the training device of the human body key point prediction model of claim 8;

and the result acquisition module is used for acquiring a human body key point position detection result of the image to be detected according to the human body key point position prediction information.

10. An electronic device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor, when executing the computer program, implements the steps of the method of any of claims 1 to 6 or of claim 7.

11. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 6 or 7.