CN109858444A - The training method and device of human body critical point detection model - Google Patents

The training method and device of human body critical point detection model Download PDF

Info

Publication number
CN109858444A
CN109858444A CN201910099356.8A CN201910099356A CN109858444A CN 109858444 A CN109858444 A CN 109858444A CN 201910099356 A CN201910099356 A CN 201910099356A CN 109858444 A CN109858444 A CN 109858444A
Authority
CN
China
Prior art keywords
human body
point detection
critical point
network
training
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201910099356.8A
Other languages
Chinese (zh)
Inventor
喻冬东
王长虎
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing ByteDance Network Technology Co Ltd
Original Assignee
Beijing ByteDance Network Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing ByteDance Network Technology Co Ltd filed Critical Beijing ByteDance Network Technology Co Ltd
Priority to CN201910099356.8A priority Critical patent/CN109858444A/en
Publication of CN109858444A publication Critical patent/CN109858444A/en
Pending legal-status Critical Current

Links

Landscapes

  • Image Analysis (AREA)

Abstract

Embodiment of the disclosure discloses the training method and device of human body critical point detection model.One specific embodiment of this method includes: to obtain training sample set;Initial human body critical point detection model is obtained, initial human body critical point detection model includes feature extraction network, at least one critical point detection network corresponding at least one position;Utilize deep learning method, using the sample human body image of the training sample in training sample set as the input of feature extraction network, the characteristic information that feature extraction network exports is separately input into a few critical point detection network, using the location information of the key point at least one position of the human body shown in the sample human body image of input as the desired output of corresponding critical point detection network, training obtains human body critical point detection model.The embodiment realizes the accuracy for improving human body key point prediction.

Description

The training method and device of human body critical point detection model
Technical field
Embodiment of the disclosure is related to field of computer technology, and in particular to the training method of human body critical point detection model And device.
Background technique
With the continuous development of artificial intelligence, computer vision also achieves many achievements.In existing attitude detection etc. In task, it is necessary first to carry out feature extraction to image.Later, characteristic information input prediction network is subjected to the prediction such as posture. For the ease of completing the tasks such as attitude detection, the position of each key point of determining human body is generally required.In the prior art, past Toward the position that human body is determined to its each key point as a whole.
Summary of the invention
Embodiment of the disclosure proposes the training method and device, human body critical point detection of human body critical point detection model Method and apparatus.
In a first aspect, embodiment of the disclosure provides a kind of training method of human body critical point detection model, comprising: obtain Training sample set is taken, training sample includes at least one portion of the human body shown in sample human body image and sample human body image The location information of the key point of position;Initial human body critical point detection model is obtained, initial human body critical point detection model includes spy Sign extracts network, at least one critical point detection network corresponding at least one position;Using deep learning method, will train Input of the sample human body image of training sample in sample set as feature extraction network, feature extraction network is exported Characteristic information is separately input into a few critical point detection network, at least by the human body shown in the sample human body image of input The location information of the key point at one position trains respectively as the desired output of corresponding critical point detection network and obtains human body Critical point detection model.
In some embodiments, using deep learning method, by the sample human body of the training sample in training sample set The characteristic information that feature extraction network exports is separately input into a few key point by input of the image as feature extraction network Network is detected, the location information of the key point at least one position of the human body shown in the sample human body image of input is distinguished As the desired output of corresponding critical point detection network, before training obtains human body critical point detection model, method further include: In at least one position of the human body shown in the sample human body image for including for the training sample in training sample set Position, the location information and targeted parameter value of the key point based on the position obtain the target position letter of the key point at the position Breath.
In some embodiments, using deep learning method, by the sample human body of the training sample in training sample set The characteristic information that feature extraction network exports is separately input into a few key point by input of the image as feature extraction network Network is detected, the location information of the key point at least one position of the human body shown in the sample human body image of input is distinguished As the desired output of corresponding critical point detection network, training obtains human body critical point detection model, comprising: utilizes depth Learning method, using the sample human body image of the training sample in training sample set as the input of feature extraction network, by feature The characteristic information for extracting network output is separately input into a few critical point detection network, will show in the sample human body image of input Phase of the target position information of the key point at least one position of the human body shown respectively as corresponding critical point detection network Output, training is hoped to obtain human body critical point detection model.
In some embodiments, at least one critical point detection network includes at least one of the following: that header key point detects Network, upper limb critical point detection network, lower limb critical point detection network and whole body critical point detection network.
Second aspect, embodiment of the disclosure provide a kind of human body critical point detection method, comprising: obtain people to be detected Body image;By human body image to be detected input human body critical point detection model trained in advance, obtain showing in image to be detected Human body key point information, wherein human body critical point detection model is according to as described in implementation any in first aspect Method training obtains.
The third aspect, embodiment of the disclosure provide a kind of training device of human body critical point detection model, comprising: instruction Practice sample set acquiring unit, is configured to obtain training sample set, training sample includes sample human body image and sample people The location information of the key point at least one position of the human body shown in body image;Initial model acquiring unit, is configured to Initial human body critical point detection model is obtained, initial human body critical point detection model includes feature extraction network and at least one At least one corresponding critical point detection network of position;Training unit is configured to using deep learning method, by training sample Input of the sample human body image of training sample in set as feature extraction network, the feature that feature extraction network is exported Information is separately input into a few critical point detection network, by least one of the human body shown in the sample human body image of input For the location information of the key point at position respectively as the desired output of corresponding critical point detection network, it is crucial that training obtains human body Point detection model.
In some embodiments, the device further include: target position information generation unit, in training sample set Position at least one position of the human body shown in the sample human body image that training sample includes, the key based on the position The location information and targeted parameter value of point, obtain the target position information of the key point at the position.
In some embodiments, training unit is further configured to: deep learning method is utilized, by training sample set In training sample input of the sample human body image as feature extraction network, the characteristic information that feature extraction network is exported It is separately input into a few critical point detection network, by least one position of the human body shown in the sample human body image of input Key point target position information respectively as corresponding critical point detection network desired output, training obtain human body key Point detection model.
In some embodiments, at least one critical point detection network includes at least one of the following: that header key point detects Network, upper limb critical point detection network, lower limb critical point detection network and whole body critical point detection network.
Fourth aspect, embodiment of the disclosure provide a kind of human body critical point detection device, comprising: image to be detected obtains Unit is taken, is configured to obtain human body image to be detected;Critical point detection unit is configured to input human body image to be detected Trained human body critical point detection model in advance, obtains the key point information of human body shown in image to be detected, wherein human body Critical point detection model is obtained according to the method training as described in implementation any in first aspect.
5th aspect, embodiment of the disclosure provide a kind of server, which includes: one or more processing Device;Storage device is stored thereon with one or more programs;When said one or multiple programs are by said one or multiple processing Device executes, so that said one or multiple processors realize the method or such as second as described in implementation any in first aspect The method of any implementation description in aspect.
6th aspect, embodiment of the disclosure provide a kind of computer-readable medium, are stored thereon with computer program, The method as described in implementation any in first aspect is realized when above procedure is executed by processor or as appointed in second aspect The method of one implementation description.
The method and apparatus that embodiment of the disclosure provides, can obtain training sample set, training sample includes first The location information of the key point at least one position of the human body shown in sample human body image and sample human body image.Then, Initial human body critical point detection model is obtained, initial human body critical point detection model includes feature extraction network and at least one At least one corresponding critical point detection network of position.It on this basis, will be in training sample set using deep learning method Training sample input of the sample human body image as feature extraction network, the characteristic information that feature extraction network is exported point At least one critical point detection network is not inputted, by least one position of the human body shown in the sample human body image of input The location information of key point trains respectively as the desired output of corresponding critical point detection network and obtains human body critical point detection Model.Wherein, since human body critical point detection model includes critical point detection network corresponding with the different parts of human body.Therefore The position of the key point of different parts can be predicted respectively.Due to the structure of human body, the proportionate relationship base at especially each position This is consistent, thus the key point at different positions is predicted by different critical point detection networks, and people can be improved The accuracy of body key point prediction.
Detailed description of the invention
By reading a detailed description of non-restrictive embodiments in the light of the attached drawings below, the disclosure is other Feature, objects and advantages will become more apparent upon:
Fig. 1 is that one embodiment of the disclosure can be applied to exemplary system architecture figure therein;
Fig. 2 is the flow chart according to one embodiment of the training method of the human body critical point detection model of the disclosure;
Fig. 3 is an application scenarios of the training method of human body critical point detection model according to an embodiment of the present disclosure Schematic diagram;
Fig. 4 is the flow chart according to another embodiment of the training method of the human body critical point detection model of the disclosure;
Fig. 5 is the structural representation according to one embodiment of the training device of the human body critical point detection model of the disclosure Figure;
Fig. 6 is the flow chart according to one embodiment of the human body critical point detection method of the disclosure;
Fig. 7 is the flow chart according to one embodiment of the human body critical point detection device of the disclosure;
Fig. 8 is adapted for the structural schematic diagram for realizing the electronic equipment of embodiment of the disclosure.
Specific embodiment
The disclosure is described in further detail with reference to the accompanying drawings and examples.It is understood that this place The specific embodiment of description is used only for explaining that correlation is open, rather than the restriction to the disclosure.It also should be noted that being Convenient for description, is illustrated only in attached drawing and to related disclose relevant part.
It should be noted that in the absence of conflict, the feature in embodiment and embodiment in the disclosure can phase Mutually combination.The disclosure is described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.
Fig. 1 is shown can close using the training method or human body of the human body critical point detection model of embodiment of the disclosure The exemplary system architecture 100 of the training device of key point detection model, human body critical point detection method and apparatus.
As shown in Figure 1, system architecture 100 may include terminal device 101,102,103, network 104 and server 105. Network 104 between terminal device 101,102,103 and server 105 to provide the medium of communication link.Network 104 can be with Including various connection types, such as wired, wireless communication link or fiber optic cables etc..
User can be used terminal device 101,102,103 and be interacted by network 104 with server 105, to receive or send out Send message etc..Various client applications, such as the application of picture processing class, figure can be installed on terminal device 101,102,103 Piece shoots class application etc..
Terminal device 101,102,103 can be hardware, be also possible to software.When terminal device 101,102,103 is hard When part, it can be the various electronic equipments for supporting image storage or transmission.It, can when terminal device 101,102,103 is software To be mounted in above-mentioned electronic equipment.Multiple softwares or software module may be implemented into (such as providing distributed clothes in it Business), single software or software module also may be implemented into.It is not specifically limited herein.
Server 105 can be to provide the server of various services, such as to the application on terminal device 101,102,103 The background server supported is provided.The training sample set that background server can be uploaded based on terminal device, training obtain people Body critical point detection model, or the human body image to be detected uploaded for terminal device detect, and obtain showing in image The key point information of the human body shown.
It should be noted that the training method of human body critical point detection model, human body provided by embodiment of the disclosure Critical point detection method is generally executed by server 105, and correspondingly, training device, the human body of human body critical point detection model close Key point detection device is generally positioned in server 105.
It should be noted that server can be hardware, it is also possible to software.When server is hardware, may be implemented At the distributed server cluster that multiple servers form, individual server also may be implemented into.It, can when server is software To be implemented as multiple softwares or software module (such as providing Distributed Services), single software or software also may be implemented into Module.It is not specifically limited herein.
It should be understood that the number of terminal device, network and server in Fig. 1 is only schematical.According to realization need It wants, can have any number of terminal device, network and server.
With continued reference to Fig. 2, an implementation of the training method of the human body critical point detection model according to the disclosure is shown The process 200 of example.The training method of human body critical point detection model, comprising the following steps:
Step 201, training sample set is obtained.
In the present embodiment, the executing subject of the training method of human body critical point detection model can be from local or communication link The electronic equipment connect obtains training sample set.Wherein, training sample includes showing in sample human body image and sample human body image The location information of the key point at least one position of the human body shown.
In practice, training sample set can also obtain in several ways.For example, can by it is existing be used for training Image set is determined as training sample set.For another example, a large amount of human body images can be obtained by shooting or from internet.Later, It is labeled by the position of the key point at multiple positions of the human body shown in the human body image for getting, obtains its position Confidence breath.So as to the key point at multiple positions of the human body that will be shown in a human body image and this human body image Location information is determined as a training sample.A large amount of training sample forms training sample set.
In the present embodiment, in general, the position of human body can include but is not limited to: head, upper limb, lower limb, whole body Etc..It is appreciated that these positions can also be refined or be summarized according to different classifications granularity.For example, upper limb can be with It is refined as hand, arm.Upper limb and lower limb may be summarized to be trunk.In practice, according to different scene demands, choose different Granularity of classification.The key point at each position can be different.For example, for head, it can be by left ear, auris dextra, left eye, right eye, mouth Position where bar equal is determined as the position of key point.For upper limb, the position where wrist, elbow etc. can be determined as closing The position of key point.Difference according to actual needs can flexibly determine quantity and the position of the key point at each position.It is right This, the application is without limitation.
Step 202, initial human body critical point detection model is obtained, initial human body critical point detection model includes feature extraction Network, at least one critical point detection network corresponding at least one position.
In the present embodiment, the available initial human body critical point detection model of above-mentioned executing subject.Wherein, initial human body Critical point detection model may include feature extraction network.Feature extraction network can be existing for extracting the image of feature Multiple network.For example, feature extraction network can be residual error network (ResNet), convolutional neural networks (CNN) etc..This Outside, initial human body critical point detection model can also include at least one critical point detection net corresponding at least one position Network.In practice, at least one position in critical point detection model and step 201 is corresponded.
In the present embodiment, critical point detection network can be existing various critical point detection networks.For example, for example each Kind Recurrent networks, sorter network etc..Specifically, can be multilayer convolutional neural networks, Recognition with Recurrent Neural Network etc..
In some optional implementations of the present embodiment, at least one critical point detection network includes following at least one : header key point detects network, upper limb critical point detection network, lower limb critical point detection network and whole body critical point detection net Network.
Step 203, using the sample human body image of the training sample in training sample set as the defeated of feature extraction network Enter, the characteristic information that feature extraction network exports is separately input into a few critical point detection network, by the sample people of input The location information of the key point at least one position of the human body shown in body image is respectively as corresponding critical point detection net The desired output of network, training obtain human body critical point detection model.
In the present embodiment, above-mentioned executing subject can be using the sample human body image in training sample as character network Input.And the characteristic information that character network exports is separately input into a few critical point detection network.On this basis, will The location information of the key point of at least one portion of the human body shown in the sample human body image of input is respectively as corresponding The desired output of critical point detection network, training obtain human body critical point detection model.
As an example, the head of human body, upper limb, lower limb can be shown in the sample human body image of training sample.Therewith Corresponding, initial human body critical point detection model includes for detecting the first critical point detection network of head key point, being used for Detect the second critical point detection network of upper limb key point and the third critical point detection network for detecting lower limb key point.? It, can be using the location information of the key point on the head of the human body shown in sample human body image as the first key point on the basis of this Detect the desired output of network.Similar, using the location information of the key point of the upper limb shown in sample human body image as The desired output of two critical point detection networks.Using the crucial location information of the lower limb shown in sample human body image as third The desired output of critical point detection network.
On this basis, it can be utilized respectively deep learning algorithm, three critical point detection networks are trained.With For one critical point detection network, the location information and sample human body of the key point of output can be calculated with preset loss function Difference between the location information of the key point on the head of the human body shown in image.It is then possible to based on resulting difference is calculated It is different, the network parameter of the first critical point detection network is adjusted, and in the case where meeting preset trained termination condition, terminates instruction Practice.The training termination condition here preset at can include but is not limited to: the training time is more than preset duration;Frequency of training is more than pre- If number;It calculates resulting difference and is less than default discrepancy threshold.
With continued reference to the application scenarios that Fig. 3, Fig. 3 are according to the training method of the human body critical point detection model of the present embodiment A schematic diagram.In the application scenarios of Fig. 3, the executing subject of the training method of human body critical point detection model can be clothes Business device.The available training sample set 301 of server.By taking a training sample in training sample set 301 as an example, training Sample includes the position of the key point at least one position of the human body shown in sample human body image 302 and sample human body image Information 303.Later, the available initial human body critical point detection model 304 of server.Wherein, initial human body critical point detection Model 304 includes residual error network 3041, header key point detection network 3042, upper limb critical point detection network 3043, lower limb pass Key point detects network 3044, whole body critical point detection network 3045.Wherein, each critical point detection network and a position pair It answers, the location information of the key point for exporting the position.Under this application scene, using residual error network as feature extraction net Network.
On this basis, by taking above-mentioned training sample as an example, sample human body image 302 can be inputted into residual error network 3041. Later, the output of residual error network 3041 is exported into four critical point detection network 3042-3045 respectively.By sample human body image The location information of the key point at four positions (head, upper limb, lower limb and whole body) of the human body shown in 302 is respectively as correspondence Critical point detection network desired output, training obtain human body critical point detection model.Network is detected with header key point For 3042, the pass that header key point detection network 3042 exports can will be calculated with preset loss function (such as mean square deviation) Difference between the location information of the key point on the head of the human body shown in the location information and sample human body image 302 of key point It is different.It is then possible to based on resulting difference, the network parameter of adjustment header key point detection network 3042 is calculated, and meeting In the case where preset trained termination condition, terminate training.Finally, the initial human body critical point detection mould that training can be completed Type 304 is determined as human body critical point detection model.
The method that embodiment of the disclosure provides, can obtain training sample set, training sample includes sample people first The location information of the key point at least one position of the human body shown in body image and sample human body image.Then, it obtains just Beginning human body critical point detection model, initial human body critical point detection model include feature extraction network and at least one position pair At least one the critical point detection network answered.On this basis, using deep learning method, by the training in training sample set Input of the sample human body image of sample as feature extraction network, the characteristic information that feature extraction network exports is inputted respectively At least one critical point detection network, by the key point at least one position of the human body shown in the sample human body image of input Location information respectively as corresponding critical point detection network desired output, training obtain human body critical point detection model. Wherein, since human body critical point detection model includes critical point detection network corresponding with the different parts of human body.It therefore can be with The position of the key point of different parts is predicted respectively.Due to the structure of human body, the proportionate relationship basic one at especially each position It causes, thus the key point at different positions is predicted by different critical point detection networks, human body pass can be improved The accuracy of key point prediction.
With further reference to Fig. 4, it illustrates the processes of another embodiment of the training method of human body critical point detection model 400.The training method 400 of human body critical point detection model, comprising the following steps:
Step 401, training sample set is obtained.
Step 402, initial human body critical point detection model is obtained, initial human body critical point detection model includes feature extraction Network, at least one critical point detection network corresponding at least one position.
In the present embodiment, the specific implementation of step 401-402 and its brought technical effect can be corresponding with reference to Fig. 2 Embodiment step 201-202, details are not described herein.
Step 403, the human body shown in the sample human body image for including for the training sample in training sample set Position at least one position, the location information and targeted parameter value of the key point based on the position, obtains the pass at the position The target position information of key point.
In practice, since the key point at least one position to the human body shown in sample human body image is labeled When, often only mark the location information of a point.For example, technical staff is labeled in left ear this key point to head When, a coordinate value can be marked.So this coordinate value location information for can be regarded as this key point.But in training When, using single coordinate value as when the desired output of critical point detection network, it will increase the training difficulty of model.It is right based on this Each training sample in training sample set, above-mentioned executing subject can be based on the human bodies shown in human body image at least The location information (for example, a coordinate value) at one position, using preset function (such as Gaussian function), obtains human body image Middle display human body at least one position target position information (probability distribution value).To which a coordinate value is converted to One response range.The training difficulty of model can be reduced in this way.
In the present embodiment, due to observing that the size of the different parts of human body is different, and the size at each position It is all relatively-stationary.Based on this, for each position at least one position of the human body shown in sample human body image, Above-mentioned executing subject can be based on different parameter values, by the location information of the key point at each position (for example, a coordinate Value) be converted to a response range.Wherein, the corresponding parameter value in each position is related to the size at the position.
Step 404, using deep learning method, using the sample human body image of the training sample in training sample set as The characteristic information that feature extraction network exports is separately input into a few critical point detection net by the input of feature extraction network Network makees the target position information of the key point at least one position of the human body shown in the sample human body image of input respectively For the desired output of corresponding critical point detection network, training obtains human body critical point detection model.
In the present embodiment, above-mentioned executing subject can be on the basis of step 403, using target position information as key The desired output of point detection network, training obtain human body critical point detection model.In addition, the tool of the other technical characteristics of this step Body is realized and its brought technical effect can be with reference to the step 203 in the corresponding embodiment of Fig. 2, and details are not described herein.
Figure 4, it is seen that embodiment adds the passes based on training sample compared with the corresponding embodiment of Fig. 2 The location information of key point, the step of obtaining the target position information of key point.To which location information be converted by a coordinate value For a response range.On this basis, initial human body critical point detection model is trained using target position information, is dropped Low trained difficulty.In addition, different positions can use different parameters value during obtaining target position information.Ginseng The setting of numerical value keeps parameter setting more accurate with reference to the size of partes corporis humani position.Further, human body key can be improved Accuracy when point detection model detection key point.
With further reference to Fig. 5, as the realization to method shown in above-mentioned each figure, present disclose provides a kind of human body key points The training device of detection model, the device is corresponding with embodiment of the method shown in Fig. 2, which specifically can be applied to various In electronic equipment.
As shown in figure 5, the training device 500 of the human body critical point detection model of the offer of the present embodiment includes: trained sample This set acquiring unit 501, initial model acquiring unit 502 and training unit 503.Wherein, training sample set acquiring unit 501 are configured to obtain training sample set, and training sample includes the people shown in sample human body image and sample human body image The location information of the key point at least one position of body.Initial model acquiring unit 502 is configured to obtain initial human body and closes Key point detection model, initial human body critical point detection model include feature extraction network, it is corresponding at least one position at least One critical point detection network.Training unit 503 is configured to using deep learning method, by the training in training sample set Input of the sample human body image of sample as feature extraction network, the characteristic information that feature extraction network exports is inputted respectively At least one critical point detection network, by the key point at least one position of the human body shown in the sample human body image of input Location information respectively as corresponding critical point detection network desired output, training obtain human body critical point detection model.
In the present embodiment, the training sample set acquiring unit in the training device 500 of human body critical point detection model 501, initial model acquiring unit 502 and the specific implementation of training unit 503 and brought technical effect can be corresponding with reference to Fig. 2 Embodiment step 201-203, details are not described herein.
In some optional implementations of the present embodiment, the device 500 further include: target position information generates single Member, at least one position of the human body shown in the sample human body image for including for the training sample in training sample set Position, the location information and targeted parameter value of the key point based on the position obtain the target position of the key point at the position Information.
In some optional implementations of the present embodiment, training unit 503 is further configured to: utilizing depth Learning method, using the sample human body image of the training sample in training sample set as the input of feature extraction network, by feature The characteristic information for extracting network output is separately input into a few critical point detection network, will show in the sample human body image of input Phase of the target position information of the key point at least one position of the human body shown respectively as corresponding critical point detection network Output, training is hoped to obtain human body critical point detection model.
In some optional implementations of the present embodiment, at least one critical point detection network includes following at least one : header key point detects network, upper limb critical point detection network, lower limb critical point detection network and whole body critical point detection net Network.
In the present embodiment, training sample set, training can be obtained by training sample set acquiring unit 501 first Sample includes the position letter of the key point at least one position of the human body shown in sample human body image and sample human body image Breath.Then, initial human body critical point detection model, initial human body critical point detection are obtained by initial model acquiring unit 502 Model includes feature extraction network, at least one critical point detection network corresponding at least one position.On this basis, lead to Training unit 503 is crossed, using deep learning method, using the sample human body image of the training sample in training sample set as spy Sign extracts the input of network, and the characteristic information that feature extraction network exports is separately input into a few critical point detection network, Using the location information of the key point at least one position of the human body shown in the sample human body image of input as correspondence Critical point detection network desired output, training obtain human body critical point detection model.Wherein, due to human body critical point detection Model includes critical point detection network corresponding with the different parts of human body.Therefore the key point of different parts can be predicted respectively Position.Due to the structure of human body, the proportionate relationship at especially each position is almost the same, thus for the pass at different positions Key point is predicted by different critical point detection networks, and the accuracy of human body key point prediction can be improved.
The process of one embodiment of the human body critical point detection method according to the disclosure is shown with continued reference to Fig. 6, Fig. 6 600.Human body critical point detection method, comprising:
Step 601, human body image to be detected is obtained.
Step 602, the human body critical point detection model that human body image to be detected input is trained in advance, obtains mapping to be checked The key point information of human body shown in as in.
In the present embodiment, human body critical point detection model is according to the method described in the embodiment as corresponding to Fig. 2 or Fig. 4 Training obtains.
The method provided by the above embodiment of the disclosure, by closing human body image to be detected input human body trained in advance Key point detection model.Obtain the key point information of human body shown in image to be detected.Due to human body critical point detection model packet Include critical point detection network corresponding with the different parts of human body.Therefore the position of the key point of different parts can be predicted respectively It sets.Due to the structure of human body, the proportionate relationship at especially each position is almost the same, thus for the key point at different positions It is predicted by different critical point detection networks, the accuracy of human body key point prediction can be improved.
With further reference to Fig. 7, present disclose provides a kind of one embodiment of human body critical point detection device, the device is real It is corresponding with embodiment of the method shown in fig. 6 to apply example, which specifically can be applied in various electronic equipments.
As shown in fig. 7, the human body critical point detection device 700 of the present embodiment includes: 701 He of image to be detected acquiring unit Critical point detection unit 702.Wherein, image to be detected acquiring unit 701 is configured to obtain human body image to be detected.Key point Detection unit 702 is configured to inputting human body image to be detected into human body critical point detection model trained in advance, obtains to be checked The key point information of the human body shown in altimetric image, wherein human body critical point detection model is according to as corresponding to Fig. 2 or Fig. 4 Embodiment description method training obtain.
The device provided by the above embodiment of the disclosure obtains human body to be detected by image to be detected acquiring unit 701 Image.Later, human body image to be detected is inputted by critical point detection unit 702 by human body critical point detection mould trained in advance Type obtains the key point information of human body shown in image to be detected.Since human body critical point detection model includes and human body The corresponding critical point detection network of different parts.Therefore the position of the key point of different parts can be predicted respectively.Due to human body Structure, the proportionate relationship at especially each position is almost the same, thus for the key point at different positions by different Critical point detection network is predicted, the accuracy of human body key point prediction can be improved.
Below with reference to Fig. 8, it illustrates the electronic equipment that is suitable for being used to realize embodiment of the disclosure, (example is as shown in figure 1 Server) 800 structural schematic diagram.Electronic equipment shown in Fig. 8 is only an example, should not be to embodiment of the disclosure Function and use scope bring any restrictions.
As shown in figure 8, electronic equipment 800 includes processing unit (such as central processing unit, graphics processor etc.) 801, Random access storage device can be loaded into according to the program being stored in read-only memory (ROM) 802 or from storage device 808 (RAM) program in 803 and execute various movements appropriate and processing.In RAM 803, it is also stored with the behaviour of electronic equipment 800 Various programs and data needed for making.Processing unit 801, ROM 802 and RAM803 are connected with each other by bus 804.Input/ Output (I/O) interface 805 is also connected to bus 804.
In general, following device can connect to I/O interface 805: including such as touch screen, touch tablet, keyboard, mouse, taking the photograph As the input unit 806 of head, microphone, accelerometer, gyroscope etc.;Including such as liquid crystal display (LCD), loudspeaker, vibration The output device 807 of dynamic device etc.;Storage device 808 including such as tape, hard disk etc.;And communication device 809.Communication device 809, which can permit electronic equipment 800, is wirelessly or non-wirelessly communicated with other equipment to exchange data.Although Fig. 8 shows tool There is the electronic equipment 800 of various devices, it should be understood that being not required for implementing or having all devices shown.It can be with Alternatively implement or have more or fewer devices.Each box shown in Fig. 8 can represent a device, can also root According to needing to represent multiple devices.
Particularly, in accordance with an embodiment of the present disclosure, it may be implemented as computer above with reference to the process of flow chart description Software program.For example, embodiment of the disclosure includes a kind of computer program product comprising be carried on computer-readable medium On computer program, which includes the program code for method shown in execution flow chart.In such reality It applies in example, which can be downloaded and installed from network by communication device 809, or from storage device 808 It is mounted, or is mounted from ROM802.When the computer program is executed by middle processing unit 801, the reality of the disclosure is executed Apply the above-mentioned function of limiting in the method for example.
It is situated between it should be noted that computer-readable medium described in embodiment of the disclosure can be computer-readable signal Matter or computer readable storage medium either the two any combination.Computer readable storage medium for example can be with System, device or the device of --- but being not limited to --- electricity, magnetic, optical, electromagnetic, infrared ray or semiconductor, or it is any more than Combination.The more specific example of computer readable storage medium can include but is not limited to: have one or more conducting wires Electrical connection, portable computer diskette, hard disk, random access storage device (RAM), read-only memory (ROM), erasable type are programmable Read-only memory (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), light storage device, magnetic are deposited Memory device or above-mentioned any appropriate combination.In embodiment of the disclosure, computer readable storage medium, which can be, appoints What include or the tangible medium of storage program that the program can be commanded execution system, device or device use or and its It is used in combination.And in embodiment of the disclosure, computer-readable signal media may include in a base band or as carrier wave The data-signal that a part is propagated, wherein carrying computer-readable program code.The data-signal of this propagation can be adopted With diversified forms, including but not limited to electromagnetic signal, optical signal or above-mentioned any appropriate combination.Computer-readable signal is situated between Matter can also be that any computer-readable medium other than computer readable storage medium, the computer-readable medium can be sent out It send, propagate or transmits for by the use of instruction execution system, device or device or program in connection.It calculates The program code for including on machine readable medium can transmit with any suitable medium, including but not limited to: electric wire, optical cable, RF Etc. or above-mentioned any appropriate combination.
Above-mentioned computer-readable medium can be included in above-mentioned electronic equipment;It is also possible to individualism, and not It is fitted into the electronic equipment.Above-mentioned computer-readable medium carries one or more program, when said one or more When a program is executed by the electronic equipment, so that the electronic equipment: obtaining training sample set, training sample includes sample human body The location information of the key point at least one position of the human body shown in image and sample human body image;Initial human body is obtained to close Key point detection model, initial human body critical point detection model include feature extraction network, it is corresponding at least one position at least One critical point detection network;Using deep learning method, by the sample human body image of the training sample in training sample set As the input of feature extraction network, the characteristic information that feature extraction network exports is separately input into a few critical point detection Network, using the location information of the key point at least one position of the human body shown in the sample human body image of input as The desired output of corresponding critical point detection network, training obtain human body critical point detection model;Or obtain human body to be detected Image;By human body image to be detected input human body critical point detection model trained in advance, obtain showing in image to be detected The key point information of human body.
The calculating of the operation for executing the disclosure can be write with one or more programming languages or combinations thereof Machine program code, described program design language include object oriented program language-such as Java, Smalltalk, C+ +, it further include conventional procedural programming language-such as " C " language or similar programming language.Program code can Fully to execute, partly execute on the user computer on the user computer, be executed as an independent software package, Part executes on the remote computer or executes on a remote computer or server completely on the user computer for part. In situations involving remote computers, remote computer can pass through the network of any kind --- including local area network (LAN) Or wide area network (WAN)-is connected to subscriber computer, or, it may be connected to outer computer (such as utilize Internet service Provider is connected by internet).
Flow chart and block diagram in attached drawing illustrate system, method and the computer of the various embodiments according to the disclosure The architecture, function and operation in the cards of program product.In this regard, each box in flowchart or block diagram can be with A part of a module, program segment or code is represented, a part of the module, program segment or code includes one or more Executable instruction for implementing the specified logical function.It should also be noted that in some implementations as replacements, institute in box The function of mark can also occur in a different order than that indicated in the drawings.For example, two boxes succeedingly indicated are practical On can be basically executed in parallel, they can also be executed in the opposite order sometimes, and this depends on the function involved.Also it wants It is noted that the combination of each box in block diagram and or flow chart and the box in block diagram and or flow chart, Ke Yiyong The dedicated hardware based system of defined functions or operations is executed to realize, or can be referred to specialized hardware and computer The combination of order is realized.
Being described in unit involved in embodiment of the disclosure can be realized by way of software, can also be passed through The mode of hardware is realized.Described unit also can be set in the processor, for example, can be described as: a kind of processor Including training sample set acquiring unit, initial model acquiring unit and training unit.Wherein, the title of these units is at certain In the case of do not constitute restriction to the unit itself, for example, training sample set acquiring unit is also described as " obtaining The unit of training sample set ".
Above description is only the preferred embodiment of the disclosure and the explanation to institute's application technology principle.Those skilled in the art Member is it should be appreciated that invention scope involved in the disclosure, however it is not limited to technology made of the specific combination of above-mentioned technical characteristic Scheme, while should also cover in the case where not departing from foregoing invention design, it is carried out by above-mentioned technical characteristic or its equivalent feature Any combination and the other technical solutions formed.Such as features described above has similar function with (but being not limited to) disclosed in the disclosure Can technical characteristic replaced mutually and the technical solution that is formed.

Claims (12)

1. a kind of training method of human body critical point detection model, comprising:
Obtain training sample set, training sample include the human body that is shown in sample human body image and sample human body image at least The location information of the key point at one position;
Obtain initial human body critical point detection model, the initial human body critical point detection model include feature extraction network, with At least one corresponding critical point detection network of at least one described position;
Using deep learning method, using the sample human body image of the training sample in the training sample set as feature extraction The characteristic information that feature extraction network exports is separately input into a few critical point detection network, will inputted by the input of network Sample human body image in the location information of the key point at least one position of human body that shows respectively as corresponding key The desired output of point detection network, training obtain human body critical point detection model.
2. according to the method described in claim 1, wherein, deep learning method is utilized described, by the training sample set In training sample input of the sample human body image as feature extraction network, the characteristic information that feature extraction network is exported It is separately input into a few critical point detection network, by least one position of the human body shown in the sample human body image of input Key point location information respectively as corresponding critical point detection network desired output, training obtain human body key point inspection It surveys before model, the method also includes:
At least one of the human body shown in the sample human body image for including for the training sample in the training sample set Position in position, the location information and targeted parameter value of the key point based on the position, obtains the mesh of the key point at the position Cursor position information.
3. according to the method described in claim 2, wherein, the utilization deep learning method will be in the training sample set Training sample input of the sample human body image as feature extraction network, the characteristic information that feature extraction network is exported point At least one critical point detection network is not inputted, by least one position of the human body shown in the sample human body image of input The location information of key point trains respectively as the desired output of corresponding critical point detection network and obtains human body critical point detection Model, comprising:
Using deep learning method, using the sample human body image of the training sample in the training sample set as feature extraction The characteristic information that feature extraction network exports is separately input into a few critical point detection network, will inputted by the input of network Sample human body image in the target position information of the key point at least one position of human body that shows respectively as corresponding The desired output of critical point detection network, training obtain human body critical point detection model.
4. method according to claim 1 to 3, wherein at least one described critical point detection network includes following At least one of: header key point detects network, upper limb critical point detection network, lower limb critical point detection network and whole body key point Detect network.
5. a kind of human body critical point detection method, comprising:
Obtain human body image to be detected;
The human body image input to be detected human body critical point detection model trained in advance is obtained in described image to be detected The key point information of the human body of display, wherein the human body critical point detection model is according to side any in such as claim 1-4 Method training obtains.
6. a kind of training device of human body critical point detection model, comprising:
Training sample set acquiring unit, is configured to obtain training sample set, training sample include sample human body image and The location information of the key point at least one position of the human body shown in sample human body image;
Initial model acquiring unit is configured to obtain initial human body critical point detection model, the initial human body key point inspection Surveying model includes feature extraction network, at least one critical point detection network corresponding at least one described position;
Training unit is configured to using deep learning method, by the sample people of the training sample in the training sample set The characteristic information that feature extraction network exports is separately input into a few key by input of the body image as feature extraction network Point detection network, by the location information of the key point at least one position of the human body shown in the sample human body image of input point Desired output not as corresponding critical point detection network, training obtain human body critical point detection model.
7. device according to claim 6, wherein described device further include:
Target position information generation unit, in the sample human body image for including for the training sample in the training sample set Position at least one position of the human body of display, the location information and targeted parameter value of the key point based on the position, obtains To the target position information of the key point at the position.
8. device according to claim 7, wherein the training unit is further configured to:
Using deep learning method, using the sample human body image of the training sample in the training sample set as feature extraction The characteristic information that feature extraction network exports is separately input into a few critical point detection network, will inputted by the input of network Sample human body image in the target position information of the key point at least one position of human body that shows respectively as corresponding The desired output of critical point detection network, training obtain human body critical point detection model.
9. according to the device any in claim 6-8, wherein at least one described critical point detection network includes following At least one of: header key point detects network, upper limb critical point detection network, lower limb critical point detection network and whole body key point Detect network.
10. a kind of human body critical point detection device, comprising:
Image to be detected acquiring unit is configured to obtain human body image to be detected;
Critical point detection unit is configured to the human body critical point detection mould that the human body image input to be detected is trained in advance Type obtains the key point information of the human body shown in described image to be detected, wherein the human body critical point detection model according to Method training as any in claim 1-4 obtains.
11. a kind of server, comprising:
One or more processors;
Storage device is stored thereon with one or more programs,
When one or more of programs are executed by one or more of processors, so that one or more of processors are real The now method as described in any or claim 5 in claim 1-4.
12. a kind of computer-readable medium, is stored thereon with computer program, wherein real when described program is executed by processor The now method as described in any or claim 5 in claim 1-4.
CN201910099356.8A 2019-01-31 2019-01-31 The training method and device of human body critical point detection model Pending CN109858444A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910099356.8A CN109858444A (en) 2019-01-31 2019-01-31 The training method and device of human body critical point detection model

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910099356.8A CN109858444A (en) 2019-01-31 2019-01-31 The training method and device of human body critical point detection model

Publications (1)

Publication Number Publication Date
CN109858444A true CN109858444A (en) 2019-06-07

Family

ID=66897297

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910099356.8A Pending CN109858444A (en) 2019-01-31 2019-01-31 The training method and device of human body critical point detection model

Country Status (1)

Country Link
CN (1) CN109858444A (en)

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110826610A (en) * 2019-10-29 2020-02-21 上海眼控科技股份有限公司 Method and system for intelligently detecting whether dressed clothes of personnel are standard
CN111178298A (en) * 2019-12-31 2020-05-19 北京达佳互联信息技术有限公司 Human body key point detection method and device, electronic equipment and storage medium
CN111523387A (en) * 2020-03-24 2020-08-11 杭州易现先进科技有限公司 Method and device for detecting hand key points and computer device
CN111582206A (en) * 2020-05-13 2020-08-25 北京字节跳动网络技术有限公司 Method and device for generating organism posture key point information
CN112597837A (en) * 2020-12-11 2021-04-02 北京百度网讯科技有限公司 Image detection method, apparatus, device, storage medium and computer program product
CN113034580A (en) * 2021-03-05 2021-06-25 北京字跳网络技术有限公司 Image information detection method and device and electronic equipment
CN113762026A (en) * 2021-03-04 2021-12-07 北京沃东天骏信息技术有限公司 Combined processing method, equipment and product for human body key point detection and position analysis
CN114224577A (en) * 2022-02-24 2022-03-25 深圳市心流科技有限公司 Training method and device for intelligent artificial limb, electronic equipment, intelligent artificial limb and medium
CN115578451A (en) * 2022-09-30 2023-01-06 北京百度网讯科技有限公司 Image processing method, and training method and device of image processing model

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9355329B2 (en) * 2011-05-09 2016-05-31 Catherine G. McVey Image analysis for determining characteristics of pairs of individuals
CN108229488A (en) * 2016-12-27 2018-06-29 北京市商汤科技开发有限公司 For the method, apparatus and electronic equipment of detection object key point
CN108460362A (en) * 2018-03-23 2018-08-28 成都品果科技有限公司 A kind of system and method for detection human body
CN108549876A (en) * 2018-04-20 2018-09-18 重庆邮电大学 The sitting posture detecting method estimated based on target detection and human body attitude
CN108985208A (en) * 2018-07-06 2018-12-11 北京字节跳动网络技术有限公司 The method and apparatus for generating image detection model
CN108985259A (en) * 2018-08-03 2018-12-11 百度在线网络技术(北京)有限公司 Human motion recognition method and device
CN109145783A (en) * 2018-08-03 2019-01-04 北京字节跳动网络技术有限公司 Method and apparatus for generating information
CN109165552A (en) * 2018-07-14 2019-01-08 深圳神目信息技术有限公司 A kind of gesture recognition method based on human body key point, system and memory

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9355329B2 (en) * 2011-05-09 2016-05-31 Catherine G. McVey Image analysis for determining characteristics of pairs of individuals
CN108229488A (en) * 2016-12-27 2018-06-29 北京市商汤科技开发有限公司 For the method, apparatus and electronic equipment of detection object key point
CN108460362A (en) * 2018-03-23 2018-08-28 成都品果科技有限公司 A kind of system and method for detection human body
CN108549876A (en) * 2018-04-20 2018-09-18 重庆邮电大学 The sitting posture detecting method estimated based on target detection and human body attitude
CN108985208A (en) * 2018-07-06 2018-12-11 北京字节跳动网络技术有限公司 The method and apparatus for generating image detection model
CN109165552A (en) * 2018-07-14 2019-01-08 深圳神目信息技术有限公司 A kind of gesture recognition method based on human body key point, system and memory
CN108985259A (en) * 2018-08-03 2018-12-11 百度在线网络技术(北京)有限公司 Human motion recognition method and device
CN109145783A (en) * 2018-08-03 2019-01-04 北京字节跳动网络技术有限公司 Method and apparatus for generating information

Cited By (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110826610A (en) * 2019-10-29 2020-02-21 上海眼控科技股份有限公司 Method and system for intelligently detecting whether dressed clothes of personnel are standard
CN111178298A (en) * 2019-12-31 2020-05-19 北京达佳互联信息技术有限公司 Human body key point detection method and device, electronic equipment and storage medium
CN111523387A (en) * 2020-03-24 2020-08-11 杭州易现先进科技有限公司 Method and device for detecting hand key points and computer device
CN111523387B (en) * 2020-03-24 2024-04-19 杭州易现先进科技有限公司 Method and device for detecting key points of hands and computer device
CN111582206B (en) * 2020-05-13 2023-08-22 抖音视界有限公司 Method and device for generating organism posture key point information
CN111582206A (en) * 2020-05-13 2020-08-25 北京字节跳动网络技术有限公司 Method and device for generating organism posture key point information
US11810319B2 (en) 2020-12-11 2023-11-07 Beijing Baidu Netcom Science & Technology Co., Ltd Image detection method, device, storage medium and computer program product
EP3852008A3 (en) * 2020-12-11 2021-12-29 Beijing Baidu Netcom Science and Technology Co., Ltd. Image detection method and apparatus, device, storage medium and computer program product
CN112597837A (en) * 2020-12-11 2021-04-02 北京百度网讯科技有限公司 Image detection method, apparatus, device, storage medium and computer program product
CN112597837B (en) * 2020-12-11 2024-05-28 北京百度网讯科技有限公司 Image detection method, apparatus, device, storage medium, and computer program product
CN113762026A (en) * 2021-03-04 2021-12-07 北京沃东天骏信息技术有限公司 Combined processing method, equipment and product for human body key point detection and position analysis
CN113034580B (en) * 2021-03-05 2023-01-17 北京字跳网络技术有限公司 Image information detection method and device and electronic equipment
CN113034580A (en) * 2021-03-05 2021-06-25 北京字跳网络技术有限公司 Image information detection method and device and electronic equipment
CN114224577A (en) * 2022-02-24 2022-03-25 深圳市心流科技有限公司 Training method and device for intelligent artificial limb, electronic equipment, intelligent artificial limb and medium
CN115578451A (en) * 2022-09-30 2023-01-06 北京百度网讯科技有限公司 Image processing method, and training method and device of image processing model
CN115578451B (en) * 2022-09-30 2024-01-23 北京百度网讯科技有限公司 Image processing method, training method and device of image processing model

Similar Documents

Publication Publication Date Title
CN109858444A (en) The training method and device of human body critical point detection model
CN109145781A (en) Method and apparatus for handling image
CN109902659B (en) Method and apparatus for processing human body image
CN108985259A (en) Human motion recognition method and device
CN110288049A (en) Method and apparatus for generating image recognition model
CN110110811A (en) Method and apparatus for training pattern, the method and apparatus for predictive information
CN109858445A (en) Method and apparatus for generating model
CN108416324A (en) Method and apparatus for detecting live body
CN108985208A (en) The method and apparatus for generating image detection model
CN108776786A (en) Method and apparatus for generating user's truth identification model
CN108898185A (en) Method and apparatus for generating image recognition model
CN108446651A (en) Face identification method and device
CN109829432A (en) Method and apparatus for generating information
CN108492364A (en) The method and apparatus for generating model for generating image
CN108363995A (en) Method and apparatus for generating data
CN110009059A (en) Method and apparatus for generating model
CN108491823A (en) Method and apparatus for generating eye recognition model
CN109410253B (en) For generating method, apparatus, electronic equipment and the computer-readable medium of information
CN108509892A (en) Method and apparatus for generating near-infrared image
CN109101919A (en) Method and apparatus for generating information
US20230206420A1 (en) Method for detecting defect and method for training model
CN109815365A (en) Method and apparatus for handling video
CN109993150A (en) The method and apparatus at age for identification
CN108510454A (en) Method and apparatus for generating depth image
CN109887077A (en) Method and apparatus for generating threedimensional model

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20190607