CN108229418A

CN108229418A - Human body critical point detection method and apparatus, electronic equipment, storage medium and program

Info

Publication number: CN108229418A
Application number: CN201810055582.1A
Authority: CN
Inventors: 刘文韬; 钱晨
Original assignee: Beijing Sensetime Technology Development Co Ltd
Current assignee: Beijing Sensetime Technology Development Co Ltd
Priority date: 2018-01-19
Filing date: 2018-01-19
Publication date: 2018-06-29
Anticipated expiration: 2038-01-19
Also published as: CN108229418B

Abstract

The embodiment of the invention discloses a kind of human body critical point detection method and apparatus, electronic equipment, storage medium and program, wherein, method includes：Face datection is carried out to image, obtains the location information of face in described image；According to the location information of the face, the location information of the corresponding human body central point of the face is determined；According to the location information of the human body central point, human body critical point detection is carried out to described image.The human body key point position accuracy that the embodiment of the present invention detects is higher.

Description

Human body critical point detection method and apparatus, electronic equipment, storage medium and program

Technical field

The present invention relates to artificial intelligence technology, especially a kind of human body critical point detection method and apparatus, are deposited electronic equipment Storage media and program.

Background technology

Human body critical point detection technology be human body video data automatically process, the base of human body behavioural analysis and human-computer interaction Plinth can provide important technical support for video structural.

A kind of relevant human body critical point detection technology is based primarily upon depth transducer realization, is obtained with depth transducer Depth image is the position of input detection human body key point.

Another relevant human body critical point detection technology is the human body critical point detection based on yellowish green blue (RGB) camera System mainly includes human body positioning and human body key point location two parts.The human body critical point detection system is imaged with common RGB The image of head acquisition is input, obtains the external frame of human body and simultaneously connects frame progress human body critical point detection in vitro based on the people.

Invention content

The embodiment of the present invention provides a kind of technical solution for human body critical point detection.

One side according to embodiments of the present invention, a kind of human body critical point detection method provided, including：

Face datection is carried out to image, obtains the location information of face in described image；

According to the location information of the face, the location information of the corresponding human body central point of the face is determined；

According to the location information of the human body central point, human body critical point detection is carried out to described image.

Optionally, in the above-mentioned each method embodiment of the present invention, the location information according to the face determines described The location information of the corresponding human body central point of face, including：

According to the location information of the face, the first image block with the first pre-set dimension is obtained from described image, Described first image block includes at least part of the corresponding human body of the face；

According to described first image block and head body mapping network, the position letter of the corresponding human body central point of the face is determined Breath.

Optionally, in the above-mentioned each method embodiment of the present invention, the location information of the face is included in the face Heart position and the dimension information of the face；

The location information according to the face obtains the first image with the first pre-set dimension from described image Block, including：

According to the dimension information of the face, the normalized parameter of described image is determined；

According to the normalized parameter of described image, size normalized is carried out to described image, obtains size normalization Image afterwards；

According to the center of the face, intercepted in the image after size normalization described default with first First image block of size.

Optionally, in the above-mentioned each method embodiment of the present invention, the dimension information according to the face determines described The normalized parameter of image, including：

According to the dimension information of the face, determine the size scaling of the face to preset normalization facial size Corresponding normalized parameter.

Optionally, in the above-mentioned each method embodiment of the present invention, the center according to the face, in the ruler First image block with the first pre-set dimension is intercepted in described image after very little normalization, including：

It is intercepted from the image after size normalization centered on the center of the face, size is returned to be described One changes M times of image block of facial size, and the described image block of interception zoomed to the described first default size, described in acquisition First image block, wherein, the value of M is more than 3 and less than 20.

Optionally, in the above-mentioned each method embodiment of the present invention, the location information according to the human body central point is right Described image carries out human body critical point detection, including：

According to the location information of the human body central point, the second figure with the second pre-set dimension is obtained from described image As block, second image block includes at least part of the corresponding human body of the face；

By human body critical point detection network, human body critical point detection is carried out in second image block.

Optionally, in the above-mentioned each method embodiment of the present invention, the location information according to the human body central point, from The second image block with the second pre-set dimension is obtained in described image, including：

From the described image after size normalization interception centered on the position of the human body central point, described have the Second image block of two pre-set dimensions.

Optionally, it is described that human body key is carried out in second image block in the above-mentioned each method embodiment of the present invention Point detection, including：

Human body critical point detection is carried out to second image block, obtains the key of human body described in second image block The position of point.

Optionally, it is described that human body key point is carried out to second image block in the above-mentioned each method embodiment of the present invention Detection obtains the position of the key point of human body described in second image block, including：

The human body in the block to second image carries out critical point detection, obtains second image block for described The confidence image of each key point at least one key point of human body, the confidence image of the key point include described the At least one position is judged as the confidence level of the affiliated key point of the confidence image in two image blocks；

The location determination of confidence level maximum in the confidence image of the key point is crucial belonging to the confidence image Point is in the second image position in the block.

Optionally, in the above-mentioned each method embodiment of the present invention, human body described in second image block is obtained described Key point position after, the method further includes：

The position of the key point of human body according to second image block and second image block are in described image In position, determine position of the key point of the human body in described image.

Optionally, in the above-mentioned each method embodiment of the present invention, the human body according to second image block The position of the position of key point and second image block in described image, determines the key point of the human body in described image In position, including：

The position of the key point of human body according to second image block and second image block are in size normalizing The position in described image after change, the human body described in the position acquisition of the key point of human body as described in second image block Position in image of the key point after size normalization；

The normalizing of position and described image in image of the key point based on the human body after size normalization Change parameter, determine position of the key point of the human body in described image.

Optionally, in the above-mentioned each method embodiment of the present invention, the human body critical point detection network includes multiple convolution Neural network；

The human body in the block to second image carries out critical point detection, obtains second image block and is directed to The confidence image of each key point at least one key point of the human body, including：

By each convolutional neural networks in the multiple convolutional neural networks, the image for extracting second image block is special Sign, wherein, the characteristics of image of the different convolutional neural networks extraction in the multiple convolutional neural networks has different scales；

The characteristics of image for the different scale that the multiple convolutional neural networks are extracted splices, and obtains splicing feature；

Using the corresponding grader of key point each at least one key point of the human body to it is described splicing feature into Row classification obtains confidence level figure of second image block for key point each at least one key point of the human body Picture.

Optionally, in the above-mentioned each method embodiment of the present invention, the first convolution god in the multiple convolutional neural networks Include multiple convolutional layers for being located at heterogeneous networks depth respectively through network；

It is described to pass through each convolutional neural networks in the multiple convolutional neural networks, the figure of extraction second image block Picture feature, including：

Fisrt feature that the first convolutional layer that network depth is i is exported and the second convolutional layer that network depth is j export Second feature carry out Fusion Features, obtain fusion feature, wherein, the second feature be by the fisrt feature pass through At least one convolutional layer carries out what feature extraction obtained successively, second image block of the first convolutional neural networks output Characteristics of image by being handled to obtain to fusion feature, 1≤i<j.

Optionally, in the above-mentioned each method embodiment of the present invention, the human body critical point detection network includes M of connection Network block, each network block include the multiple convolutional neural networks, the M network pth network block in the block it is defeated Go out the splicing feature that the multiple convolutional neural networks included for the pth network block obtain, and pth network block output Splicing feature is input in+1 network block of pth, wherein, M >=2, p=1 ..., M-1；

In multiple key points using the human body each corresponding grader of key point to it is described splicing feature into Row classification obtains confidence image of second image block for key point each in multiple key points of the human body, packet It includes：

The splicing exported using the corresponding grader of key point each in multiple key points of the human body to M network blocks Feature is classified, and obtains confidence level figure of second image block for key point each in multiple key points of the human body Picture.

Optionally, in the above-mentioned each method embodiment of the present invention, the human body critical point detection network includes multiple difference Positioned at the convolutional layer of heterogeneous networks depth；

Fisrt feature that the first convolutional layer that network depth is i is exported and the second convolutional layer that network depth is j export Second feature carry out Fusion Features, obtain fusion feature, wherein, the second feature be by the fisrt feature pass through At least one convolutional layer carries out what feature extraction obtained successively, and the characteristics of image of second image block is by the fusion What feature was handled, 1≤i<j.

Other side according to embodiments of the present invention, the human body critical point detection device provided, including：

Face detection module for carrying out Face datection to image, obtains the location information of face in described image；

Head body mapping network, for the location information according to the face, determines the corresponding human body central point of the face Location information；

Human body critical point detection network, for the location information according to the human body central point, to described image into pedestrian Body critical point detection.

Optionally, it in the above-mentioned each device embodiment of the present invention, further includes：

First interception module, for the location information according to the face, being obtained from described image has first to preset First image block of size, described first image block include at least part of the corresponding human body of the face；

The head body mapping network, specifically for determining the corresponding human body center of the face according to described first image block The location information of point.

Optionally, in the above-mentioned each device embodiment of the present invention, the location information of the face is included in the face Heart position and the dimension information of the face；

First interception module, is specifically used for：

Optionally, in the above-mentioned each device embodiment of the present invention, first interception module is according to the size of the face Information when determining the normalized parameter of described image, is specifically used for：According to the dimension information of the face, determine the people Normalized parameter corresponding to the size scaling of face to preset normalization facial size.

Optionally, in the above-mentioned each device embodiment of the present invention, the center according to the face, in the ruler When there is the first image block of the first pre-set dimension described in being intercepted in the described image after very little normalization, it is specifically used for：From described It is intercepted in image after size normalization centered on the center of the face, size is the normalization facial size M times of image block, and the described image block of interception is zoomed into the described first default size, described first image block is obtained, In, the value of M is more than 3 and less than 20.

Second interception module, for the location information according to the human body central point, being obtained from described image has the Second image block of two pre-set dimensions, second image block include at least part of the corresponding human body of the face；

The human body critical point detection network, specifically for passing through human body critical point detection network, in second image Human body critical point detection is carried out in block.

Optionally, in the above-mentioned each device embodiment of the present invention, second interception module is specifically used for：From the ruler In described image after very little normalization interception centered on the position of the human body central point, described there is the second pre-set dimension Second image block.

Optionally, in the above-mentioned each device embodiment of the present invention, the human body critical point detection network is in second figure During as carrying out human body critical point detection in block, it is specifically used for：Human body critical point detection is carried out to second image block, obtains institute State the position of the key point of human body described in the second image block.

Optionally, in the above-mentioned each device embodiment of the present invention, the human body critical point detection network is to second figure As block progress human body critical point detection, when obtaining the position of the key point of human body described in second image block, it is specifically used for：

The human body in the block to second image carries out critical point detection, obtains second image block for described The confidence image of each key point at least one key point of human body, the confidence image of the key point include described the At least one position is judged as the confidence level of the affiliated key point of the confidence image in two image blocks；By putting for the key point The location determination of confidence level maximum is the affiliated key point of the confidence image in the second image position in the block in reliability image It puts.

Acquisition module, for the position of the key point of human body according to second image block and second image Position of the block in described image determines position of the key point of the human body in described image.

Optionally, in the above-mentioned each device embodiment of the present invention, the acquisition module is specifically used for：

Optionally, in the above-mentioned each device embodiment of the present invention, the human body critical point detection network includes：

Multiple convolutional networks are respectively used to through convolutional network each in the multiple convolutional network, extraction described second The characteristics of image of image block, wherein, the characteristics of image of the different convolutional networks extraction in the multiple convolutional network has difference Scale；

Concatenation unit, the characteristics of image of the different scale for the multiple convolutional network to be extracted splice, and obtain Splice feature；

The corresponding grader of each key point at least one key point of human body is respectively used to divide the splicing feature Class obtains confidence image of second image block for key point each at least one key point of the human body.

Optionally, in the above-mentioned each device embodiment of the present invention, each network includes multiple respectively positioned at different nets The convolutional layer of network depth；

The first convolutional network in the multiple convolutional network, is specifically used for：Network depth is defeated for the first convolutional layer of i The second feature that the second convolutional layer that the fisrt feature and network depth gone out is j exports carries out Fusion Features, obtains fusion feature, Wherein, the second feature is obtained by carrying out feature extraction successively by least one convolutional layer to the fisrt feature , the characteristics of image of second image block of the first convolutional network output is by being handled to obtain to fusion feature , 1≤i<j.

Optionally, in the above-mentioned each device embodiment of the present invention, the human body critical point detection network includes M network Block, each network block include the multiple convolutional network, and the output of the M network pth network block in the block is described The splicing feature that multiple convolutional networks that pth network block includes obtain, and the splicing feature input of pth network block output Into+1 network block of pth, wherein, M >=2, p=1 ..., M-1；

The corresponding grader of each key point, is specifically used for respectively at least one key point of human body：M network blocks are exported Splicing feature classify, obtain second image block and put for each key point in multiple key points of the human body Reliability image.

Multiple convolutional layers for being located at heterogeneous networks depth respectively are respectively used to carry out feature extraction；

Integrated unit, fisrt feature and network depth for the first convolutional layer that network depth is i to be exported are the of j The second feature of two convolutional layers output carries out Fusion Features, obtains fusion feature, wherein, the second feature is by described Fisrt feature carries out what feature extraction obtained successively by least one convolutional layer, and the characteristics of image of second image block is logical Cross what the fusion feature was handled, 1≤i<j；

The corresponding grader of each key point, is respectively used to the figure to second image block at least one key point of human body As feature is classified, obtain second image block and put for each key point at least one key point of the human body Reliability image.

Another aspect according to embodiments of the present invention, a kind of electronic equipment provided, including：

Memory, for storing executable instruction；And

Processor, it is any of the above-described thereby completing the present invention to perform the executable instruction for communicating with the memory The operation of human body critical point detection method described in embodiment.

Another aspect according to embodiments of the present invention, a kind of computer storage media provided, for storing computer The instruction that can be read, described instruction are performed human body critical point detection method described in any of the above-described embodiment of the present invention of realizing Operation.

Another aspect according to embodiments of the present invention, a kind of computer program product provided, for storing computer Readable instruction, described instruction is performed so that computer performs the human body key described in any of the above-described possible realization method Point detecting method.

In an optional embodiment, the computer program product is specially computer storage media, at another In optional embodiment, the computer program product is specially software product, such as software development kit (Software Development Kit, SDK), etc..

Another aspect according to embodiments of the present invention, a kind of computer program provided, including computer-readable Instruction, when the computer-readable instruction is run in a device, the processor execution in the equipment is used to implement this Invent the executable instruction of the step in human body critical point detection method described in any of the above-described embodiment.

Human body critical point detection method and apparatus, electronic equipment based on the above embodiment of the present invention offer, storage medium And program, Face datection is carried out to image, is obtained in image after the location information of face, according to the location information of the face, really Determine the location information of the corresponding human body central point of face；According to the location information of human body central point, it is crucial that human body is carried out to image Point detection.The embodiment of the present invention is by detecting face location, human body center position carries out human body critical point detection, without The external frame of complete human body is exported, since human body center position is normally in the range of the upper part of the body trunk of human body, and upper half The deformation that body is done is relatively small, therefore the embodiment of the present invention is influenced smaller, to detect human body pass by human body complexity posture Key point position accuracy is higher.

Below by drawings and examples, technical scheme of the present invention is described in further detail.

Description of the drawings

The attached drawing of a part for constitution instruction describes the embodiment of the present invention, and is used to explain together with description The principle of the present invention.

With reference to attached drawing, according to following detailed description, the present invention can be more clearly understood, wherein：

Fig. 1 is the flow chart of human body critical point detection method one embodiment of the present invention.

Fig. 2 is the flow chart of another embodiment of human body critical point detection method of the present invention.

Fig. 3 is the structure diagram of human body critical point detection device one embodiment of the present invention.

Fig. 4 is the structure diagram of another embodiment of human body critical point detection device of the present invention.

Fig. 5 is the structure diagram of human body critical point detection network one embodiment in the embodiment of the present invention.

Fig. 6 is the structure diagram of another embodiment of human body critical point detection network in the embodiment of the present invention.

Fig. 7 is the structure diagram of one Application Example of electronic equipment of the present invention.

Specific embodiment

Carry out the various exemplary embodiments of detailed description of the present invention now with reference to attached drawing.It should be noted that：Unless in addition have Body illustrates that the unlimited system of component and the positioned opposite of step, numerical expression and the numerical value otherwise illustrated in these embodiments is originally The range of invention.

Simultaneously, it should be appreciated that for ease of description, the size of the various pieces shown in attached drawing is not according to reality Proportionate relationship draw.

It is illustrative to the description only actually of at least one exemplary embodiment below, is never used as to the present invention And its application or any restrictions that use.

Technology, method and apparatus known to person of ordinary skill in the relevant may be not discussed in detail, but suitable In the case of, the technology, method and apparatus should be considered as part of specification.

It should be noted that：Similar label and letter represents similar terms in following attached drawing, therefore, once a certain Xiang Yi It is defined in a attached drawing, then in subsequent attached drawing does not need to that it is further discussed.

The embodiment of the present invention can be applied to the electronic equipments such as terminal device, computer system, server, can with it is numerous Other general or specialized computing system environments or configuration operate together.Suitable for electric with terminal device, computer system, server etc. The example of well-known terminal device, computing system, environment and/or configuration that sub- equipment is used together includes but not limited to： Personal computer system, server computer system, thin client, thick client computer, hand-held or laptop devices, based on microprocessor System, set-top box, programmable consumer electronics, NetPC Network PC, little types Ji calculate machine Xi Tong ﹑ large computer systems and Distributed cloud computing technology environment including any of the above described system, etc..

The electronic equipments such as terminal device, computer system, server can be in the department of computer science performed by computer system It is described under the general linguistic context of system executable instruction (such as program module).In general, program module can include routine, program, mesh Beacon course sequence, component, logic, data structure etc., they perform specific task or realize specific abstract data type.Meter Calculation machine systems/servers can be implemented in distributed cloud computing environment, and in distributed cloud computing environment, task is by by logical What the remote processing devices of communication network link performed.In distributed cloud computing environment, program module can be located at and include storage On the Local or Remote computing system storage medium of equipment.

In the implementation of the present invention, the present inventor by the study found that the prior art at least there are the following problems：

Human body critical point detection technology based on depth transducer realization is due to needing depth transducer to realize, depth sensing Device is expensive, is not easy to install and deploy；

Human body critical point detection system based on RGB cameras needs accurately detection position of human body, and position of human body by Human body attitude is affected, inaccurate so as to cause the human body key point position detected.

Fig. 1 is the flow chart of human body critical point detection method one embodiment of the present invention.As shown in Figure 1, the embodiment Human body critical point detection method includes：

102, Face datection is carried out to image, obtains the location information of face in the image.

Image in the embodiment of the present invention can acquire the head portrait obtained by arbitrary camera, for example, RGB cameras or The image that the first-class acquisition of depth camera obtains, correspondingly, which can be coloured picture image etc., and the embodiment of the present application does not do this It limits.

104, according to the location information of above-mentioned face, determine the position of the corresponding human body central point of above-mentioned face (i.e.：Human body Center) information.

106, according to the location information of above-mentioned human body central point, human body critical point detection is carried out to above-mentioned image.

Based on the human body critical point detection method that the above embodiment of the present invention provides, Face datection is carried out to image, is obtained In image after the location information of face, according to the location information of the face, the position letter of the corresponding human body central point of face is determined Breath；According to the location information of human body central point, human body critical point detection is carried out to image.The embodiment of the present invention can pass through detection Face location and human body center position carry out human body critical point detection, since human body center position is normally at the upper of human body In the range of half body trunk, and the deformation of upper part of the body trunk is relatively small, therefore is carried out with other according to the external frame of complete human body The mode of critical point detection is compared, and the embodiment of the present invention is influenced smaller, to detect human body key point by human body complexity posture Position accuracy is higher.

When the embodiment of the present invention carries out human body critical point detection by the image that RGB cameras acquire, due to that need not borrow Help the input information of additional such as depth transducer, equipment is easily installed and disposes, and price is relatively low.

In the one of which embodiment of each embodiment of human body critical point detection method provided in an embodiment of the present invention, on Stating operation 104 can include：According to the location information of above-mentioned face, the with the first pre-set dimension is obtained from above-mentioned image One image block, first image block can include at least part of the corresponding human body of above-mentioned face；According to first image block With head body mapping network, the location information of the corresponding human body central point of above-mentioned face is determined.

In an optional mode of various embodiments of the present invention, head body mapping network can be neural network, such as depth Neural network.

Optionally, the first image block includes at least part of human body, and as an optional example, which can With include human body central point and its more than part or including upper half of human body, etc., the embodiment of the present application is to the first image block Specific implementation do not limit.

In one or more realization methods, the first image block can be inputted body mapping network to the end, be reflected using head body It penetrates network to handle first image block, obtains the location information of human body central point.

In one or more realization methods of the embodiment of the present invention, human body central point can be included positioned at human body center One point or multiple points positioned at human body central area, the embodiment of the present application do not limit the specific implementation of the human body central point It is fixed.Optionally, the location information of face can serve to indicate that the position of face, in one or more embodiments, face Location information can include the center of face or the position of multiple boundary points including face, etc., and the application is implemented Example does not limit the specific implementation of the location information of face.As an optional embodiment, the location information of face can be with The dimension information of center and face including face.Wherein, the dimension information of face here can scheme including face Size as in can also include other information, and the embodiment of the present application not limit this.

In an optional embodiment, according to the location information of face, being obtained from above-mentioned image has first to preset First image block of size, can include：

According to the dimension information of face, the normalized parameter of image is determined；

According to the normalized parameter of the image, size normalized is carried out to image, obtains the figure after size normalization Picture；

According to the center of above-mentioned face, in the image after size normalization interception with the first pre-set dimension the One image block.For example, in wherein one or more optional embodiments, interception is with above-mentioned in the image after size normalization The first image block centered on the center of face, with the first pre-set dimension, can also be with apart from the center of above-mentioned face Centered on the position of position pre-determined distance, first image of the interception with the first pre-set dimension in the image after size normalization Block.

Illustratively, it according to the dimension information of face, determines the normalized parameter of image, can include：

According to the dimension information of above-mentioned face, determine the size normalization of above-mentioned face (i.e.：Scaling) to default normalizing Change the corresponding normalized parameter of facial size.

Illustratively, it is intercepted centered on the center of above-mentioned face, tool in the above-mentioned image after size normalization There is the first image block of the first pre-set dimension, can include：Interception is in above-mentioned face from the image after size normalization Centered on heart position, M times of the image block that size is default normalization facial size, and the truncated picture block is zoomed to the One default size obtains above-mentioned first image block, wherein, the value of M is more than 3 and less than 20.

In the above-described embodiments, according to the location information of above-mentioned face, first interception has first to preset from original image First image block of size, then predict by head body mapping network the location information of the corresponding human body central point of the face, it reduces The detection range of human body center, improves the forecasting efficiency and predictablity rate of human body center, and can improve The training effectiveness of head body mapping network.

In the another embodiment of each embodiment of human body critical point detection method of the present invention, aforesaid operations 106 can be with Including：According to the location information of above-mentioned human body central point, second with the second pre-set dimension is obtained from above-mentioned original image Image block, second image block include at least part of the corresponding human body of above-mentioned face；By human body critical point detection network, Human body critical point detection is carried out in second image block.

In an optional mode of various embodiments of the present invention, human body critical point detection network can be neural network, example Such as deep neural network.

Optionally, the second image block includes at least part of human body, and as an optional example, which can To include centered on human body central point, there is the part of the second pre-set dimension or the part including upper half of human body or simultaneously Part including upper half of human body and the lower part of the body, etc., the embodiment of the present application do not limit the specific implementation of the second image block It is fixed.

In one or more realization methods, the second image block can be input to human body critical point detection network, utilized Human body critical point detection network carries out human body critical point detection to second image block.

Optionally, in a wherein example, according to the location information of above-mentioned human body central point, from above-mentioned original image The second image block with the second pre-set dimension is obtained, can be included：Interception is with above-mentioned people from the image after size normalization The second image block centered on the position of body central point, with the second pre-set dimension.

Optionally, in the above-described embodiment, human body critical point detection is carried out in the second image block, can be included：It is right Second image block carries out human body critical point detection, obtains the position of the key point of human body in the second image block.

Illustratively, human body critical point detection is carried out to the second image block, obtains the key point of human body in the second image block Position, can include：

Critical point detection is carried out to the second image human body in the block, obtains the second image block is directed to human body at least one respectively The confidence image of each key point in a key point, wherein, the confidence image of a key point is included in the second image block extremely A few position is judged as the confidence level of the affiliated key point of the confidence image；

The location determination of confidence level maximum in the confidence image of key point is existed for the affiliated key point of the confidence image Second image position in the block.

Based on the above embodiment, in another embodiment of human body critical point detection method of the present invention, the second figure is obtained After the position of the key point of human body in block, it can also include：According to the position of the key point of human body in the second image block and The position of second image block in the picture determines the position of the key point of human body in the picture.

It, can in the following way, according to the pass of human body in the second image block in wherein one optional embodiment Position of the position and the second image block of key point in described image, determines the position of the key point of human body in the picture：

According in the image of the position of the key point of human body in the second image block and the second image block after size normalization Position, by the position acquisition human body of the key point of human body in the second image block key point size normalization after image in Position；

Based on the key point of human body size normalization after image in position and above-mentioned normalized parameter, by human body Position of the key point of location determination human body in image of the key point after size normalization in original image.

In above-described embodiment, first intercepted from original image according to the location information of above-mentioned human body central point pre- with second If the second image block of size, then by human body critical point detection network, human body key point inspection is carried out in second image block It surveys, reduces critical point detection range, improve the efficiency and accuracy rate of critical point detection, above-described embodiment is crucial for human body During the training of point detection network, the training effectiveness of human body critical point detection network can be improved.

Fig. 2 is the flow chart of another embodiment of human body critical point detection method of the present invention.As shown in Fig. 2, the embodiment Human body critical point detection method include：

202, Face datection is carried out to image, obtains the location information of face in the image, the center including face With the size of face (i.e.：Picture size) information.

Wherein, the dimension information of face can indicate the size of face in the picture, as an example, the size of face Information can include face length w and face width h, i.e. the width h of the length w and face of face in the picture in the picture. The center of face can refer to the position coordinates at the center of face in image, but the embodiment of the present application does not limit this.

204, by the dimension information of face, calculate and the size of above-mentioned face is normalized into default normalization facial size Corresponding normalized parameter.

Can image be normalized in (i.e. size normalized) according to the dimension information of face, so that The facial size in image after normalized is preset normalization facial size.Wherein, the preset normalization face Size can be specially normalized face width or normalized face length or be that normalized face is long The sum of degree and normalized face width, etc., the embodiment of the present application does not limit this.The numerical value of the normalization facial size It can set according to actual needs, such as can be 29 pixels, the embodiment of the present application implements it and is not construed as limiting.

Optionally, the normalized parameter used when image being normalized can be the dimension information according to face It is determined with preset normalization facial size.In the optional example of one of which of various embodiments of the present invention, normalized parameter S It can be determined by following formula：S=Wr/ (w+h), wherein, Wr is preset normalization facial size.As normalization facial size Wr When being normalized face width, normalized parameter S is calculated by above formula, the face brought due to facial angle can be prevented long Degree and/or face widths affect.

206, size normalized is carried out to above-mentioned image according to normalized parameter, obtains the figure after size normalization Picture.

As an optional example, interpolation method may be used, image is normalized.For example, original image Length and width is respectively W and H, then can by original image into row interpolation, such as bilinear interpolation or other interpolation Original image is normalized to size as W × S, the image of H × S, but the embodiment of the present application does not limit this by method.

208, it is intercepted from the image after size normalization centered on the center of face, there is the first pre-set dimension The first image block.

In the optional example of one of which of various embodiments of the present invention, can from size normalization after image in interception with Centered on the center of face, size be normalize facial size M times of image block and zoom to the above-mentioned first default ruler It is very little, the first image block is obtained, wherein, the value of M can be more than 3 and less than 20.

First default size can be set according to actual needs, and in an optional example, the first default size for example may be used To be 256*256 pixels, but the specific implementation of the embodiment of the present application default size to first does not limit.

Optionally, it is contemplated that the head body ratio of people, the value of M can be 6~11, at this point, the first image block can include Human body central point can specifically include the major part of upper half of human body or upper half of human body or including upper half of human body and people A part for the body lower part of the body, can be different according to the difference of actual conditions, and the embodiment of the present application does not limit this.

As an optional example, M=9.The present inventor has found that personal body length is about that the number of people is long by data statistics 8 times of degree when the value of M is 9, can be intercepted centered on the center of face from the image after size normalization, be returned One changes the image block of 9 times of length of facial size, which can include the complete upper half body of human body, and can also wrap A part of background information in image is included, so as to contribute to subsequent image characteristics extraction and critical point detection.Further, since the One image block is used to predicting the center of human body, and human body center is normally at the upper part of the body of human body, therefore human body Lower itself is not critically important for the prediction of the center of human body, in the first image block can not including leg etc. the human bodies lower part of the body Part, so as to reduce data processing amount, improve detection efficiency.

210, by head body mapping network, determine the position of the corresponding human body central point of face in above-mentioned first image block.

The first image block can be handled using head body mapping network, obtain the position of human body central point.Optionally, First image block can be directly inputted to body mapping network to the end or can the first image block be subjected to one or more pre- places Reason, and pretreated first image block is inputted into body mapping network to the end.Optionally, the position of human body central point can include In the output of head body mapping network or by the output of correct body mapping network obtain after one or more processing It arrives, the embodiment of the present application does not limit this.

Optionally, head body mapping network can be that network or other kinds of network, the embodiment of the present application do not do this It limits.

212, it intercepted from the image after size normalization centered on the position of above-mentioned human body central point, have second in advance If the second image block of size.

In the optional example of one of which of various embodiments of the present invention, the first pre-set dimension can be with the second pre-set dimension phase It together, can also be different from the second pre-set dimension.

Optionally, the image after size normalization here can be that above image is carried out using normalized parameter S The image that is obtained after normalized or the image obtained after being normalized using other normalized parameters, The embodiment of the present application does not limit this.

As an example, the process of second image block of interception is referred to above to the interception process of the first image block Description, which is not described herein again.

214, by human body critical point detection network, critical point detection is carried out to above-mentioned second image block, obtains the second figure As the position of the key point of human body in block.

Based on the present embodiment, size normalized is carried out to image according to the dimension information of face, human body quilt can be made Normalize to pre-set dimension, from size normalization after image in intercept the second image block prediction human body central point position, into Row critical point detection, can improve the efficiency of the prediction of human body center and critical point detection, and the present embodiment maps for head body During the training of network, the image study difficulty of a body mapping network and human body critical point detection network can be reduced.

216, based on the key point of human body in the second image position in the block and the second image block after size normalization Position relationship in image, by human body key point the second image position acquisition human body in the block key point in size normalizing The position in image after change.

In the optional example of one of which of various embodiments of the present invention, which can realize in the following way：

Critical point detection is carried out to the second image human body in the block, obtains second image block respectively for above-mentioned human body The confidence image of each key point at least one key, the confidence image include each position in the second image block and are judged Confidence level for the affiliated key point of the confidence image；Location determination by confidence level maximum in the confidence image of key point is Affiliated key point is in the second image position in the block.

In some embodiments, which can be some or all key points of human body.Optionally, needle To the confidence image of some key point, which can be including each position at least one position in the second image block The confidence level for being judged as the key point is put, wherein, which can refer specifically to the second image part in the block Or all positions or pixel, the embodiment of the present application do not limit this.

218, according to the position of the key point of human body in the second image block and second image block in above-mentioned image (i.e.：It is former Beginning image) in position, determine position of the key point of human body in above-mentioned image.

In the optional example of one of which of various embodiments of the present invention, which can realize in the following way：Root According to the position in the image of the position of the key point of human body in the second image block and the second image block after size normalization, by the Position in two image blocks in image of the key point of the position acquisition human body of the key point of human body after size normalization；It is based on Position and above-mentioned normalized parameter in image of the key point of human body after size normalization, are existed by the key point of above-mentioned human body Position of the key point of the position acquisition human body in image after size normalization in above-mentioned image.

In first optional embodiment of various embodiments of the present invention, human body critical point detection network can include multiple Convolutional neural networks, multiple convolutional neural networks form the convolutional Neural net of multiple and different branches respectively as a branch Network, the convolutional neural networks of different branches can include the network layer of different number and/or the convolutional neural networks of different branches In more than one network layer parameter can be different.Correspondingly, in the optional example, the second image human body in the block is carried out crucial Point detection obtains confidence image of second image block for key point each at least one key point of human body, can wrap It includes：

Respectively by each convolutional neural networks in the convolutional neural networks of above-mentioned multiple and different branches, the second figure is extracted As the characteristics of image of block；Wherein, the characteristics of image of different convolutional neural networks extractions has different scales；

The characteristics of image for the different scale that the convolutional neural networks of above-mentioned multiple and different branches are extracted splices, and obtains Splice feature；

Classified respectively by the corresponding grader of above-mentioned each key point to above-mentioned splicing feature, obtain the second image block For the confidence image of each key point at least one key point of above-mentioned human body.

The embodiment constructs multiple-limb network structure, can extract the second figure by the convolutional neural networks of different branches As block different scale characteristics of image and spliced, so as to be directed to the overall situation that the second image block captures different levels simultaneously Information and detailed information, the feature based on obtained splicing feature rather than single scale carry out key point decision, improve key The accuracy rate of point detection.

One of them in an optional example of the above embodiment, in the convolutional neural networks of above-mentioned multiple branches Or any number of convolutional neural networks, referred to herein as the first convolutional neural networks, it can include multiple respectively positioned at heterogeneous networks The convolutional layer of depth.Correspondingly, in the example, by the first convolutional neural networks in multiple convolutional neural networks, extraction the The characteristics of image of two image blocks can include：

The fisrt feature and network depth that the first convolutional layer that network depth in first convolutional neural networks is i is exported be The second feature of the second convolutional layer output of j carries out Fusion Features, obtains fusion feature, wherein, second feature is by the One feature carries out what feature extraction obtained successively by least one convolutional layer, the second image of the first convolutional neural networks output The characteristics of image of block to fusion feature by being handled to obtain, 1≤i<j.

Wherein, the mode merged to the feature of the convolutional layer of heterogeneous networks depth extraction can be heterogeneous networks depth The feature of the convolutional layer extraction of degree is spliced or is added.

In some embodiments, the characteristics of image of the first convolutional neural networks output can be the fusion feature, Huo Zheke It is obtained, such as the fusion feature can be input to follow-up volume with carrying out any one or more processing to the fusion feature Lamination, and feature extraction processing is carried out to it by follow-up convolutional layer, the characteristics of image exported, but the embodiment of the present application is unlimited In this.

In the embodiment of the present invention, the first convolutional layer and the second convolutional layer are used only for distinguishing the difference in convolutional neural networks Any two convolutional layer of network depth, is not offered as specific convolutional layer, also, the first convolutional layer and the second convolutional layer can be with It is two convolutional layers of two adjacent convolutional layers or interphase every at least one convolutional layer, in addition, the present invention is implemented In convolutional neural networks in example, one or more convolutional layers may be included before the first convolutional layer, the second convolutional layer it After may also include one or more convolutional layers, the embodiment of the present invention be not intended to be limited in any this.

In second optional embodiment of various embodiments of the present invention, human body critical point detection network can include multiple Network block (block), such as M network block, the input exported as latter network block of previous network block；Each network block can To include the convolutional neural networks of multiple and different branches, i.e.,：Human body critical point detection net in above-mentioned first optional embodiment The convolutional neural networks of multiple and different branches that network includes form a network block in the example, i.e.,：M network in the block The splicing feature that the output of p network blocks is obtained for multiple convolutional neural networks that the pth network block includes, and the pth network The splicing feature of block output is input in+1 network block of pth, wherein, M >=2, p=1 ..., M-1.Correspondingly, in the embodiment, Classified using the corresponding grader of key point each in multiple key points of human body to splicing feature, obtain the second image block For the confidence image of key point each in multiple key points of human body, can include：Using in multiple key points of human body The splicing feature that each corresponding grader of key point exports M network blocks is classified, and is obtained the second image block and is directed to people The confidence image of each key point in multiple key points of body.

In some embodiments, the feature of each network block output can be input in its latter network block, wherein, each The feature of network block output can be the spelling spliced by the feature of previously described multiple convolutional neural networks Connect feature.At this point, optionally, the confidence image of key point can be the spy exported by grader to the last one network block What sign (splicing feature) was classified.

In the third optional embodiment of various embodiments of the present invention, human body critical point detection network can include L Nested networks block.Wherein, nested networks block is specifically as follows nested Inception modules or other kinds of nested WFTA module, this Open embodiment does not limit this.The nested networks block of various embodiments of the present invention includes the network block of multiple branches, multiple The network block of each branch respectively includes the convolution god of multiple branches in the network block of at least one branch in the network block of branch Through network, the feature of the convolutional neural networks extraction of different branches has different scales in same network block.Wherein, L Value is the integer more than 0.In one of embodiment of various embodiments of the present invention, each or wherein at least one is nested In the network block of multiple branches that network block includes, the quantity of the network block of different branches is different.In various embodiments of the present invention In one of embodiment, the convolutional neural networks of different branches can include the network layer of different number and/or difference point More than one network layer parameter can be different in the convolutional neural networks of branch.Correspondingly, in the optional example, to the second image block In human body carry out critical point detection, obtain the second image block and put for each key point at least one key point of human body Reliability image can include：The network block of each branch is to defeated in the multiple branches included respectively by the first nested networks block Enter object and carry out feature extraction, obtain the first output feature, input object therein includes the second image block or the first nesting The feature of the previous nested networks block output of network block.Specifically, the first nested networks block is L nested networks in the block the During 1 nested networks block, input object is above-mentioned second image block；First nested networks block is L nested networks the in the block 2nd During a any network block in the block to l-th nested networks, input object for previous nested networks block output feature (i.e.： First output feature)；The first output feature that the network block of multiple branches that first nested networks block includes exports is spelled (concate) is met, obtains the second output feature；Utilize the corresponding grader of key point each at least one key point of human body Classify to the second output feature, obtain above-mentioned second image block for each key point at least one key point of human body Confidence image, wherein, the confidence image of key point, which includes at least one position in above-mentioned image block and is judged as this, to be put The confidence level of the affiliated key point of reliability image；It is the confidence by the location determination of confidence level maximum in the confidence image of key point The affiliated key point of image is spent in image position in the block.

Further optionally, in above-mentioned third optional embodiment, above-mentioned L nested networks block can also include the Two nested networks blocks, wherein, the input terminal of the second nested networks block is connect with the output terminal of above-mentioned first nested networks block.Phase Ying Di carries out critical point detection to the second image human body in the block, obtains at least one key that the second image block is directed to human body The confidence image of each key point, can also include in point：The first nested networks block is exported by the second nested networks block Second output feature carry out feature extraction, obtain third output feature.Correspondingly, using at least one key point of human body Each corresponding grader of key point classifies to third output feature, obtains above-mentioned second image block and is directed to human body at least The confidence image of each key point in one key point, wherein, the confidence image of key point is included in above-mentioned image block extremely A few position is judged as the confidence level of the affiliated key point of the confidence image；By confidence level in the confidence image of key point Maximum location determination is the affiliated key point of the confidence image in image position in the block.

In the 4th optional embodiment of various embodiments of the present invention, human body critical point detection network can include multiple It is located at the convolutional layer of heterogeneous networks depth respectively.Correspondingly, in the optional example, the second image human body in the block is carried out crucial Point detection obtains confidence image of second image block for key point each at least one key point of human body, can wrap It includes：

Fisrt feature that the first convolutional layer that network depth is i is exported and the second convolutional layer that network depth is j export Second feature carry out Fusion Features, obtain fusion feature, wherein, second feature be by fisrt feature by least one Convolutional layer carries out what feature extraction obtained successively, and the characteristics of image of the second image block is by being handled to obtain to fusion feature , 1≤i<j.

This embodiment increases parallel link, the feature that the two or more convolutional layers positioned at heterogeneous networks depth are extracted It is merged, the feature that lower level convolutional layer extracts is connected to the feature of higher level convolutional layer extraction, obtained from The detailed information of the information content of feature and the feature of high-level convolutional layer extraction is increased in fusion feature, is further key point Decision gives particulars feature, so as to help to improve the accuracy rate of critical point detection.

The embodiment of the present invention realizes human body critical point detection network using convolutional neural networks, ensure that critical point detection Robustness and accuracy.The embodiment of the present invention further improves pass by building multiple-limb network structure, increasing parallel link The accuracy of key point detection.

In addition, above-mentioned 4th optional embodiment can also can with above-mentioned second optional embodiment or third Embodiment is selected to be used in combination, it, can be with when being used in combination with second optional embodiment or third optional embodiment To wherein one or more network blocks, merged in the convolutional network of one of which or multiple and different branches using features described above Technical solution；Can also the feature that different levels network block exports be subjected to parallel link, for example, first network block is defeated The feature gone out and the feature of the 5th convolutional layer output are merged, the feature as the 5th network block output.

Further, before the human body critical point detection embodiment of the method flow of the various embodiments described above of the present invention, may be used also To include：

It is trained by first sample image block enemy's body mapping network, first sample image block is labeled with human body center The position of point；And/or

Human body critical point detection network is trained by the second sample image block, the second sample image block marks someone The key point information of body.

In a kind of optional example of the embodiment of the present invention, head body mapping network includes multiple convolutional layers and a classification Layer.It being trained by first sample image block enemy's body mapping network, first sample image block is labeled with human body center, Such as following training flow can be included：

Using the method for any of the above-described embodiment of the present invention, using normalized parameter S to the original images of all acquisitions into Row size normalized makes face long side or broadside be equal to normalization facial size Wr, such as 29 pixels；

It is intercepted from the image after size normalization centered on the center of face, size is normalization facial size M times of image block and zoom to the first default size, for example, being cut out from original image small for normalization facial size 9 times of image block simultaneously zooms to 256*256 pixel sizes, and as first sample image block input head body mapping network, this first The position of accurate human body central point is labeled in sample image block as supervision label；

Multiple convolutional layers in head body mapping network pass through point successively to carrying out feature extraction in first sample image block Class layer judges that each position is (i.e. in first sample image block based on the feature that multiple convolutional layers export：Each pixel) it is judged as The confidence level of weber's point chooses position of the position of confidence level maximum as prediction human body central point；

Between the position of the human body central point marked by first sample image block and the position for predicting human body central point Difference is trained by stochastic gradient descent method enemy's body mapping network, adjusts the net of each network layer of head body mapping network Network parameter value, until meeting preset condition.

Above-mentioned training process can be a repetitive exercise process, i.e.,：Above-mentioned trained flow is repeated, until meeting pre- If condition, for example, frequency of training reaches the position of the human body central point of preset times or first sample image block mark and pre- The difference surveyed between the position of human body central point is less than the first preset value.

After the completion of the training of head body mapping network, a body mapping can be compared to head body mapping network input test image Whether the position of human body central point is correct in the image of network output.

In the optional example of another kind of the embodiment of the present invention, human body critical point detection network includes many levels network Block, the input for exporting the network block as latter level of the network block of previous level；Each network block includes multiple and different points The convolutional network of branch.

Human body critical point detection network is trained by the second sample image block, the second sample image block marks someone The key point information of body, such as following training flow can be included：

After the position of human body central point and normalized parameter S is obtained, first with normalized parameter S to original image into Row scaling obtains the normalized human body image of size；

According to the position of above-mentioned human body central point, the image block that specific second pre-set dimension is cut out from original image is made Human body critical point detection network is inputted for the second sample image block.Wherein the second pre-set dimension for example can be 256*256 pixels Size.For each key point of human body, each position putting there are the key point in a second sample image block is generated respectively Training supervision message of the reliability image as human body critical point detection network, confidence image therein is by being labeled as human body Around the position of key point, given birth to using distance of the Gaussian response function based on each position in the second sample image block to the key point Into；

Each network layer block carries out feature extraction to the second sample image block successively.Wherein, different branches in each network layer block Convolutional network extract the characteristics of image of different scale respectively and carry out merging features, obtain splicing feature；Heterogeneous networks layer block Or the convolutional layer parallel link of heterogeneous networks depth carries out Fusion Features in consolidated network block；

Respectively by the corresponding grader of key point each at least one key point of human body, to many levels network block most The feature exported eventually is classified, and obtains confidence image of the second sample image block for each key point of human body, i.e.,：It is every One key point to be detected exports a corresponding confidence image, which includes every in the second sample image block A position is judged as the confidence level of the key point.Wherein, each key point of human body corresponds to a grader, for judging second Everybody is set to the confidence image of the key point in sample image block；

It is directed to the confidence image of each key point respectively, chooses the position of a confidence level maximum as key point position, So as to obtain the predicted position of each key point of human body；

The human body key point position that is determined by the training supervision message of the second sample image block and each key point of human body Difference between predicted position is trained human body critical point detection network by stochastic gradient descent method, and adjustment human body closes Key point detects the network parameter values of each network layer of network, until meeting preset condition.

Above-mentioned training process can be a repetitive exercise process, i.e.,：Above-mentioned trained flow is repeated, until meeting pre- If condition, for example, frequency of training reaches preset times or by the human body key point position that determines of training supervision message and people Difference between the predicted position of each key point of body is less than the second preset value.

After the completion of human body critical point detection network training, can to human body critical point detection network inputs test image, Whether the predicted position for comparing each key point of human body in the image of human body critical point detection network output is correct.

It, can be in each net in the training flow of above-mentioned human body critical point detection network in further optional example Increase the corresponding grader of each key point of human body after the output layer of network block, classify to the feature of current network block output, The confidence image of each key point of human body is obtained, and thus obtains the predicted position of each key point of human body, then passes through second Each key point of human body that the human body key point position and current network block that the training supervision message of sample image block determines obtain Difference between predicted position as current difference, further combined with each network block current difference to human body critical point detection net Network is trained, so as to training for promotion efficiency and training result.

In the training flow of the above-mentioned correct body mapping network of the present invention and human body critical point detection network, with above-mentioned each human body The realization of technology contents is corresponded in critical point detection method, the mode that corresponding any embodiment may be used is realized, the present invention It repeats no more.

Any human body critical point detection method provided in an embodiment of the present invention can have data by any suitable The equipment of reason ability performs, including but not limited to：Terminal device and server etc..It is alternatively, provided in an embodiment of the present invention any Kind of human body critical point detection method can be performed by processor, as processor is held by the command adapted thereto that memory is called to store Any human body critical point detection method that the row embodiment of the present invention refers to.Hereafter repeat no more.

One of ordinary skill in the art will appreciate that：Realizing all or part of step of above method embodiment can pass through The relevant hardware of program instruction is completed, and aforementioned program can be stored in a computer read/write memory medium, the program When being executed, step including the steps of the foregoing method embodiments is performed；And aforementioned storage medium includes：ROM, RAM, magnetic disc or light The various media that can store program code such as disk.

Fig. 3 is the structure diagram of human body critical point detection device one embodiment of the present invention.The human body of the embodiment closes Key point detection device can be used for realizing the above-mentioned each human body critical point detection embodiment of the method for the present invention.As shown in figure 3, the embodiment Human body critical point detection device include：Face detection module, head body mapping network and human body critical point detection network.Wherein：

Face detection module for carrying out Face datection to image, obtains the location information of face in the image.

Head body mapping network module, for the location information according to above-mentioned face, determines in the corresponding human body of above-mentioned face The location information of heart point.

Human body critical point detection network module, for the location information according to above-mentioned human body central point, to above-mentioned image into Pedestrian's body critical point detection.

Based on the human body critical point detection device that the above embodiment of the present invention provides, Face datection is carried out to image, is obtained In image after the location information of face, according to the location information of the face, the position letter of the corresponding human body central point of face is determined Breath；According to the location information of human body central point, human body critical point detection is carried out to image.The embodiment of the present invention is by detecting face Position, human body center position carry out human body critical point detection, without exporting the complete external frame of human body, due in human body Heart point position is normally in the range of the upper part of the body trunk of human body, and the deformation of upper part of the body trunk is relatively small, therefore the present invention Embodiment is influenced smaller by human body complexity posture, and the human body key point position accuracy detected is higher.

Fig. 4 is the structure diagram of another embodiment of human body critical point detection device of the present invention.As shown in figure 4, and Fig. 3 Shown embodiment is compared, and the human body critical point detection device of the embodiment further includes：First interception module, for according to above-mentioned The location information of face, from above-mentioned image (i.e.：Original image) in obtain with the first pre-set dimension the first image block, first Image block includes at least part of the corresponding human body of face.Correspondingly, in the embodiment, head body mapping network module is specifically used In the location information that the corresponding human body central point of above-mentioned face is determined according to the first image block.

In one optional embodiment of human body critical point detection device embodiment shown in Fig. 4, the location information of face The dimension information of center and face including face.Correspondingly, in the embodiment, the first interception module is specifically used for： According to the dimension information of face, the normalized parameter of image is determined；According to the normalized parameter of image, size is carried out to image and is returned One change is handled, and obtains the image after size normalization；And the center according to face, in the image after size normalizes First image block of the interception with the first pre-set dimension.

Illustratively, the first interception module when determining the normalized parameter of image, is specifically used according to the dimension information of face In：According to the dimension information of face, the normalizing corresponding to by the size scaling of face to preset normalization facial size is determined Change parameter.

Further illustratively, the first interception module is according to the center of face, in the image after size normalization When interception has the first image block of the first pre-set dimension, it is specifically used for：Interception is with face from the image after size normalization Center centered on, size be to normalize M times of image block of facial size, and truncated picture block is zoomed to first Default size, obtains the first image block, wherein, the value of M is more than 3 and less than 20.

In addition, referring back to Fig. 4, in another embodiment of human body critical point detection device, further include：Second interception mould Block for the location information according to above-mentioned human body central point, obtains the second figure with the second pre-set dimension from above-mentioned image As block, which includes at least part of the corresponding human body of face.Correspondingly, in the embodiment, human body key point Network module is detected to be specifically used for carrying out human body critical point detection in the second image block.

In an optional embodiment of above-mentioned human body critical point detection device embodiment, the second interception module is specifically used In：It intercepted from the image after size normalization centered on the position of human body central point, have the second of the second pre-set dimension Image block.

Illustratively, when human body critical point detection network module carries out human body critical point detection in the second image block, tool Body is used for：Human body critical point detection is carried out to the second image block, obtains the position of the key point of human body in the second image block.

Further illustratively, human body critical point detection network module carries out human body critical point detection to the second image block, When obtaining the position of the key point of human body in the second image block, it is specifically used for：Key point is carried out to the second image human body in the block Detection obtains confidence image of second image block for each key point at least one key point of human body, wherein, Mei Geguan The confidence image of key point includes the confidence level that each position in the second image block is judged as the key point；By putting for key point The location determination of confidence level maximum is the affiliated key point of the confidence image in the second image position in the block in reliability image.

In addition, referring back to Fig. 4, in the further embodiment of human body critical point detection device, can also include：Obtain mould Block, for according to the position of the position of the key point of human body and the second image block in the picture in the second image block, determining human body Key point position in the picture.

In a wherein optional embodiment, acquisition module is specifically used for：According to the key of human body in the second image block Position in image of the position and the second image block of point after size normalization, by the key point of human body in the second image block Position in image of the key point of position acquisition human body after size normalization；And the key point based on human body is returned in size One change after image in position and above-mentioned normalized parameter, by human body key point size normalization after image in position Put the position of the key point of determining human body in the picture.

Fig. 5 is the structure diagram of human body critical point detection network module one embodiment in the embodiment of the present invention.Such as Fig. 5 Shown, the human body critical point detection network module of the embodiment includes：Multiple convolutional network modules, concatenation unit and human body respectively close The corresponding grader of key point.Wherein：

Multiple convolutional network modules are respectively used to through convolutional network module each in multiple convolutional network modules, extraction The characteristics of image of second image block, wherein, the characteristics of image of multiple convolutional network moulds different convolutional network module extractions in the block With different scales.

Concatenation unit, the characteristics of image of the different scale for multiple convolutional network modules to be extracted splice, and obtain Splice feature.

The corresponding grader of each key point of human body is respectively used to classify to splicing feature, obtains the second image block needle To the confidence image of each key point at least one key point of human body.

In one optional embodiment of embodiment shown in Fig. 5, each network module can respectively be located at including multiple The convolutional layer of heterogeneous networks module depth.Wherein, multiple convolutional network moulds the first convolution network module in the block, is specifically used for： Fisrt feature that the first convolutional layer that network depth is i is exported and the second convolutional layer that network depth is j export second special Sign carry out Fusion Features, obtain fusion feature, wherein, second feature be by fisrt feature by least one convolutional layer according to Secondary to carry out what feature extraction obtained, the characteristics of image of the second image block of the first convolutional neural networks output is by special to fusion What sign was handled, 1≤i<j.

In a wherein optional example, human body critical point detection network module can include multiple network blocks, such as M Network block, the input exported as latter network block of previous network module block；Each network block includes multiple convolutional network moulds Block, i.e.,：The splicing that the output of M network pth network block in the block is obtained for multiple convolutional neural networks that pth network block includes Feature, and the splicing feature of pth network block output is input in+1 network block of pth, wherein, M >=2, p=1 ..., M-1.Phase Ying Di, in the embodiment, each network concatenation unit in the block in human body critical point detection network module, specifically for M The splicing feature of network block output is classified, and obtains the second image block for each crucial in multiple key points of the human body The confidence image of point.

Fig. 6 is the structure diagram of another embodiment of human body critical point detection network module in the embodiment of the present invention.Such as Shown in Fig. 6, the human body critical point detection network module of the embodiment includes：Multiple volumes for being located at heterogeneous networks module depth respectively Lamination, integrated unit and the corresponding grader of each key point of human body.Wherein：

Multiple convolutional layers for being located at heterogeneous networks module depth respectively are respectively used to carry out feature to the second image block successively Extraction.

Integrated unit, fisrt feature and network depth for the first convolutional layer that network depth is i to be exported are the of j The second feature of two convolutional layers output carries out Fusion Features, obtains fusion feature, wherein, second feature is by fisrt feature Carry out what feature extraction obtained successively by least one convolutional layer, the characteristics of image of the second image block is by fusion feature It is handled, 1≤i<j.

The corresponding grader of each key point at least one key point of human body is respectively used to special to the image of the second image block Sign is classified, and obtains confidence image of second image block for each key point at least one key point of human body.

In addition, the embodiment of the present invention additionally provides a kind of electronic equipment, include the human body of any of the above-described embodiment of the present invention Critical point detection device.

In addition, the embodiment of the present invention additionally provides another electronic equipment, including：

Memory, for storing executable instruction；And

Processor, for communicating with memory to perform executable instruction any of the above-described embodiment thereby completing the present invention The operation of human body critical point detection method.

Fig. 7 is the structure diagram of one Application Example of electronic equipment of the present invention.Below with reference to Fig. 7, it illustrates suitable In for realizing the structure diagram of the electronic equipment of the terminal device of the embodiment of the present application or server.As shown in fig. 7, the electricity Sub- equipment includes one or more processors, communication unit etc., and one or more of processors are for example：One or more centres Manage unit (CPU) and/or one or more image processor (GPU) etc., processor can be according to being stored in read-only memory (ROM) executable instruction in is held from the executable instruction that storage section is loaded into random access storage device (RAM) Row various appropriate actions and processing.Communication unit may include but be not limited to network interface card, and the network interface card may include but be not limited to IB (Infiniband) network interface card, processor can communicate to perform executable finger with read-only memory and/or random access storage device It enables, is connected by bus with communication unit and communicated through communication unit with other target devices, provided so as to complete the embodiment of the present application The corresponding operation of either method, for example, to image carry out Face datection, obtain described image in face location information；Root According to the location information of the face, the location information of the corresponding human body central point of the face is determined；According to the human body center The location information of point carries out human body critical point detection to described image.

In addition, in RAM, it can also be stored with various programs and data needed for device operation.CPU, ROM and RAM lead to Bus is crossed to be connected with each other.In the case where there is RAM, ROM is optional module.RAM store executable instruction or at runtime to Executable instruction is written in ROM, executable instruction makes processor perform the corresponding operation of any of the above-described method of the present invention.Input/ Output (I/O) interface is also connected to bus.Communication unit can be integrally disposed, may be set to be with multiple submodule (such as Multiple IB network interface cards), and in bus link.

I/O interfaces are connected to lower component：Include the importation of keyboard, mouse etc.；Including such as cathode-ray tube (CRT), the output par, c of liquid crystal display (LCD) etc. and loud speaker etc.；Storage section including hard disk etc.；And including all Such as communications portion of the network interface card of LAN card, modem.Communications portion performs logical via the network of such as internet Letter processing.Driver is also according to needing to be connected to I/O interfaces.Detachable media, such as disk, CD, magneto-optic disk, semiconductor are deposited Reservoir etc. is installed as needed on a drive, in order to be mounted into as needed from the computer program read thereon Storage section.

Need what is illustrated, framework as shown in Figure 7 is only a kind of optional realization method, can root during concrete practice The component count amount and type of above-mentioned Fig. 7 are selected, are deleted, increased or replaced according to actual needs；It is set in different function component Put, can also be used it is separately positioned or integrally disposed and other implementations, such as GPU and CPU separate setting or can be by GPU collection Into on CPU, communication unit separates setting, can also be integrally disposed on CPU or GPU, etc..These interchangeable embodiments Each fall within protection domain disclosed by the invention.

In addition, the embodiment of the present invention additionally provides a kind of computer storage media, for storing computer-readable finger It enables, which is performed the operation for realizing any of the above-described embodiment human body critical point detection method of the present invention.

In addition, the embodiment of the present invention additionally provides a kind of computer program, including computer-readable instruction, work as calculating When the instruction that machine can be read is run in a device, the processor execution in equipment is used to implement any of the above-described embodiment people of the present invention The executable instruction of step in body critical point detection method.

In an optional embodiment, the computer program is specially software product, such as software development kit (Software Development Kit, SDK), etc..

In one or more optional embodiments, the embodiment of the present invention additionally provides a kind of computer program program production Product, for storing computer-readable instruction, described instruction is performed so that computer performs any of the above-described possible realization side Human body critical point detection method described in formula.

The computer program product can be realized especially by hardware, software or its mode combined.In an alternative embodiment In son, the computer program product is embodied as computer storage media, in another optional example, the computer Program product is embodied as software product, such as SDK etc..

In one or more optional embodiments, the embodiment of the present invention additionally provides a kind of human body critical point detection method And its corresponding device and electronic equipment, computer storage media, computer program and computer program product, wherein, it should Method includes：First device sends the instruction of human body critical point detection to second device, and it is above-mentioned which so that second device performs Human body critical point detection method in any possible embodiment；First device receives the human body key point letter that second device is sent Breath.

In some embodiments, human body critical point detection instruction can be specially call instruction, and first device can lead to It crosses the mode called and indicates that second device performs the detection of human body key point, accordingly, in response to receiving call instruction, second Device can perform the step and/or flow in any embodiment in above-mentioned human body critical point detection method.

Particularly, embodiment according to embodiments of the present invention may be implemented as above with reference to the process of flow chart description Computer software programs.For example, the embodiment of the embodiment of the present invention includes a kind of computer program product, including visibly wrapping Containing computer program on a machine-readable medium, computer program was included for the program generation of the method shown in execution flow chart Code, program code may include the corresponding instruction of corresponding execution method and step provided in an embodiment of the present invention, for example, being carried out to image Face datection obtains the instruction of the location information of face in described image；According to the location information of the face, the people is determined The instruction of the location information of the corresponding human body central point of face；According to the location information of the human body central point, to described image into The instruction of pedestrian's body critical point detection.

Each embodiment is described by the way of progressive in this specification, the highlights of each of the examples are with its The difference of its embodiment, the same or similar part cross-reference between each embodiment.For system embodiment For, since it is substantially corresponding with embodiment of the method, so description is fairly simple, referring to the portion of embodiment of the method in place of correlation It defends oneself bright.

Methods and apparatus of the present invention may be achieved in many ways.For example, can by software, hardware, firmware or Software, hardware, firmware any combinations realize methods and apparatus of the present invention.The said sequence of the step of for the method Merely to illustrate, the step of method of the invention, is not limited to sequence described in detail above, special unless otherwise It does not mentionlet alone bright.In addition, in some embodiments, the present invention can be also embodied as recording program in the recording medium, these programs Including being used to implement machine readable instructions according to the method for the present invention.Thus, the present invention also covering stores to perform basis The recording medium of the program of the method for the present invention.

Description of the invention provides for the sake of example and description, and is not exhaustively or will be of the invention It is limited to the form of institute's inventive embodiments.Many modifications and variations are obvious for the ordinary skill in the art.Choosing It is to more preferably illustrate the principle of the present invention and practical application to select and describe embodiment, and makes those of ordinary skill in the art It will be appreciated that the present invention is so as to design the various embodiments with various modifications suitable for special-purpose.

Claims

A kind of 1. human body critical point detection method, which is characterized in that including：

Face datection is carried out to image, obtains the location information of face in described image；

According to the location information of the face, the location information of the corresponding human body central point of the face is determined；

According to the location information of the human body central point, human body critical point detection is carried out to described image.
2. according to the method described in claim 1, it is characterized in that, the location information according to the face, determines described The location information of the corresponding human body central point of face, including：

According to the location information of the face, the first image block with the first pre-set dimension is obtained from described image, it is described First image block includes at least part of the corresponding human body of the face；

According to described first image block and head body mapping network, the location information of the corresponding human body central point of the face is determined.
3. according to the method described in claim 2, it is characterized in that, the location information of the face includes the center of the face Position and the dimension information of the face；

The location information according to the face obtains the first image block with the first pre-set dimension from described image, Including：

According to the dimension information of the face, the normalized parameter of described image is determined；

According to the normalized parameter of described image, size normalized is carried out to described image, after obtaining size normalization Image；

According to the center of the face, intercepted in the image after size normalization described with the first pre-set dimension The first image block.
4. according to the method described in claim 3, it is characterized in that, the dimension information according to the face, determines described The normalized parameter of image, including：

According to the dimension information of the face, it is right to preset normalization facial size institute by the size scaling of the face to determine The normalized parameter answered.
5. according to the method described in claim 4, it is characterized in that, the center according to the face, in the ruler First image block with the first pre-set dimension is intercepted in described image after very little normalization, including：

It is intercepted from the image after size normalization centered on the center of the face, size is the normalization M times of image block of facial size, and the described image block of interception is zoomed into the described first default size, obtain described first Image block, wherein, the value of M is more than 3 and less than 20.
6. according to any methods of claim 1-5, which is characterized in that described to be believed according to the position of the human body central point Breath carries out human body critical point detection to described image, including：

According to the location information of the human body central point, the second image with the second pre-set dimension is obtained from described image Block, second image block include at least part of the corresponding human body of the face；

By human body critical point detection network, human body critical point detection is carried out in second image block.
7. a kind of human body critical point detection device, which is characterized in that including：

Face detection module for carrying out Face datection to image, obtains the location information of face in described image；

Head body mapping network module, for the location information according to the face, determines the corresponding human body central point of the face Location information；

Human body critical point detection network module, for the location information according to the human body central point, to described image into pedestrian Body critical point detection.
8. a kind of electronic equipment, which is characterized in that including：

Memory and processor, wherein, the memory is for storing computer-readable instruction, and the processor is to the calculating The execution of machine readable instruction is so that any human body critical point detection methods of the processor claim 1-6.
A kind of 9. computer storage media, for storing computer-readable instruction, which is characterized in that the computer-readable instruction When running in a device, any human body critical point detection methods of processor perform claim requirement 1-6 in the equipment.
10. a kind of computer program, including computer-readable instruction, which is characterized in that when the computer-readable instruction is being set During standby middle operation, any human body critical point detection methods of processor perform claim requirement 1-6 in the equipment.