Disclosure of Invention
In view of the above-mentioned shortcomings of the prior art, an object of the present invention is to provide a human body posture recommendation method, device, machine-readable medium and apparatus based on human-computer interaction, which are used to solve the problems in the prior art.
In order to achieve the above objects and other related objects, the present invention provides a human body posture recommendation method based on human-computer interaction, including:
identifying a target portrait in a shooting visual field;
extracting key points of the human body based on the target portrait;
and acquiring one or more target posture templates matched with the postures of the target portrait from a posture library based on the human body key points.
Optionally, the recommendation method further includes:
displaying the one or more target pose templates.
Optionally, the gesture template comprises an example graph.
Optionally, the gesture template further comprises a human gesture guide frame, and the example graph is located in the human gesture guide frame.
Optionally, the human body key points include: the vertex, five sense organs, neck, four limbs.
Optionally, obtaining one or more target pose templates matching the pose of the target portrait from a pose library based on the human key points includes:
key point alignment is carried out on the human body key points corresponding to the posture template and the extracted human body key points;
calculating Euclidean distances of each pair of aligned key points, wherein the sum of all the Euclidean distances is the attitude deviation between the attitude template and the target portrait;
and selecting one or more posture templates with the posture deviation smaller than the preset deviation value as one or more target posture templates matched with the postures of the target portrait.
To achieve the above and other related objects, the present invention provides a human body posture recommendation device based on human-computer interaction, comprising:
the image acquisition module is used for identifying a target portrait in a shooting visual field;
the key point extraction module is used for extracting key points of a human body based on the target portrait;
and the matching module is used for acquiring one or more target posture templates matched with the postures of the target portrait from a posture library based on the human body key points. .
Optionally, the recommendation device further includes:
a display module to display the one or more target pose templates.
Optionally, the gesture template comprises an example graph.
Optionally, the gesture template further comprises a human gesture guide frame, and the example graph is located in the human gesture guide frame.
Optionally, the human body key points include: the vertex, five sense organs, neck, four limbs.
Optionally, obtaining one or more target pose templates matching the pose of the target portrait from a pose library based on the human key points includes:
key point alignment is carried out on the human body key points corresponding to the posture template and the extracted human body key points;
calculating Euclidean distances of each pair of aligned key points, wherein the sum of all the Euclidean distances is the attitude deviation between the attitude template and the target portrait;
and selecting one or more posture templates with the posture deviation smaller than the preset deviation value as one or more target posture templates matched with the postures of the target portrait.
To achieve the foregoing and other related objectives, the present invention provides one or more machine-readable media having instructions stored thereon, which when executed by one or more processors, cause an apparatus to perform one or more of the methods described above.
To achieve the above and other related objects, the present invention provides an apparatus comprising:
one or more processors; and
one or more machine-readable media having instructions stored thereon that, when executed by the one or more processors, cause the apparatus to perform one or more of the methods described previously.
As described above, the human body posture recommendation method, device, machine readable medium and apparatus based on human-computer interaction provided by the invention have the following beneficial effects:
the invention can automatically detect the human body posture, intelligently recommend a plurality of photographing postures according to the initial posture of the portrait, and display a corresponding personalized human body posture guide frame on the display equipment, so that the user can adjust the body through visual indication on the basis of the initial posture to obtain a proper posture.
Detailed Description
The embodiments of the present invention are described below with reference to specific embodiments, and other advantages and effects of the present invention will be easily understood by those skilled in the art from the disclosure of the present specification. The invention is capable of other and different embodiments and of being practiced or of being carried out in various ways, and its several details are capable of modification in various respects, all without departing from the spirit and scope of the present invention. It is to be noted that the features in the following embodiments and examples may be combined with each other without conflict.
It should be noted that the drawings provided in the following embodiments are only for illustrating the basic idea of the present invention, and the components related to the present invention are only shown in the drawings rather than drawn according to the number, shape and size of the components in actual implementation, and the type, quantity and proportion of the components in actual implementation may be changed freely, and the layout of the components may be more complicated.
As shown in fig. 1, a human body posture recommendation method based on human-computer interaction includes:
s11 identifying the target portrait in the shooting visual field;
and identifying a target portrait appearing in the shooting visual field through a human body detection algorithm, and calculating a circumscribed rectangular frame of the portrait. The position of the target portrait can be confirmed through the circumscribed rectangle frame. The human body detection algorithm includes, but is not limited to, HOG + AdaBoost and Faster-RCNN algorithm.
S12 extracting key points of the human body based on the target portrait;
and extracting human key points from the target portrait by a human key point detection algorithm. The key points of the human body can include the positions of the main joints such as the top of the head, the five sense organs, the neck, the four limbs and the like. The human body key point detection algorithm includes, but is not limited to, G-RMI, CFN (full convolution network).
S13, one or more target pose templates matching the pose of the target portrait are obtained from a pose library based on the human key points.
The gesture library stores a plurality of gesture templates, which can be template photos carried by the terminal device or photos in a photo library shot by the terminal device. For example, a user may set a series of template photographs of different expressions, e.g., lovely, skintone, funny, and selling, each type may have many different templates. The terminal equipment is used for selecting one or more photos from the gesture library as gesture templates, and acquiring a target gesture template selected by a user.
In an embodiment, the recommendation method further comprises displaying one or more target pose templates. The user can adjust the self photographing gesture according to the displayed one or more target gesture templates.
In one embodiment, the gesture template includes a human gesture guiding frame, and an exemplary diagram is disposed in the human gesture guiding frame, as shown in fig. 3. The human body posture guide frame is used for indicating the photographing position and the photographing posture which the photographed person should have, and can visually show the difference between the self posture and the guide frame to the photographed person, so that the self posture can be dynamically adjusted with a target, and the photographing in the standard posture can be completed.
In one embodiment, obtaining one or more target pose templates matching the pose of the target portrait from a pose library based on the human key points comprises:
s131, aligning key points of the human body key points corresponding to the posture template with the extracted human body key points;
specifically, a Procrustes algorithm is adopted to align key points of a human body corresponding to the posture template with the extracted key points of the human body;
s132, calculating Euclidean distances of each pair of aligned key points, wherein the sum of all the Euclidean distances is the attitude deviation between the attitude template and the target portrait;
s133 selects one or more pose templates having a pose deviation smaller than the preset deviation value as one or more target pose templates matched with the pose of the target portrait.
In the process of selecting the attitude template, the attitude deviations can be sequenced from small to large, the templates corresponding to the first N attitude deviations are selected as target attitude templates, and N can be set according to actual conditions.
It can be understood that, when the posture deviation is larger, the difference between the posture of the portrait in the target posture template and the posture of the portrait of the user is considered to be larger; and when the posture deviation is smaller, the difference between the posture of the portrait in the target posture template and the posture of the portrait of the user is considered to be smaller.
The invention can automatically detect the human body posture, intelligently recommend a plurality of photographing postures according to the initial posture of the portrait, and display a corresponding personalized human body posture guide frame on the display equipment, so that the user can adjust the body through visual indication on the basis of the initial posture to obtain a proper posture.
As shown in fig. 4, a human posture recommendation device based on human-computer interaction includes: the system comprises an image acquisition module 11, an image acquisition module 12 and a matching module 13;
the image acquisition module is used for identifying a target portrait in a shooting visual field;
and identifying a target portrait appearing in the shooting visual field through a human body detection algorithm, and calculating a circumscribed rectangular frame of the portrait. The position of the target portrait can be confirmed through the circumscribed rectangle frame. The human body detection algorithm includes, but is not limited to, HOG + AdaBoost and Faster-RCNN algorithm.
The image acquisition module extracts key points of a human body based on the target portrait;
and extracting human key points from the target portrait by a human key point detection algorithm. The key points of the human body can include the positions of the main joints such as the head, the five sense organs, the neck, the four limbs and the like. The human body key point detection algorithm includes, but is not limited to, G-RMI, CFN (full convolution network).
The matching module is used for acquiring one or more target posture templates matched with the postures of the target portrait from a posture library based on the human body key points. .
The gesture library stores a plurality of gesture templates, which can be template photos carried by the terminal device or photos in a photo library taken by the terminal device. For example, a user may set a series of template photographs of different expressions, e.g., lovely, skintone, funny, and selling, each type may have many different templates. The terminal equipment is used for selecting one or more photos from the gesture library as gesture templates, and acquiring a target gesture template selected by a user.
In one embodiment, the recommendation apparatus further comprises:
a display module to display one or more target pose templates. The user can adjust the self photographing gesture according to the displayed one or more target gesture templates.
In an embodiment, the gesture template includes an example graph. The gesture template further comprises a human gesture guide frame, and the example graph is located in the human gesture guide frame.
As shown in fig. 3. The human body posture guide frame is used for indicating the photographing position and the photographing posture which the photographed person should have, and can visually show the difference between the self posture and the guide frame to the photographed person, so that the self posture can be adjusted in a targeted manner, and the photographing in the standard posture can be completed.
In an embodiment, optionally, obtaining one or more target pose templates matching the pose of the target portrait from a pose library based on the human key points includes:
key point alignment is carried out on the human body key points corresponding to the posture template and the extracted human body key points;
specifically, a Procrustes algorithm is adopted to align key points of a human body corresponding to the posture template with the extracted key points of the human body;
calculating Euclidean distances of each pair of aligned key points, wherein the sum of all the Euclidean distances is the attitude deviation between the attitude template and the target portrait;
and selecting one or more posture templates with the posture deviation smaller than the preset deviation value as one or more target posture templates matched with the postures of the target portrait.
In the process of selecting the attitude template, the attitude deviations can be sequenced from small to large, the templates corresponding to the first N attitude deviations are selected as target attitude templates, and N can be set according to actual conditions.
It can be understood that, when the posture deviation is larger, the difference between the posture of the portrait in the target posture template and the posture of the portrait of the user is considered to be larger; and when the posture deviation is smaller, the difference between the posture of the portrait in the target posture template and the posture of the portrait of the user is considered to be smaller.
An embodiment of the present application further provides an apparatus, which may include: one or more processors; and one or more machine readable media having instructions stored thereon that, when executed by the one or more processors, cause the apparatus to perform the method of fig. 1. In practical applications, the device may be used as a terminal device, and may also be used as a server, where examples of the terminal device may include: the mobile terminal includes a smart phone, a tablet computer, an electronic book reader, an MP3 (Moving Picture Experts Group Audio Layer III) player, an MP4 (Moving Picture Experts Group Audio Layer IV) player, a laptop, a vehicle-mounted computer, a desktop computer, a set-top box, an intelligent television, a wearable device, and the like.
The present embodiment also provides a non-volatile readable storage medium, where one or more modules (programs) are stored in the storage medium, and when the one or more modules are applied to a device, the device may be caused to execute instructions (instructions) of steps included in the face recognition method in fig. 1 according to the present embodiment.
Fig. 5 is a schematic diagram of a hardware structure of a terminal device according to an embodiment of the present application. As shown, the terminal device may include: an input device 1100, a first processor 1101, an output device 1102, a first memory 1103, and at least one communication bus 1104. The communication bus 1104 is used to implement communication connections between the elements. The first memory 1103 may include a high-speed RAM memory, and may also include a non-volatile storage NVM, such as at least one disk memory, and the first memory 1103 may store various programs for performing various processing functions and implementing the method steps of the present embodiment.
Alternatively, the first processor 1101 may be, for example, a Central Processing Unit (CPU), an Application Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Digital Signal Processing Device (DSPD), a Programmable Logic Device (PLD), a Field Programmable Gate Array (FPGA), a controller, a microcontroller, a microprocessor, or other electronic components, and the first processor 1101 is coupled to the input device 1100 and the output device 1102 through a wired or wireless connection.
Optionally, the input device 1100 may include a variety of input devices, such as at least one of a user-oriented user interface, a device-oriented device interface, a software programmable interface, a camera, and a sensor. Optionally, the device interface facing the device may be a wired interface for data transmission between devices, or may be a hardware plug-in interface (e.g., a USB interface, a serial port, etc.) for data transmission between devices; optionally, the user-facing user interface may be, for example, a user-facing control key, a voice input device for receiving voice input, and a touch sensing device (e.g., a touch screen with a touch sensing function, a touch pad, etc.) for receiving user touch input; optionally, the programmable interface of the software may be, for example, an entry for a user to edit or modify a program, such as an input pin interface or an input interface of a chip; the output devices 1102 may include output devices such as a display, audio, and the like.
In this embodiment, the processor of the terminal device includes a module for executing the functions of the modules of the face recognition apparatus in each device, and specific functions and technical effects may refer to the foregoing embodiments, which are not described herein again.
Fig. 6 is a schematic hardware structure diagram of a terminal device according to an embodiment of the present application. FIG. 6 is a specific embodiment of the implementation of FIG. 5. As shown, the terminal device of the present embodiment may include a second processor 1201 and a second memory 1202.
The second processor 1201 executes the computer program code stored in the second memory 1202 to implement the method described in fig. 5 in the above embodiment.
The second memory 1202 is configured to store various types of data to support operations at the terminal device. Examples of such data include instructions for any application or method operating on the terminal device, such as messages, pictures, videos, and so forth. The second memory 1202 may include a Random Access Memory (RAM) and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.
Optionally, a second processor 1201 is provided in the processing assembly 1200. The terminal device may further include: communication component 1203, power component 1204, multimedia component 1205, speech component 1206, input/output interfaces 1207, and/or sensor component 1208. The specific components included in the terminal device are set according to actual requirements, which is not limited in this embodiment.
The processing component 1200 generally controls the overall operation of the terminal device. The processing assembly 1200 may include one or more second processors 1201 to execute instructions to perform all or part of the steps of the data processing method described above. Further, the processing component 1200 can include one or more modules that facilitate interaction between the processing component 1200 and other components. For example, the processing component 1200 can include a multimedia module to facilitate interaction between the multimedia component 1205 and the processing component 1200.
The power supply component 1204 provides power to the various components of the terminal device. The power components 1204 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the terminal device.
The multimedia components 1205 include a display screen that provides an output interface between the terminal device and the user. In some embodiments, the display screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the display screen includes a touch panel, the display screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.
The voice component 1206 is configured to output and/or input voice signals. For example, the voice component 1206 includes a Microphone (MIC) configured to receive external voice signals when the terminal device is in an operational mode, such as a voice recognition mode. The received speech signal may further be stored in the second memory 1202 or transmitted via the communication component 1203. In some embodiments, the speech component 1206 further comprises a speaker for outputting speech signals.
The input/output interface 1207 provides an interface between the processing component 1200 and peripheral interface modules, which may be click wheels, buttons, etc. These buttons may include, but are not limited to: a volume button, a start button, and a lock button.
The sensor component 1208 includes one or more sensors for providing various aspects of status assessment for the terminal device. For example, the sensor component 1208 may detect an open/closed state of the terminal device, relative positioning of the components, presence or absence of user contact with the terminal device. The sensor assembly 1208 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact, including detecting the distance between the user and the terminal device. In some embodiments, the sensor assembly 1208 may also include a camera or the like.
The communication component 1203 is configured to facilitate communications between the terminal device and other devices in a wired or wireless manner. The terminal device may access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In one embodiment, the terminal device may include a SIM card slot therein for inserting a SIM card therein, so that the terminal device may log onto a GPRS network to establish communication with the server via the internet.
As can be seen from the above, the communication component 1203, the voice component 1206, the input/output interface 1207 and the sensor component 1208 referred to in the embodiment of fig. 6 can be implemented as the input device in the embodiment of fig. 5.
The foregoing embodiments are merely illustrative of the principles and utilities of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or change the above-mentioned embodiments without departing from the spirit and scope of the present invention. Accordingly, it is intended that all equivalent modifications or changes which can be made by those skilled in the art without departing from the spirit and technical spirit of the present invention be covered by the claims of the present invention.