CN109558842A - A kind of method, apparatus, equipment and medium adjusting image display direction - Google Patents
A kind of method, apparatus, equipment and medium adjusting image display direction Download PDFInfo
- Publication number
- CN109558842A CN109558842A CN201811455968.8A CN201811455968A CN109558842A CN 109558842 A CN109558842 A CN 109558842A CN 201811455968 A CN201811455968 A CN 201811455968A CN 109558842 A CN109558842 A CN 109558842A
- Authority
- CN
- China
- Prior art keywords
- image
- positive direction
- face
- partly
- detection model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
- G06V40/165—Detection; Localisation; Normalisation using facial parts and geometric relationships
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/451—Execution arrangements for user interfaces
Abstract
This application discloses a kind of method and apparatus for adjusting image display direction, this method comprises: obtaining image to be displayed and at least partly facial image；Image to be displayed is detected using the first detection model, obtains the picture material positive direction of image to be displayed；Using the second detection model detection at least partly facial image, the face positive direction of at least partly facial image is obtained；Image to be displayed is adjusted according to face positive direction, makes picture material positive direction is consistent with face positive direction to be shown to user.It can be seen that, consider the picture material positive direction of image to be displayed, after determining user's face positive direction by least partly facial image, picture material positive direction based on the first detection model detection image to be displayed that deep learning obtains, the display direction of image content-based positive direction and user's face positive direction adjustment image to be displayed, the display correctness of image to be displayed is improved, image to be displayed is watched and understood conducive to user.
Description
Technical field
This application involves technical field of image processing more particularly to a kind of method, apparatus for adjusting image display direction, set
Standby and medium.
Background technique
With the fast development of intelligence science and technology, for intelligent subscriber terminal device shows image, it will usually consider
Device orientation and the whether consistent problem in user's face direction.
Traditionally, when intelligent subscriber terminal device shows image, can be sentenced based on the image of the relevant user face of acquisition
Whether user's face direction of breaking is consistent with device orientation, if inconsistent, needs to adjust display image in intelligent subscriber terminal
Display direction in equipment.
But inventor has found that the method that above-mentioned adjustment shows image direction is direct adjustment display image
Display direction in intelligent subscriber terminal device, there is no the positive directions for considering picture material in display image, it is most likely that
So that be eventually displayed to user display image picture material positive direction and user's face direction it is inconsistent, be unfavorable for user
It watches and understands the display image, cause certain viewing and understand difficulty, it is poor to eventually lead to user experience.
Summary of the invention
Technical problems to be solved in this application be to provide it is a kind of adjust the method, apparatus of image display direction, equipment and
Medium considers the picture material positive direction problem of image to be displayed, so that the picture material positive direction of image to be displayed and user
Face positive direction is consistent, improves the display correctness of image to be displayed, watches conducive to user and understands the image to be displayed, is promoted
User experience.
In a first aspect, the embodiment of the present application provides a kind of method for adjusting image display direction, this method comprises:
Obtain image to be displayed and at least partly facial image；
According to the image to be displayed and the first detection model, the picture material positive direction of the image to be displayed is obtained；
According at least partly facial image and the second detection model, the face of at least partly facial image is obtained
Positive direction；
The display direction of the image to be displayed is adjusted according to described image content positive direction and the face positive direction.
Optionally, first detection model is picture material positive direction detection model.
Optionally, described image content positive direction detection model is based on the first training set of images using densnet network
It is obtained with L2 loss function learning training, the first image training set includes multiple display images and corresponding picture material
Positive direction.
Optionally, second detection model is face positive direction detection model or face critical point detection model.
Optionally, the face positive direction detection model is based on the second training set of images using densnet network and L2
Loss function learning training obtains, and second training set of images includes multiple at least partly facial images and corresponding face
Positive direction；The face critical point detection model is based on third training set of images using 2 rank stacked-hourglass networks
It is obtained with L2 loss function learning training, the third training set of images includes multiple at least partly facial images and corresponding
Face key point coordinate.
Optionally, the face key point coordinate includes right and left eyes canthus coordinate and nose coordinate.
Optionally, when second detection model be the face positive direction detection model when, it is described according at least
Partial face image and the second detection model obtain the face positive direction of at least partly facial image, specifically:
According at least partly facial image and the face positive direction detection model, at least partly face is determined
The face positive direction of image.
Optionally, described according to when second detection model is the face key point coordinate measurement model
At least partly facial image and the second detection model obtain the face positive direction of at least partly facial image, comprising:
According at least partly facial image and face critical point detection model, at least partly facial image is obtained
Target face key point coordinate, the target face key point coordinate includes that target right and left eyes canthus coordinate and target nose are sat
Mark；
According to target right and left eyes canthus coordinate, the target nose coordinate and face positive direction calculation formula, determine
The face positive direction of at least partly facial image.
Optionally, described according to target right and left eyes canthus coordinate, the target nose coordinate and face positive direction meter
Formula is calculated, the face positive direction of at least partly facial image is obtained, comprising:
According to target right and left eyes canthus coordinate, target right and left eyes canthus midpoint coordinates is obtained；
It is calculated according to target right and left eyes canthus midpoint coordinates, the target nose coordinate and the face positive direction public
Formula obtains the face positive direction of at least partly facial image.
Optionally, described that the image to be displayed is adjusted according to described image content positive direction and the face positive direction
Display direction, comprising:
According to described image content positive direction and the face positive direction, rotation angle is obtained；
The image to be displayed is rotated according to the rotation angle.
Second aspect, the embodiment of the present application provide a kind of device for adjusting image display direction, which includes:
First obtains unit, for obtaining image to be displayed and at least partly facial image；
Second obtaining unit, for obtaining the image to be displayed according to the image to be displayed and the first detection model
Picture material positive direction；
Third obtaining unit, for according at least partly facial image and the second detection model, acquisition to be described at least
The face positive direction of partial face image；
Adjustment unit, for adjusting the image to be displayed according to described image content positive direction and the face positive direction
Display direction.
The third aspect, the embodiment of the present application provide a kind of terminal device, and the terminal device includes processor and deposits
Reservoir:
Said program code is transferred to the processor for storing program code by the memory；
The processor is used for according to the adjustment of any one of the above-mentioned first aspect of instruction execution in said program code
The method of image display direction.
Fourth aspect, the embodiment of the present application provide a kind of computer readable storage medium, the computer-readable storage
Medium is for storing program code, and said program code is for executing any one of above-mentioned first aspect adjustment image display side
To method.
Compared with prior art, the application has at least the following advantages:
Using the technical solution of the embodiment of the present application, firstly, obtaining image to be displayed and at least partly facial image；So
Afterwards, the image to be displayed is detected using the first detection model, obtains the picture material positive direction of the image to be displayed；And benefit
With the second detection model detection at least partly facial image, the face positive direction of at least partly facial image is obtained；
Finally, adjusting the display direction of the image to be displayed according to described image content positive direction and the face positive direction.Thus
It can be seen that, it is contemplated that the picture material positive direction problem of image to be displayed is determining user's face by least partly facial image
After positive direction, the picture material positive direction based on the first detection model detection image to be displayed that deep learning obtains, based on figure
As the display direction of content positive direction and user's face positive direction adjustment image to be displayed, so that the image of image to be displayed
Content positive direction is consistent with user's face positive direction, improves the display correctness of image to be displayed, is conducive to user and watches and understand
The image to be displayed promotes user experience.
Detailed description of the invention
Technical solution in ord to more clearly illustrate embodiments of the present application, below will be to required in the embodiment of the present application description
Attached drawing to be used is briefly described, it should be apparent that, the accompanying drawings in the following description is only more as described in this application
Embodiment for those of ordinary skill in the art without creative efforts, can also be attached according to these
Figure obtains other attached drawings.
Fig. 1 is system framework schematic diagram involved in application scenarios a kind of in the embodiment of the present application；
Fig. 2 is a kind of flow diagram of method for adjusting image display direction provided by the embodiments of the present application；
Fig. 3 is picture material positive direction angle, θ provided by the embodiments of the present application1Schematic diagram；
Fig. 4 is face positive direction angle, θ provided by the embodiments of the present application2Schematic diagram；
Fig. 5 is face positive direction angle, θ provided by the embodiments of the present application2Greater than picture material positive direction angle, θ1Schematic diagram；
Fig. 6 is face positive direction angle, θ provided by the embodiments of the present application2Less than picture material positive direction angle, θ1Schematic diagram；
Fig. 7 is a kind of flow diagram of method for obtaining face positive direction provided by the embodiments of the present application；
Fig. 8 is a kind of structural schematic diagram of device for adjusting image display direction provided by the embodiments of the present application.
Specific embodiment
In order to make those skilled in the art more fully understand application scheme, below in conjunction in the embodiment of the present application
Attached drawing, the technical scheme in the embodiment of the application is clearly and completely described, it is clear that described embodiment is only this
Apply for a part of the embodiment, instead of all the embodiments.Based on the embodiment in the application, those of ordinary skill in the art exist
Every other embodiment obtained under the premise of creative work is not made, shall fall in the protection scope of this application.
At this stage, when using intelligent subscriber terminal device show image when, processor by determine user's face direction with
Whether device orientation is consistent, display direction of the adjustment display image in intelligent subscriber terminal device.But inventor is by grinding
Study carefully discovery, the method that above-mentioned adjustment shows image direction is display of the direct adjustment display image in intelligent subscriber terminal device
Direction, there is no the positive directions for considering picture material in display image, it is most likely that so that being eventually displayed to the display figure of user
The picture material positive direction of picture and user's face direction are inconsistent, are unfavorable for user's viewing and understand the display image, cause
Certain viewing and understand difficulty, it is poor to eventually lead to user experience.
In order to solve this problem, in the embodiment of the present application, firstly, obtaining image to be displayed and at least partly face figure
Picture；Then, the image to be displayed is detected using the first detection model, the picture material for obtaining the image to be displayed is square
To；And at least partly facial image is detected using the second detection model, obtain the face of at least partly facial image
Positive direction；Finally, adjusting the display side of the image to be displayed according to described image content positive direction and the face positive direction
To.It can be seen that, it is contemplated that the picture material positive direction problem of image to be displayed is determining to use by least partly facial image
After the face positive direction of family, the picture material positive direction based on the first detection model detection image to be displayed that deep learning obtains,
The display direction of image content-based positive direction and user's face positive direction adjustment image to be displayed, so that image to be displayed
Picture material positive direction it is consistent with user's face positive direction, improve the display correctness of image to be displayed, be conducive to user watch
With understand the image to be displayed, promote user experience.
For example, one of the scene of the embodiment of the present application, can be applied in scene as shown in Figure 1.The scene
Including intelligent subscriber terminal device 101, processor 102 and camera 103, wherein processor 102 and camera 103 are loaded in
Intelligent subscriber terminal device 101.Camera 103 shoots at least partly facial image and is sent to processor 102；Processor 102 obtains
At least partly facial image that the image to be displayed and camera 103 for obtaining intelligent subscriber terminal device 101 are sent；Processor 102
According to the image to be displayed and the first detection model, the picture material positive direction of the image to be displayed is obtained；Processor 102
According at least partly facial image and the second detection model, the face positive direction of at least partly facial image is obtained；
Processor 102 adjusts the display direction of the image to be displayed according to described image content positive direction and the face positive direction,
It is shown on 101 screen of intelligent subscriber terminal device so that described image content positive direction is consistent with the face positive direction.
It is understood that in above-mentioned application scenarios, although by the action description of the application embodiment by server
101 execute, but these movements can also be executed by client 102, or can also partially be executed by client 102, part
It is executed by server 101.The application is unrestricted in terms of executing subject, as long as performing disclosed in the application embodiment
Movement.
It is understood that above-mentioned scene is only a Sample Scenario provided by the embodiments of the present application, the embodiment of the present application
It is not limited to this scene.
With reference to the accompanying drawing, the method that image display direction is adjusted in the embodiment of the present application is described in detail by embodiment
With the specific implementation of device.
Illustrative methods
Referring to fig. 2, a kind of flow diagram for the method for adjusting image display direction in the embodiment of the present application is shown.?
In the present embodiment, the method for example be may comprise steps of:
Step 201: obtaining image to be displayed and at least partly facial image.
It is understood that in order to realize image positive direction that intelligent subscriber terminal device is shown and user's face positive direction
Unanimously, it is necessary first to which obtaining intelligent subscriber terminal device needs image (that is, image to be displayed) to be shown and watch the image
At least partly facial image, wherein partial face image includes at least local facial feature relevant to face positive direction, example
Such as, eyebrow, eyes, nose or mouth etc. local facial feature.
Step 202: according to the image to be displayed and the first detection model, obtaining the picture material of the image to be displayed
Positive direction.
It is understood that in order to solve the positive direction for not accounting for picture material in display image in the prior art,
So that be eventually displayed to user display image picture material positive direction and user's face direction it is inconsistent, be unfavorable for user see
The problem of seeing and understanding display image obtains image to be displayed and at least partly after facial image in step 202, need clearly to
Show the picture material positive direction of image and the face positive direction of at least partly facial image.In some embodiment party of this fact Example
In formula, the picture material positive direction of image to be displayed can be first obtained.
It should be noted that for image to be displayed, it can be based on to the picture material for largely showing image before
The positive direction deep learning obtained picture material positive direction detection model of training as the first detection model, to image to be displayed into
Row detection, to obtain the picture material positive direction of image to be displayed.Therefore, in some embodiments of the present embodiment, institute
Stating the first detection model is picture material positive direction detection model.
It should be noted that the learning training of picture material positive direction detection model are as follows: firstly, obtaining multiple display images
And determining the picture material positive direction of each display image, picture material positive direction refers to that the true positive direction of picture material is opposite
In angle vertically upward, value range is 0 to 360 degree, by multiple display image AiIt is square with corresponding picture material
(display image A is indicated to Aalpha_iiPicture material positive direction angle) be used as training set of images；Then, using densnet
Network and L2 loss function carry out deep learning training to obtain picture material positive direction detection model, in the present embodiment, the mould
The input of type is image to be displayed, is exported as corresponding picture material positive direction.Therefore, in some embodiments of the present embodiment
In, described image content positive direction detection model is based on the first training set of images using densnet network and L2 loss function
What learning training obtained, the first image training set includes multiple display images and corresponding picture material positive direction.Wherein,
Corresponding L2 loss function specifically:
Loss=‖ y_pred-y_truth ‖；
‖ * ‖ indicates 2- norm in the formula, and y_pred indicates picture material positive direction detection model to the display figure of input
The forecast image content positive direction of picture exports, and y_truth indicates the corresponding true picture content positive direction of above-mentioned display image.
Step 203: according at least partly facial image and the second detection model, obtaining at least partly face figure
The face positive direction of picture.
It is understood that after step 203 obtains the picture material positive direction of true image to be displayed, it is also necessary to obtain extremely
The face positive direction of a minority's face image.Certainly, what the present embodiment was not intended to limit step 202 and step 203 executes sequence, both
Step 202 can be first carried out, then executes step 203；Step 203 can also be first carried out, then executes step 202；It can also be simultaneously
Execute step 202 and step 203.
It should be noted that at least partly facial image, it can be based on to a large amount of at least partly faces before
The face positive direction detection model that the face positive direction deep learning training of image obtains is as the second detection model, at least portion
Facial image is divided to be detected, to obtain the face positive direction of at least partly facial image；Can also have due to face
Organ key point characteristic relevant to face positive direction, it is deep based on the face positive direction to a large amount of at least partly facial images before
The face critical point detection model that degree learning training obtains examines at least partly facial image as the second detection model
It surveys, to obtain the face key point of at least partly facial image, then face positive direction is determined based on face key point.Therefore,
In some embodiments of the present embodiment, second detection model is that face positive direction detection model or face key point are examined
Survey model.
Wherein, similarly in picture material positive direction detection model, the learning training of face positive direction detection model are as follows: first
First, multiple at least partly facial images are obtained and determine the face positive direction of each at least partly facial image, face positive direction
Refer to the true positive direction of face relative to angle vertically upward, value range is 0 to 360 degree, by multiple at least portions
Divide facial image Bi(at least partly facial image B is indicated with corresponding face positive direction Balpha_iiFace positive direction angle)
As training set of images；Then, densnet network and L2 loss function is equally used to carry out deep learning training to obtain face
Positive direction detection model, in the present embodiment, the input of the model is at least partly facial image, export for corresponding face it is square
To.It should be noted that corresponding L2 loss function specifically:
Loss=‖ y_pred-y_truth ‖；
‖ * ‖ indicates 2- norm in the formula, and y_pred indicates face positive direction detection model at least partly people of input
The prediction face positive direction of face image exports, and y_truth indicates that the corresponding real human face of above-mentioned at least partly facial image is square
To.
Wherein, the learning training of face critical point detection model are as follows: firstly, obtaining multiple at least partly facial images and true
The coordinate of fixed each at least partly face key point of facial image, wherein closed with the higher face of the face positive direction degree of correlation
Key point is left and right canthus and nose, and certainly, face key point may be the key points such as left and right pupil, left and right labial angle.It will be multiple
At least partly facial image CiWith corresponding face key point coordinate Yi(xi1, yi1, xi2, yi2, xi3, yi3, xi4, yi4, xi5,
yi5) (at least partly facial image C is indicatediEach two canthus coordinates in left and right canthus and nose coordinate) be used as training set of images.
Then, it is changed to that 2 rank stacked-hourglass networks and L2 loss function is used to learn to carry out deep learning training to obtain people
Face critical point detection model, in the present embodiment, the input of the model is at least partly facial image, is exported as corresponding face pass
Key point coordinate.It should be noted that corresponding L2 loss function specifically:
Loss=‖ y_pred-y_truth ‖；
‖ * ‖ indicates 2- norm in the formula, and y_pred indicates face critical point detection model at least partly people of input
The prediction face key point coordinate of face image exports, and y_truth indicates the corresponding real human face of above-mentioned at least partly facial image
Key point coordinate.In conclusion in some embodiments of the present embodiment, the face positive direction detection model is based on the
Two training set of images are obtained using densnet network and L2 loss function learning training, and second training set of images includes
Multiple at least partly facial images and corresponding face positive direction；The face critical point detection model is instructed based on third image
Practice what collection was obtained using 2 rank stacked-hourglass networks and L2 loss function learning training, the third training set of images
Including multiple at least partly facial images and corresponding face key point coordinate, the face key point coordinate includes right and left eyes eye
Angular coordinate and nose coordinate.
It should be noted that based on the above-mentioned explanation to the second detection model, the second detection model either face just
Angle detecting model is also possible to face critical point detection model.When the second detection model is face positive direction detection model,
Near a minority's face image inputs the face positive direction detection model, can be directly output to the face of a minority's face image
Positive direction；When the second detection model is face critical point detection model, near a minority's face image inputs face key
Point detection model, can only export the face key point coordinate of at least partly facial image, that is, right and left eyes canthus coordinate and nose are sat
Mark, at this time, it is also necessary to be obtained at least partly based on right and left eyes canthus coordinate and nose coordinate using face positive direction calculation formula
The face positive direction of facial image, concrete mode is referring to next embodiment.Therefore, in some embodiments of the present embodiment
In, when second detection model is the face positive direction detection model, the step 203 is for example specifically as follows: root
According at least partly facial image and the face positive direction detection model, at least partly facial image is directly obtained
Face positive direction.When second detection model is the face key point coordinate measurement model, the step 203 for example may be used
With the following steps are included:
Step A: according at least partly facial image and face critical point detection model, at least partly people is obtained
The target face key point coordinate of face image, the target face key point coordinate include target right and left eyes canthus coordinate and target
Nose coordinate；
Step B: it is calculated according to target right and left eyes canthus coordinate, the target nose coordinate and face positive direction public
Formula obtains the face positive direction of at least partly facial image.
Step 204: the aobvious of the image to be displayed is adjusted according to described image content positive direction and the face positive direction
Show direction.
It is understood that the picture material positive direction and step 203 in step 202 acquisition image to be displayed obtain extremely
After the face positive direction of a minority's face image, in order to enable the picture material for being eventually displayed to the image to be displayed of user is square
To consistent with user's face direction, watched conducive to user and understand image to be displayed, then according to described image content positive direction and
The face positive direction adjusts the display direction of the image to be displayed, just so as to described image content positive direction and the face
Direction is consistent, that is, execute step 204.
It should be noted that since picture material positive direction refers to the true positive direction of picture material relative to vertically upward
The angle in direction, face positive direction refer to the true positive direction of face relative to angle vertically upward, in order to enable figure
It is shown as content positive direction is consistent with face positive direction, image to be displayed can be rotated so as to picture material positive direction and face just
Direction is overlapped.Specifically, based on two angles available two represented by above-mentioned picture material positive direction and face positive direction
The difference is rotated image to be displayed by difference between a angle.Therefore, in some implementations of the present embodiment
In mode, the step 204 for example be may comprise steps of:
Step C: according to described image content positive direction and the face positive direction, rotation angle is obtained；
Step D: the image to be displayed is rotated according to the rotation angle.
In practical applications, the picture material positive direction for the image to be displayed that step 202 obtains is image as shown in Figure 3
Content positive direction angle, θ1, step 203 obtain at least partly facial image face positive direction be face as shown in Figure 4 just
Orientation angle θ2, then according to picture material positive direction angle, θ1With face positive direction angle, θ1Adjust the display side of image to be displayed
To.Specifically according to picture material positive direction angle, θ1With face positive direction angle, θ2, it is θ that image to be displayed, which rotates angle, is obtained
The formula of θ is as follows:
θ=θ2-θ1；
Wherein, in formula, if θ is greater than 0, that is, face positive direction angle, θ as shown in Figure 52Greater than picture material positive direction angle
Spend θ1, then need to rotate clockwise image to be displayed θ | angle；If θ is less than 0, that is, face positive direction angle, θ as shown in FIG. 62
It is less than picture material positive direction angle, θ greatly1, then need to rotate image to be displayed θ counterclockwise | angle.Certainly, if θ is equal to 0,
Without rotating image to be displayed；
The various embodiments provided through this embodiment, firstly, obtaining image to be displayed and at least partly facial image；
Then, the image to be displayed is detected using the first detection model, obtains the picture material positive direction of the image to be displayed；And
At least partly facial image is detected using the second detection model, the face for obtaining at least partly facial image is square
To；Finally, adjusting the display direction of the image to be displayed according to described image content positive direction and the face positive direction.By
This is visible, it is contemplated that the picture material positive direction problem of image to be displayed is determining user people by least partly facial image
After face positive direction, the picture material positive direction based on the first detection model detection image to be displayed that deep learning obtains is based on
The display direction of picture material positive direction and user's face positive direction adjustment image to be displayed, so that the figure of image to be displayed
Picture content positive direction is consistent with user's face positive direction, improves the display correctness of image to be displayed, is conducive to user and watches and manage
The image to be displayed is solved, user experience is promoted.
Referring to Fig. 7, a kind of flow diagram for the method for obtaining face positive direction in the embodiment of the present application is shown.At this
In embodiment, the method for example be may comprise steps of:
Step 701: according at least partly facial image and face critical point detection model, obtaining at least partly face
The target face key point coordinate of image, the target face key point coordinate include target right and left eyes canthus coordinate and target nose
Sharp coordinate.
Explanation in based on the above embodiment it is found that near a minority's face image inputs face critical point detection model,
It can export and obtain the face key point coordinate including target right and left eyes canthus coordinate and target nose coordinate, for example, target is left
Right eye canthus coordinate is respectively (x1, y1), (x2, y2), (x3, y3) and (x4, y4), and target nose coordinate is (x5, y5).
Step 702: being calculated according to target right and left eyes canthus coordinate, the target nose coordinate and face positive direction public
Formula obtains the face positive direction of at least partly facial image.
It should be noted that based on face right and left eyes canthus and nose for face positive direction correlation it is found that at least
Vector and at least partly facial image X-axis (0 degree direction vector) of the nose to right and left eyes canthus midpoint in partial face image
Angle can indicate face positive direction angle, as at least partly face positive direction of facial image.Then in order to obtain at least portion
The face positive direction for dividing facial image, after step 701 obtains target right and left eyes canthus coordinate and target nose coordinate, first
It needs first to obtain right and left eyes canthus midpoint coordinates based on target right and left eyes canthus coordinate, then, is based on right eye canthus midpoint coordinates
The resulting face positive direction calculation formula of principle that above-mentioned acquisition face positive direction is utilized with target nose coordinate, finally calculates
To at least partly facial image face positive direction angle as face positive direction.Therefore, in some embodiment party of the present embodiment
In formula, the step 702 for example be may comprise steps of:
Step C: according to target right and left eyes canthus coordinate, target right and left eyes canthus midpoint coordinates is obtained.
Wherein, it is based on target right and left eyes canthus coordinate (x1, y1), (x2, y2), (x3, y3) and (x4, y4), calculates target
The formula of right and left eyes canthus midpoint coordinates (xc, yc) is specific as follows:
Step D: according to target right and left eyes canthus midpoint coordinates, the target nose coordinate and the face positive direction
Calculation formula obtains the face positive direction of at least partly facial image.
In practical applications, vector (face positive direction vector) and at least partly face of the nose to right and left eyes canthus midpoint
Image X-axis (0 degree of direction vector v2(0,1)) angle can indicate face positive direction, firstly, based in target right and left eyes canthus
Point coordinate (xc, yc) and target nose coordinate are (x5, y5), calculate face positive direction vector v1The formula of (x, y) is specific as follows:
Then, it is based on face positive direction vector v1(x, y) and 0 degree of direction vector v2(0,1) calculates face positive direction angle
θ2Formula it is specific as follows:
The various embodiments provided through this embodiment, firstly, according at least partly facial image and face key point
Detection model obtains the target face key point coordinate of at least partly facial image, the target face key point coordinate
Including target right and left eyes canthus coordinate and target nose coordinate；Then, according to target right and left eyes canthus coordinate, the target
Nose coordinate and face positive direction calculation formula obtain the face positive direction of at least partly facial image.It can be seen that examining
Consider the faces key point such as right and left eyes canthus and nose at least partly facial image to be first based on the correlation of face positive direction
The face key point coordinate for the face critical point detection model inspection at least partly facial image that deep learning obtains, then it is based on people
Face key point obtains accurately the face positive direction of at least partly facial image using face positive direction calculation formula, so as to
Above-described embodiment be based on the face positive direction adjust image to be displayed so that the picture material positive direction of image to be displayed with
User's face positive direction is consistent, improves the display correctness of image to be displayed, and the image to be displayed is watched and understood conducive to user,
Promote user experience.
Example devices
Referring to Fig. 8, a kind of structural schematic diagram for the device for adjusting image display direction in the embodiment of the present application is shown.?
In the present embodiment, described device for example be can specifically include:
First obtains unit 801, for obtaining image to be displayed and at least partly facial image；
Second obtaining unit 802, for obtaining the figure to be shown according to the image to be displayed and the first detection model
The picture material positive direction of picture；
Third obtaining unit 803, for according at least partly facial image and the second detection model, acquisition to be described extremely
The face positive direction of a minority's face image；
Adjustment unit 804, for described to be shown according to described image content positive direction and face positive direction adjustment
The display direction of image.
Optionally, first detection model is picture material positive direction detection model.
Optionally, described image content positive direction detection model is based on the first training set of images using densnet network
It is obtained with L2 loss function learning training, the first image training set includes multiple display images and corresponding picture material
Positive direction.
Optionally, second detection model is face positive direction detection model or face critical point detection model.
Optionally, the face positive direction detection model is based on the second training set of images using densnet network and L2
Loss function learning training obtains, and second training set of images includes multiple at least partly facial images and corresponding face
Positive direction；The face critical point detection model is based on third training set of images using 2 rank stacked-hourglass networks
It is obtained with L2 loss function learning training, the third training set of images includes multiple at least partly facial images and corresponding
Face key point coordinate.
Optionally, the face key point coordinate includes right and left eyes canthus coordinate and nose coordinate.
Optionally, when second detection model is the face positive direction detection model, the third obtaining unit
803 are specifically used for:
According at least partly facial image and the face positive direction detection model, directly obtain it is described at least partly
The face positive direction of facial image.
Optionally, when second detection model is the face key point coordinate measurement model, the third is obtained
Unit 803 includes that the first acquisition subelement and second obtain subelement；
Described first obtains subelement, is used for according at least partly facial image and face critical point detection model,
The target face key point coordinate of at least partly facial image is obtained, the target face key point coordinate includes a target left side
Right eye canthus coordinate and target nose coordinate；
It is described second obtain subelement, for according to target right and left eyes canthus coordinate, the target nose coordinate and
Face positive direction calculation formula obtains the face positive direction of at least partly facial image.
Optionally, the second acquisition subelement includes that the first acquisition module and second obtain module；
Described first obtains module, for obtaining in target right and left eyes canthus according to target right and left eyes canthus coordinate
Point coordinate；
Described second obtains module, for according to target right and left eyes canthus midpoint coordinates, the target nose coordinate
With the face positive direction calculation formula, the face positive direction of at least partly facial image is obtained.
Optionally, the adjustment unit 804 includes that third obtains subelement and rotation subelement；
The third obtains subelement, for being revolved according to described image content positive direction and the face positive direction
Gyration；
The rotation subelement, for rotating the image to be displayed according to the rotation angle.
The various embodiments provided through this embodiment, the device of adjustment image display direction include that the first acquisition is single
Member, the second obtaining unit, third obtaining unit and adjustment unit.Wherein, first obtains unit obtains image to be displayed and at least
Partial face image；Second obtaining unit obtains the image to be displayed according to the image to be displayed and the first detection model
Picture material positive direction；Third obtaining unit is according at least partly facial image and the second detection model, described in acquisition
At least partly face positive direction of facial image；Adjustment unit is according to described image content positive direction and the face positive direction tune
The display direction of the whole image to be displayed.It can be seen that, it is contemplated that the picture material positive direction problem of image to be displayed, logical
After crossing at least partly facial image judgement user's face positive direction, the first detection model obtained based on deep learning is detected to aobvious
The picture material positive direction of diagram picture, the display of image content-based positive direction and user's face positive direction adjustment image to be displayed
Image to be displayed is improved so that the picture material positive direction of image to be displayed is consistent with user's face positive direction in direction
It shows correctness, the image to be displayed is watched and understood conducive to user, promote user experience.
Separately the embodiment of the present application also provides a kind of terminal device, the terminal device includes processor and memory:
Said program code is transferred to the processor for storing program code by the memory；
The processor is used for the adjustment image according to instruction execution above-described embodiment in said program code and shows
The method in direction.
In addition, the embodiment of the present application also provides a kind of computer readable storage mediums, which is characterized in that the computer
Readable storage medium storing program for executing is for storing program code, and said program code is for executing adjustment image display side described in above-described embodiment
To method.
Each embodiment in this specification is described in a progressive manner, the highlights of each of the examples are with other
The difference of embodiment, the same or similar parts in each embodiment may refer to each other.For device disclosed in embodiment
For, since it is corresponded to the methods disclosed in the examples, so being described relatively simple, related place is said referring to method part
It is bright.
Professional further appreciates that, unit described in conjunction with the examples disclosed in the embodiments of the present disclosure
And algorithm steps, can be realized with electronic hardware, computer software, or a combination of the two, in order to clearly demonstrate hardware and
The interchangeability of software generally describes each exemplary composition and step according to function in the above description.These
Function is implemented in hardware or software actually, the specific application and design constraint depending on technical solution.Profession
Technical staff can use different methods to achieve the described function each specific application, but this realization is not answered
Think beyond scope of the present application.
It should be noted that, in this document, relational terms such as first and second and the like are used merely to a reality
Body or operation are distinguished with another entity or operation, are deposited without necessarily requiring or implying between these entities or operation
In any actual relationship or order or sequence.The terms "include", "comprise" or its any other variant are intended to non-row
His property includes, so that the process, method, article or equipment for including a series of elements not only includes those elements, and
And further include other elements that are not explicitly listed, or further include for this process, method, article or equipment institute it is intrinsic
Element.In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that including institute
State in the process, method, article or equipment of element that there is also other identical elements.
The above is only the preferred embodiment of the application, not makes any form of restriction to the application.Though
Right the application has been disclosed in a preferred embodiment above, however is not limited to the application.It is any to be familiar with those skilled in the art
Member, in the case where not departing from technical scheme ambit, all using the methods and technical content of the disclosure above to the application
Technical solution makes many possible changes and modifications or equivalent example modified to equivalent change.Therefore, it is all without departing from
The content of technical scheme, any simple modification made to the above embodiment of the technical spirit of foundation the application are equal
Variation and modification, still fall within technical scheme protection in the range of.
Claims (13)
1. a kind of method for adjusting image display direction characterized by comprising
Obtain image to be displayed and at least partly facial image；
According to the image to be displayed and the first detection model, the picture material positive direction of the image to be displayed is obtained；
According at least partly facial image and the second detection model, the face for obtaining at least partly facial image is square
To；
The display direction of the image to be displayed is adjusted according to described image content positive direction and the face positive direction.
2. the method according to claim 1, wherein first detection model is the detection of picture material positive direction
Model.
3. according to the method described in claim 2, it is characterized in that, described image content positive direction detection model is based on first
Training set of images is obtained using densnet network and L2 loss function learning training, and the first image training set includes more
A display image and corresponding picture material positive direction.
4. the method according to claim 1, wherein second detection model is face positive direction detection model
Or face critical point detection model.
5. according to the method described in claim 4, it is characterized in that, the face positive direction detection model is based on the second image
What training set was obtained using densnet network and L2 loss function learning training, second training set of images include it is multiple extremely
A minority's face image and corresponding face positive direction；The face critical point detection model is adopted based on third training set of images
It is obtained with 2 rank stacked-hourglass networks and L2 loss function learning training, the third training set of images includes more
A at least partly facial image and corresponding face key point coordinate.
6. according to the method described in claim 5, it is characterized in that, the face key point coordinate includes right and left eyes canthus coordinate
With nose coordinate.
7. according to the method described in claim 4, it is characterized in that, when second detection model is face positive direction inspection
When surveying model, at least partly facial image and the second detection model according to obtain at least partly facial image
Face positive direction, specifically:
According at least partly facial image and the face positive direction detection model, at least partly facial image is determined
Face positive direction.
8. according to the method described in claim 6, it is characterized in that, when second detection model is face key point seat
When marking detection model, at least partly facial image and the second detection model according to obtain at least partly face
The face positive direction of image, comprising:
According at least partly facial image and face critical point detection model, the mesh of at least partly facial image is obtained
Face key point coordinate is marked, the target face key point coordinate includes target right and left eyes canthus coordinate and target nose coordinate；
According to target right and left eyes canthus coordinate, the target nose coordinate and face positive direction calculation formula, determine described in
At least partly face positive direction of facial image.
9. according to the method described in claim 8, it is characterized in that, it is described according to target right and left eyes canthus coordinate, it is described
Target nose coordinate and face positive direction calculation formula obtain the face positive direction of at least partly facial image, comprising:
According to target right and left eyes canthus coordinate, target right and left eyes canthus midpoint coordinates is obtained；
According to target right and left eyes canthus midpoint coordinates, the target nose coordinate and the face positive direction calculation formula,
Obtain the face positive direction of at least partly facial image.
10. the method according to claim 1, wherein described according to described image content positive direction and the people
Face positive direction adjusts the display direction of the image to be displayed, comprising:
According to described image content positive direction and the face positive direction, rotation angle is obtained；
The image to be displayed is rotated according to the rotation angle.
11. a kind of device for adjusting image display direction characterized by comprising
First obtains unit, for obtaining image to be displayed and at least partly facial image；
Second obtaining unit, for obtaining the figure of the image to be displayed according to the image to be displayed and the first detection model
As content positive direction；
Third obtaining unit, for according at least partly facial image and the second detection model, acquisition to be described at least partly
The face positive direction of facial image；
Adjustment unit, for adjusting the aobvious of the image to be displayed according to described image content positive direction and the face positive direction
Show direction.
12. a kind of terminal device, which is characterized in that the terminal device includes processor and memory:
Said program code is transferred to the processor for storing program code by the memory；
The processor is used for according to any one of instruction execution claim 1-10 in the said program code adjustment image
The method of display direction.
13. a kind of computer readable storage medium, which is characterized in that the computer readable storage medium is for storing program generation
Code, the method that said program code requires any one of 1-10 adjustment image display direction for perform claim.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811455968.8A CN109558842A (en) | 2018-11-30 | 2018-11-30 | A kind of method, apparatus, equipment and medium adjusting image display direction |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811455968.8A CN109558842A (en) | 2018-11-30 | 2018-11-30 | A kind of method, apparatus, equipment and medium adjusting image display direction |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109558842A true CN109558842A (en) | 2019-04-02 |
Family
ID=65868206
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811455968.8A Pending CN109558842A (en) | 2018-11-30 | 2018-11-30 | A kind of method, apparatus, equipment and medium adjusting image display direction |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109558842A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109257489A (en) * | 2018-08-23 | 2019-01-22 | 维沃移动通信有限公司 | A kind of display methods and mobile terminal |
| CN110597391A (en) * | 2019-09-17 | 2019-12-20 | 腾讯科技（深圳）有限公司 | Display control method, display control device, computer equipment and storage medium |
| WO2022042111A1 (en) * | 2020-08-26 | 2022-03-03 | 深圳创维－Rgb电子有限公司 | Video image display method and apparatus, and multimedia device and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101639933A (en) * | 2009-07-16 | 2010-02-03 | 上海合合信息科技发展有限公司 | Image rotation correction method and system and electronic device |
| US20120081392A1 (en) * | 2010-09-30 | 2012-04-05 | Apple Inc. | Electronic device operation adjustment based on face detection |
| CN103049084A (en) * | 2012-12-18 | 2013-04-17 | 深圳国微技术有限公司 | Electronic device and method for adjusting display direction according to face direction |
| CN104506836A (en) * | 2014-11-28 | 2015-04-08 | 深圳市亿思达科技集团有限公司 | Personal holographic three-dimensional display method and device based on eyeball tracking |
| CN106325484A (en) * | 2015-06-30 | 2017-01-11 | 芋头科技（杭州）有限公司 | Face directional display system and method |
| CN106648344A (en) * | 2015-11-02 | 2017-05-10 | 重庆邮电大学 | Screen content adjustment method and equipment |
-
2018
- 2018-11-30 CN CN201811455968.8A patent/CN109558842A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101639933A (en) * | 2009-07-16 | 2010-02-03 | 上海合合信息科技发展有限公司 | Image rotation correction method and system and electronic device |
| US20120081392A1 (en) * | 2010-09-30 | 2012-04-05 | Apple Inc. | Electronic device operation adjustment based on face detection |
| CN103049084A (en) * | 2012-12-18 | 2013-04-17 | 深圳国微技术有限公司 | Electronic device and method for adjusting display direction according to face direction |
| CN104506836A (en) * | 2014-11-28 | 2015-04-08 | 深圳市亿思达科技集团有限公司 | Personal holographic three-dimensional display method and device based on eyeball tracking |
| CN106325484A (en) * | 2015-06-30 | 2017-01-11 | 芋头科技（杭州）有限公司 | Face directional display system and method |
| CN106648344A (en) * | 2015-11-02 | 2017-05-10 | 重庆邮电大学 | Screen content adjustment method and equipment |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109257489A (en) * | 2018-08-23 | 2019-01-22 | 维沃移动通信有限公司 | A kind of display methods and mobile terminal |
| CN110597391A (en) * | 2019-09-17 | 2019-12-20 | 腾讯科技（深圳）有限公司 | Display control method, display control device, computer equipment and storage medium |
| WO2022042111A1 (en) * | 2020-08-26 | 2022-03-03 | 深圳创维－Rgb电子有限公司 | Video image display method and apparatus, and multimedia device and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2021047396A1 (en) | Image processing method and apparatus, electronic device and computer-readable storage medium | |
| CN110766777B (en) | Method and device for generating virtual image, electronic equipment and storage medium | |
| US10533925B2 (en) | Fixtureless lensmeter and methods of operating same | |
| US10855909B2 (en) | Method and apparatus for obtaining binocular panoramic image, and storage medium | |
| CN109558842A (en) | A kind of method, apparatus, equipment and medium adjusting image display direction | |
| CN110443230A (en) | Face fusion method, apparatus and electronic equipment | |
| EP3992919B1 (en) | Three-dimensional facial model generation method and apparatus, device, and medium | |
| CN107079141A (en) | Image mosaic for 3 D video | |
| CN105763829A (en) | Image processing method and electronic device | |
| CN109345553A (en) | A kind of palm and its critical point detection method, apparatus and terminal device | |
| CN110288614A (en) | Image processing method, device, equipment and storage medium | |
| CN111163303B (en) | Image display method, device, terminal and storage medium | |
| CN106199066A (en) | The direction calibration method of intelligent terminal, device | |
| CN108596070A (en) | Character recognition method, device, storage medium, program product and electronic equipment | |
| CN112766215A (en) | Face fusion method and device, electronic equipment and storage medium | |
| CN110570383A (en) | image processing method and device, electronic equipment and storage medium | |
| CN108573192A (en) | Match the glasses try-on method and device of face | |
| TWI808321B (en) | Object transparency changing method for image display and document camera | |
| CN108510541A (en) | A kind of information adjusting method, electronic equipment and computer readable storage medium | |
| CN110363132A (en) | Biopsy method, device, electronic equipment and storage medium | |
| WO2022252871A1 (en) | Video generation method and apparatus, device, and storage medium | |
| CN113223128B (en) | Method and apparatus for generating image | |
| CN113326751B (en) | Hand 3D key point labeling method | |
| CN109598786A (en) | The simulation method of measuring instrument | |
| CN115550563A (en) | Video processing method, video processing device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190402 |
|
| RJ01 | Rejection of invention patent application after publication |