CN105763829A - Image processing method and electronic device - Google Patents
Image processing method and electronic device Download PDFInfo
- Publication number
- CN105763829A CN105763829A CN201410796708.2A CN201410796708A CN105763829A CN 105763829 A CN105763829 A CN 105763829A CN 201410796708 A CN201410796708 A CN 201410796708A CN 105763829 A CN105763829 A CN 105763829A
- Authority
- CN
- China
- Prior art keywords
- image
- face
- dimensional
- facial image
- reference picture
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Image Processing (AREA)
Abstract
The invention discloses an image processing method used for solving a technical problem that the video effect of an electronic device in the prior art is poor in effect and improves video effect and enhances user experience. The method includes collecting a first image containing a human face image of a current user through a collection unit, wherein the human eye part in the human face image is partly in a non-front-view state; processing the first image according to adjusting parameters and obtaining a second image corresponding to the first image and including a human face rectification image corresponding to the human face image, wherein the human eye part in the human face rectification image is partly in a front-view state; and sending the second image to the display unit for displaying. The invention also discloses an electronic device realizing the method.
Description
Technical field
The present invention relates to electronic technology field, particularly to a kind of image processing method and electronic equipment.
Background technology
Along with scientific and technological develops the fierceness day by day with the market competition rapidly, the performance of electronic equipment and outward appearance obtain and promote energetically, wherein notebook computer with its small volume and less weight, be easy to carry, the recreational advantage such as by force is just being subject to liking of increasing people, becomes an indispensable part in studying and living.User utilizes the thing that electronic equipment can do also to get more and more, as: user can use the panel computer with communication function to carry out voice/video communication, listens music, play game etc..
At present, the video communication software commonly used it is mounted in the electronic device, for instance Tencent QQ video, Microsoft Skype or Google Hangout.When user opens video communication software, displayed by the video image that camera collection internal or external in electronic equipment is real-time and can realize video communication.
But the photographic head owing to being used for gathering video image is generally disposed at above the display screen of notebook computer or mobile phone, or part electronic equipment is external separate camera, and in video communication, video window is typically in the zone line of display screen, and the video window that user's ordinary practice is watched attentively in display screen exchanges, therefore it cannot look at photographic head when seeing video window simultaneously.Therefore, when passing through to be positioned at the extra-regional camera collection video image of display screen and carry out video communication, because of the position difference between photographic head and video window, therefore the sight of user is in shift state more in the video image of finally display, it is unable to reach the visual effect of front exchange, video effect is poor, and user experience is relatively low.
Although problem above can be obtained a degree of improvement by part commercial video conference system (such as Cisco etc.), but its video equipment is expensive, and installation is complicated, is difficulty with the popularization in consumer electronics level field.
In summary, prior art exists the technical problem that electronic equipment video effect in video communication is poor.
Summary of the invention
The embodiment of the present invention provides a kind of image processing method and electronic equipment, for solving the technical problem that in prior art, the video effect of electronic equipment is poor, it is achieved that improve video effect, strengthens the effect of user experience.
On the one hand, disclosure one image processing method, it is applied in electronic equipment, described electronic equipment includes collecting unit and display unit, and the method comprises the following steps:
Comprised the first image of the facial image of active user by described collecting unit collection, and in described facial image, the human eye of user is partially in and non-faces state；
According to adjusting parameter, described first image is processed, it is thus achieved that the second corresponding with described first image, to comprise the face correcting image corresponding with described facial image image；Wherein, in described face correcting image, the human eye of user is partially in the state of facing；
Described second image is sent and displays to described display unit.
On the other hand, disclosure one electronic equipment, including:
First operation module, for being comprised the first image of the facial image of active user by described collecting unit collection, and in described facial image, the human eye of user is partially in and non-faces state；
Processing module, for processing described first image according to adjustment parameter, it is thus achieved that the second corresponding with described first image, to comprise the face correcting image corresponding with described facial image image；Wherein, in described face correcting image, the human eye of user is partially in the state of facing；
Sending module, displays to described display unit for being sent by described second image.
nullOwing to the first image is the image comprising facial image by collecting unit collection，And the human eye of user is partially in and non-faces state in this facial image，Therefore by adjusting parameter, the first image is processed，Such as，Can be adjusted people's eye portion of the face part in the first image processing，To obtain the human eye of user in and facial image corresponding with the first image being partially in the second image of the state of facing，Thus realizing in video communication，By the video image of collecting unit Real-time Collection is carried out according to adjust parameter process，Obtain the corresponding image being in the state of facing，After being adjustment with the image realizing showing in display unit、Visually there is the image to visual sense，Therefore solve in the video effect caused because of the gap between photographic head and video window in video communication without the technical problem to visual sense，Therefore enhance the video effect of video communication，Provide the user the experience of more naturally face-to-face exchange.
Additionally, due to the professional equipment of costliness need not be used or increases extra members, simply use can the photographic head of sampling depth image, therefore less costly, it is easy to promote.And in the process that the first image is processed, it is not necessary to user carries out the setting of additional complexity, therefore without changing the operating habit of original video chat so that user experience is higher.
Accompanying drawing explanation
Fig. 1 is the broad flow diagram of image processing method in the embodiment of the present invention；
Fig. 2 is the primary structure figure of electronic equipment in the embodiment of the present invention.
Detailed description of the invention
The embodiment of the present invention provides a kind of image processing method, it is applied in electronic equipment, described electronic equipment includes collecting unit and display unit, the method includes: comprised the first image of the facial image of active user by described collecting unit collection, and in described facial image, the human eye of user is partially in and non-faces state；According to adjusting parameter, described first image is processed, it is thus achieved that the second corresponding with described first image, to comprise the face correcting image corresponding with described facial image image；Wherein, in described face correcting image, the human eye of user is partially in the state of facing；Described second image is sent and displays to described display unit.
nullOwing to the first image is the image comprising facial image by collecting unit collection，And the human eye of user is partially in and non-faces state in this facial image，Therefore by adjusting parameter, the first image is processed，Such as，Can be adjusted people's eye portion of the face part in the first image processing，To obtain the human eye of user in and facial image corresponding with the first image being partially in the second image of the state of facing，Thus realizing in video communication，By the video image of collecting unit Real-time Collection is carried out according to adjust parameter process，Obtain the corresponding image being in the state of facing，After being adjustment with the image realizing showing in display unit、Visually there is the image to visual sense，Therefore solve in the video effect caused because of the gap between photographic head and video window in video communication without the technical problem to visual sense，Therefore enhance the video effect of video communication，Provide the user the experience of more naturally face-to-face exchange.
Additionally, due to the professional equipment of costliness need not be used or increases extra members, simply use can the photographic head of sampling depth image, therefore less costly, it is easy to promote.And in the process that the first image is processed, it is not necessary to user carries out the setting of additional complexity, therefore without changing the operating habit of original video chat so that user experience is higher.
For making the purpose of the embodiment of the present invention, technical scheme and advantage clearly, below in conjunction with the accompanying drawing in the embodiment of the present invention, technical scheme in the embodiment of the present invention is clearly and completely described, obviously, described embodiment is a part of embodiment of the present invention, rather than whole embodiments.Based on the embodiment in the present invention, the every other embodiment that those of ordinary skill in the art obtain under not making creative work premise, broadly fall into the scope of protection of the invention.
In the embodiment of the present invention, described electronic equipment can be the electronic equipments such as notebook, PAD (panel computer), mobile phone, the invention is not limited in this regard.
It addition, the terms "and/or", being only a kind of incidence relation describing affiliated partner, can there are three kinds of relations in expression, for instance, A and/or B, it is possible to represent: individualism A, there is A and B, individualism B these three situation simultaneously.It addition, character "/" herein, typically represent forward-backward correlation to as if the relation of a kind of "or".
Below in conjunction with accompanying drawing, the preferred embodiment of the present invention is described in detail.
Referring to Fig. 1, the embodiment of the present invention provides a kind of image processing method, is applied in electronic equipment, and described electronic equipment includes collecting unit and display unit, and the method may comprise steps of:
Step 11: comprised the first image of the facial image of active user by described collecting unit collection, and in described facial image, the human eye of user is partially in and non-faces state.
In the embodiment of the present invention, first image can be the video image being in video call process to be gathered by collecting unit at electronic equipment, this video image at least can include object video, i.e. user's face, in order to video both sides can observe the other side mutually.Wherein, video calling typically refers to based on the Internet and mobile Internet (3G the Internet), transmits the voice of people and a kind of communication mode of image in real time by video communication software between mobile phone or computer.
Additionally, people's eye portion of facial image that this first image comprises can be in non-facing state.As a rule, when collecting unit collection comprises the image of this user's face, if user does not look at collecting unit straight, then gathering in the facial image of gained, the human eye of user is it is possible to be in and non-face state.Concrete, when the human eye of user be partially in non-face state time, it is believed that what the human eye sight of people's eye portion mainly collected in collecting unit gathers the emission source place that light is corresponding.
Or, by the band of position at pupil place in detection people's eye portion, electronic equipment can judge whether facial image is the non-image facing state.Such as can detecting and judge whether pupil is in the predeterminable area in region shared by people's eye portion, this predeterminable area can be sum up out the human eye of user according to history test repeatedly to be in and face or the non-iris region facing state.As a rule, when human eye is partially in and faces state, pupil may be at the zone line of iris.
In reality, due to when the eyes of people are seen to different directions, eye has trickle change, these changes can produce the feature that can extract, computer can pass through picture catching or these features are extracted in scanning, thus the change of real-time tracing eyes, such that it is able to the state of prediction user and demand, and respond.Such as, captured the movable information of eyeball by multiple photographic head, calculate the position that human eye is paid close attention to, and then can be used for the purpose reaching to be controlled equipment by eyes, as user can stir the page without touching screen.
Therefore, the application can pass through dynamic follow the trail of (EyeTracking) of eye and judges in the first image gathered that human eye is partially in the state of facing or non-faces state.Concrete, eyeball tracking can pass through to study the acquisition of Eyeball motion information, modeling and simulation, come real-time tracing or the running status of prediction eyeball, namely realized oculomotor tracking by the position of point of fixation of measurement eyes or the motion of eyeball opposing headers.Therefore when gathering the first image, the position of people's eye portion point of fixation in the first image is may determine that by the tracing process that human eye is rotated, when the position of this point of fixation is positioned at the dead ahead of people's eye portion, it is believed that human eye is partially in the state of facing, otherwise face state for non-.
Wherein, obtaining the equipment of Eyeball motion information except infrared equipment, it is also possible to be image capture device, the photographic head on even general computer or mobile phone, it can also realize eye tracking under the support of software.
Optionally, in the embodiment of the present invention, if in video call process, the video image gathered by this collecting unit can be shown in display unit by electronic equipment in real time, and, also this video image can be shown in the opposite equip. corresponding to object video, in order to video both sides' prodigiosin watches identical video pictures.
In the embodiment of the present invention, this collecting unit is specifically as follows binocular camera, can be gathered the depth image of its acquisition range by this binocular camera.Such as, the first image of the facial image comprising active user is gathered.
Additionally, described collecting unit is except for except binocular camera, it is also possible to be that ToF (Time-of-Flight can survey depth camera) or SL (Structured-Light, structured light) images the first-class photographic head obtaining image depth information.Because ToF and SL both photographic head need projection near-infrared to complete depth sensing, so cannot use preferably in an outdoor environment, and it is usually used in the depth image collection of indoor environment.
Optionally, in the embodiment of the present invention, before or after passing through collection acquisition the first image, the method can also include: in preset duration, gathered people's eye portion by collecting unit respectively and watch the first reference picture of collecting unit attentively, and people's eye portion watches the second reference picture of video display window corresponding with current video call in display unit attentively；The adjustment parameter corresponding with facial image is determined according to the first reference picture and the second reference picture.
Wherein, gathering the first reference picture and the second reference picture, it is believed that be a calibration process, electronic equipment output class can be similar to the information that function is calibrated.Such as, when electronic equipment detects that user starts shooting or opens video capability, it is possible to output voice or Word message, prompting user notes collecting unit and video display window respectively, in order to gathers and obtains corresponding reference picture.
Certainly, when gathering reference picture by above step, it is possible to be that electronic equipment performs when starting video capability, or can also be the execution when detecting that user object change.Such as, the head portrait of the user object of history video can be stored by electronic equipment, when being again started up video capability, can by the recognition of face of active user is determined if with last time video user object identical, if differing, then export information, in order to obtain the reference picture for new user, so that it is determined that video process is suitable for the adjustment parameter of active user；If identical, then can directly invoke the adjustment parameter corresponding with this user by precalculating and storing.
Concrete, the first reference picture and the second reference picture that gather can be the reference pictures including facial image, owing to when gathering the first reference picture, user watches described collecting unit attentively, therefore it is believed that in the first reference picture the human eye of user be partially in the state of facing；And when gathering the second reference picture, due to position deviation between collecting unit and display unit in electronic equipment, therefore gathering in the image that user watches display unit attentively now by collecting unit, people's eye portion of user should be at non-front view picture.
Certainly, gathering the first reference picture and during the second reference picture, it is also possible to be move the method for tracking by eye to carry out the collection of image.Such as, when by the dynamic pursuitmeter of eye calculate human eye watch collecting unit attentively time, gather the first reference picture, and in people's eye fixation display unit during video display window, gather the second reference picture.
Wherein, this preset duration can as calibration duration, for instance 5s or 10s, namely, within this time period, it is possible to the state that detection user is current, namely whether have and watch collecting unit and video display window respectively attentively according to prompting, and determine be time, reference picture that can be current corresponding.Generally, this preset duration can pre-set according to user's use habit, it is also possible to is that electronic equipment dispatches from the factory and sets at once, the invention is not limited in this regard.
Further, determine that the detailed process of the adjustment parameter corresponding with facial image may is that detection obtains the three-dimensional feature point set one corresponding with face characteristic in described first reference picture and the three-dimensional feature point set two corresponding with face characteristic in the second reference picture according to reference picture；The correlation matrix between described three-dimensional feature point set one and described three-dimensional feature point set two is determined by singular value decomposition method；Using the spin matrix determined according to described correlation matrix and translation vector as the adjustment parameter corresponding with described facial image.
Due to can sampling depth image by collecting unit, therefore, three-dimensional feature point set can be determine according to depth image corresponding with face in reference picture, such as face three-dimensional point cloud, namely for the massive point set in the same space referential following table intelligent's face spatial distribution and face surface characteristic.In addition, because there is noise in the view data of binocular camera collection, therefore can also carry out such as pretreatment such as smooth, filling-up holes by raw image data before image is processed, it is simple to accurately determine the original visual of reference picture, obtain, to improve, the accuracy adjusting parameter.
In the embodiment of the present invention, it is determined that adjust in the process of parameter, specifically can have following steps:
First, open image according to the first reference picture and the second ginseng and obtain three-dimensional feature point set one and three-dimensional feature point set two, it is possible to respectively with { piAnd { pi' represent, wherein i=1,2 ..., N, wherein, N is positive integer.
Then, according to three-dimensional feature point set according to below equation estimation spin matrix and translation vector:
pi'=Rpi+T+NiFormula 1
In formula 1, NiFor noise vector, R represents the spin matrix of 3 × 3, and T represents the translation vector of 3 × 1.
Then, according to formula 1, by minimizing, it is possible to obtain:
Subsequently, according to formula 2 by the singular value decomposition of correlation matrix can be in the hope of correlation matrix H:
In formula 3,
Finally, according to correlation matrix H, it may be determined that the adjustment parameter corresponding with facial image, i.e. spin matrix R=UVT, and translation vector
Step 12: according to adjusting parameter, described first image is processed, it is thus achieved that the second corresponding with described first image, to comprise the face correcting image corresponding with described facial image image；Wherein, in described face correcting image, the human eye of user is partially in the state of facing.
In the embodiment of the present invention, after obtaining adjustment parameter, it is possible to the first image is carried out respective handling, this process may include that and obtains the depth information that described first image is corresponding, sets up the threedimensional model corresponding with described first image based on described collecting unit；According to the transformation matrix comprising described spin matrix and described translation vector, described threedimensional model is adjusted, it is thus achieved that second image corresponding with described first image.
Optionally, before the first image is processed, it is possible to its view data denoising, for instance the pretreatment such as smooth, filling-up hole.Owing to collecting unit is binocular camera, therefore after it gathers the first image, electronic equipment can obtain and the first corresponding depth image of image, in depth image, the value of each pixel can represent the distance value of distance photographic head, therefore face part in the first image can be calculated according to depth image and, relative to the three-dimensional coordinate of collecting unit (photographic head), namely set up the user's face three-dimensional coordinate relative to acquisition camera.
For example, if user's face point coordinate on depth map is (x1, y1), it is assumed that the focal length of photographic head is f, and resolution is that (w h), then can obtain this three-dimensional coordinate relative to photographic head, i.e. (X1, Y1, Z1), and:
In formula 4, D1For the distance value of this pixel distance photographic head in depth map.
Then by that analogy, it is possible to obtain in the first image, each of facial image is relative to the three-dimensional coordinate of collecting unit, therefore can obtain the threedimensional model corresponding with the first image.
Further, described according to the transformation matrix comprising described spin matrix and described translation vector, described threedimensional model is adjusted, obtain second image corresponding with described first image, specifically may include that and determine face three-dimensional point cloud corresponding with described facial image in described threedimensional model, and reference three-dimensional point cloud corresponding with user people's eye portion in described face three-dimensional point cloud；Based on described with reference to three-dimensional point cloud, described face three-dimensional point cloud is adjusted according to the transformation matrix comprising described spin matrix and described translation vector, it is thus achieved that the three-dimensional correction point cloud corresponding with described facial image；To map on two-dimensional coordinate system with described three-dimensional correction point cloud, it is thus achieved that the face correcting image corresponding with described facial image；Described second image corresponding with described first image is obtained according to described user's face correcting image.
Wherein, face three-dimensional point cloud can be the three-dimensional point set expressing face spatial distribution and face surface characteristic in three-dimensional coordinate, and according to face three-dimensional point cloud, it is also possible to determine the reference three-dimensional point cloud corresponding with user people's eye portion.And transformation matrix can be a correction matrix, this matrix includes the spin matrix R and the translation vector T that determine according to reference picture, and namely transformation matrix can be the matrix of 4 × 4.Such as, transformation matrix can use letter M to represent, then have
Therefore based on reference three-dimensional point cloud, the point in face three-dimensional point cloud can be adjusted according to this transformation matrix one by one, to obtain the three-dimensional correction point cloud corresponding with facial image.
For example, it is assumed that certain some three-dimensional coordinate before adjustment is (X in three-dimensional face point cloud0, Y0, Z0), then can calculate the three-dimensional coordinate after obtaining the adjustment that this point is corresponding according to transformation matrix, (X' can be represented0, Y0', Z'0), then have:
[X'0Y0'Z'01]=M × [X0Y0Z01] formula 5
According to formula 5, it is possible to obtain the three-dimensional coordinate after the adjustment that in three-dimensional face point cloud, each point is corresponding, namely can determine that the three-dimensional correction point cloud of correspondence.Hereafter, just can three-dimensional correction point cloud be mapped in two-dimensional coordinate system, to obtain the second image corresponding with the first image.
Continue above-mentioned example, obtain the three-dimensional coordinate after this point adjusts, i.e. (X'0, Y0', Z'0) after, it is assumed that this mapping point in two-dimensional coordinate system is (x0, y0), then have:
[x0y01]=P × [X'0Y0'Z'01] formula 6
In formula 6, P is the photographic head projection matrix of 3 × 4.
According to the mapping point that formula 6 obtains, it may be determined that with the face normalization image corresponding to the facial image before adjustment.Now, the people's eye portion in face correcting image can be in facing state.
Further, after obtaining face correcting image, this face correcting image can be replaced the facial image in the first image, to obtain described second image.
Concrete, when face correcting image being replaced the facial image in the first image, it is also possible to including:
First, face correcting image is carried out facial feature points detection, for instance the face of face and cheek characteristic point etc., and the characteristic point at cheek edge is carried out data scaling, simulate external ellipse, and intercept out the face replacement image-region for merging with this ellipse for border；
Then, in the first image, by demarcating corresponding feature point detection, generate face fusion border corresponding with face partitioning boundary；
Finally, based on people's eye portion and fusion border, face is replaced image co-registration to the first image, it is thus achieved that the second image, it is achieved the rectification of sight.
By face correcting image and the second image seamless are merged, it is possible to be effectively ensured in the second image of acquisition the integrity of image background, under the premise not destroying image background, namely complete sight correct, thus improving the visual experience of video user.And in the embodiment of the present invention, GPU (GraphicProcessingUnit be can be through for image processing process, graphic process unit) acceleration, the real-time process of video image can be ensured preferably, it is not take up extra cpu resource, improves the electronic equipment disposal ability to video image on the whole.
Step 13: described second image is sent and displays to described display unit.
A series of process by step firm 12 after obtaining the second image, can be displayed the second image by display unit.Such as, in video process, the video image after processing is shown in video display window.Herein, display unit may refer to electronic equipment local terminal, it is also possible to refers to the display unit of the opposite equip. communicated with electronic equipment.Such as, when carrying out video conversation, video user can see the video pictures of the other side by display screen.
It should be noted that, the position deviation being mainly based upon between collecting unit and display unit owing to adjusting parameter is determined, the present invention can be as reference standard using this adjustment parameter, video capture image is processed, therefore in most of the cases, according in the second image after this adjustment parameter processing, people's eye portion should be and is in the state of facing.But for some special circumstances, such as active user does not inherently see video display window, such as side face or bow, in the second image after then processing, people's eye portion of user is still likely to be at non-state of facing, but the face part that the face normalization image section in the second image presents has certain new vision relative to the face part of the first image.
Embodiment two
Referring to Fig. 2, based on same idea, the present invention also provides for a kind of electronic equipment, comprising: the first operation module 201, processing module 202 and sending module 203.
First operation module 201 may be used for the first image of the facial image comprising active user by described collecting unit collection, and in described facial image, the human eye of user is partially in and non-faces state.
Processing module 202 may be used for according to adjustment parameter, described first image being processed, it is thus achieved that the second corresponding with described first image, to comprise the face correcting image corresponding with described facial image image；Wherein, in described face correcting image, the human eye of user is partially in the state of facing.
Sending module 203 may be used for sending described second image displaying to described display unit.
Optionally, in the embodiment of the present invention, described electronic equipment also includes:
Second operation module, for in preset duration, gather described people's eye portion respectively by described collecting unit and watch the first reference picture of described collecting unit attentively, and described people's eye portion watches the second reference picture of video display window corresponding with current video call in described display unit attentively；
Determine module, for determining the adjustment parameter corresponding with described facial image according to described first reference picture with described second reference picture.
Optionally, in the embodiment of the present invention, described determine module specifically for:
Detection obtains the three-dimensional feature point set one corresponding with face characteristic in described first reference picture and the three-dimensional feature point set two corresponding with face characteristic in the second reference picture；
The correlation matrix between described three-dimensional feature point set one and described three-dimensional feature point set two is determined by singular value decomposition method；
Using the spin matrix determined according to described correlation matrix and translation vector as the adjustment parameter corresponding with described facial image.
Optionally, in the embodiment of the present invention, described processing module 202 specifically may be used for:
Obtain the depth information that described first image is corresponding, set up the threedimensional model corresponding with described first image based on described collecting unit；
According to the transformation matrix comprising described spin matrix and described translation vector, described threedimensional model is adjusted, it is thus achieved that second image corresponding with described first image.
Optionally, in the embodiment of the present invention, described processing module 202 specifically can be also used for:
Determine face three-dimensional point cloud corresponding with described facial image in described threedimensional model and reference three-dimensional point cloud corresponding with user people's eye portion in described face three-dimensional point cloud；
Based on described with reference to three-dimensional point cloud, described face three-dimensional point cloud is adjusted according to the transformation matrix comprising described spin matrix and described translation vector, it is thus achieved that the three-dimensional correction point cloud corresponding with described facial image；
To map on two-dimensional coordinate system with described three-dimensional correction point cloud, it is thus achieved that the face correcting image corresponding with described facial image；
Described second image corresponding with described first image is obtained according to described face correcting image.
Optionally, in the embodiment of the present invention, described processing module 202 specifically can be also used for:
Described face correcting image is made to replace the facial image in described first image, it is thus achieved that described second image.
Optionally, in the embodiment of the present invention, described collecting unit is binocular camera.
The embodiment of the present invention provides a kind of image processing method, it is applied in electronic equipment, described electronic equipment includes collecting unit and display unit, the method includes: comprised the first image of the facial image of active user by described collecting unit collection, and in described facial image, the human eye of user is partially in and non-faces state；According to adjusting parameter, described first image is processed, it is thus achieved that the second corresponding with described first image, to comprise the face correcting image corresponding with described facial image image；Wherein, in described face correcting image, the human eye of user is partially in the state of facing；Described second image is sent and displays to described display unit.
nullOwing to the first image is the image comprising facial image by collecting unit collection，And the human eye of user is partially in and non-faces state in this facial image，Therefore by adjusting parameter, the first image is processed，Such as，Can be adjusted people's eye portion of the face part in the first image processing，To obtain the human eye of user in and facial image corresponding with the first image being partially in the second image of the state of facing，Thus realizing in video communication，By the video image of collecting unit Real-time Collection is carried out according to adjust parameter process，Obtain the corresponding image being in the state of facing，After being adjustment with the image realizing showing in display unit、Visually there is the image to visual sense，Therefore solve in the video effect caused because of the gap between photographic head and video window in video communication without the technical problem to visual sense，Therefore enhance the video effect of video communication，Provide the user the experience of more naturally face-to-face exchange.
Additionally, due to the professional equipment of costliness need not be used or increases extra members, simply use can the photographic head of sampling depth image, therefore less costly, it is easy to promote.And in the process that the first image is processed, it is not necessary to user carries out the setting of additional complexity, therefore without changing the operating habit of original video chat so that user experience is higher.
Specifically, the computer program instructions that information processing method in the embodiment of the present application is corresponding can be stored in CD, hard disk, on the storage mediums such as USB flash disk, when the computer program instructions corresponding with image processing method in storage medium is read by an electronic equipment or be performed, comprise the steps:
Comprised the first image of the facial image of active user by described collecting unit collection, and in described facial image, the human eye of user is partially in and non-faces state；
According to adjusting parameter, described first image is processed, it is thus achieved that the second corresponding with described first image, to comprise the face correcting image corresponding with described facial image image；Wherein, in described face correcting image, the human eye of user is partially in the state of facing；
Described second image is sent and displays to described display unit.
Optionally, in described storage medium, also storage has other computer instruction, these computer instructions with step: comprised the first image of the facial image of active user by described collecting unit collection, and in described facial image, the human eye of user is partially in non-computer instruction corresponding to state of facing and is performed before or is performed afterwards, comprises the steps: during execution
In preset duration, gather described people's eye portion respectively by described collecting unit and watch the first reference picture of described collecting unit attentively, and described people's eye portion watches the second reference picture of video display window corresponding with current video call in described display unit attentively；
The adjustment parameter corresponding with described facial image is determined with described second reference picture according to described first reference picture.
Optionally, in described storage medium storage with step: determine that computer instruction corresponding to the adjustment parameter corresponding with described facial image is being specifically performed in process, comprises the steps: according to described first reference picture and described second reference picture
Detection obtains the three-dimensional feature point set one corresponding with face characteristic in described first reference picture and the three-dimensional feature point set two corresponding with face characteristic in the second reference picture；
The correlation matrix between described three-dimensional feature point set one and described three-dimensional feature point set two is determined by singular value decomposition method；
Using the spin matrix determined according to described correlation matrix and translation vector as the adjustment parameter corresponding with described facial image.
Optionally, in described storage medium storage and step: according to adjust parameter described first image is processed, obtain computer instruction corresponding to the second corresponding with described first image, to comprise the face correcting image corresponding with described facial image image being specifically performed in process, comprise the steps:
Obtain the depth information that described first image is corresponding, set up the threedimensional model corresponding with described first image based on described collecting unit；
According to the transformation matrix comprising described spin matrix and described translation vector, described threedimensional model is adjusted, it is thus achieved that second image corresponding with described first image.
Optionally, in described storage medium storage and step: according to the transformation matrix comprising described spin matrix and described translation vector, described threedimensional model is adjusted, obtain computer instruction corresponding to second image corresponding with described first image being specifically performed in process, comprise the steps:
Determine face three-dimensional point cloud corresponding with described facial image in described threedimensional model and reference three-dimensional point cloud corresponding with user people's eye portion in described face three-dimensional point cloud；
Based on described with reference to three-dimensional point cloud, described face three-dimensional point cloud is adjusted according to the transformation matrix comprising described spin matrix and described translation vector, it is thus achieved that the three-dimensional correction point cloud corresponding with described facial image；
To map on two-dimensional coordinate system with described three-dimensional correction point cloud, it is thus achieved that the face correcting image corresponding with described facial image；
Described second image corresponding with described first image is obtained according to described face correcting image.
Optionally, in described storage medium storage with step: obtain computer instruction corresponding to described second image being specifically performed in process according to described face correcting image, comprise the steps:
Described face correcting image is made to replace the facial image in described first image, it is thus achieved that described second image.
Obviously, the present invention can be carried out various change and modification without deviating from the spirit and scope of the present invention by those skilled in the art.So, if these amendments of the present invention and modification belong within the scope of the claims in the present invention and equivalent technologies thereof, then the present invention is also intended to comprise these change and modification.
Claims (14)
1. an image processing method, is applied in electronic equipment, and described electronic equipment includes collecting unit and display unit, and described method includes:
Comprised the first image of the facial image of active user by described collecting unit collection, and in described facial image, the human eye of user is partially in and non-faces state；
According to adjusting parameter, described first image is processed, it is thus achieved that the second corresponding with described first image, to comprise the face correcting image corresponding with described facial image image；Wherein, in described face correcting image, the human eye of user is partially in the state of facing；
Described second image is sent and displays to described display unit.
2. the method for claim 1, it is characterized in that, at the first image of the facial image being comprised active user by described collecting unit collection, and in described facial image the human eye of user be partially in non-face state before or after, described method also includes:
In preset duration, gather described people's eye portion respectively by described collecting unit and watch the first reference picture of described collecting unit attentively, and described people's eye portion watches the second reference picture of video display window corresponding with current video call in described display unit attentively；
The adjustment parameter corresponding with described facial image is determined with described second reference picture according to described first reference picture.
3. method as claimed in claim 2, it is characterised in that determine the adjustment parameter corresponding with described facial image according to described first reference picture with described second reference picture, specifically include:
Detection obtains the three-dimensional feature point set one corresponding with face characteristic in described first reference picture and the three-dimensional feature point set two corresponding with face characteristic in the second reference picture；
The correlation matrix between described three-dimensional feature point set one and described three-dimensional feature point set two is determined by singular value decomposition method；
Using the spin matrix determined according to described correlation matrix and translation vector as the adjustment parameter corresponding with described facial image.
4. method as claimed in claim 3, it is characterised in that according to adjusting parameter, described first image is processed, it is thus achieved that the second corresponding with described first image, to comprise the face correcting image corresponding with described facial image image, specifically include:
Obtain the depth information that described first image is corresponding, set up the threedimensional model corresponding with described first image based on described collecting unit；
According to the transformation matrix comprising described spin matrix and described translation vector, described threedimensional model is adjusted, it is thus achieved that second image corresponding with described first image.
5. method as claimed in claim 4, it is characterised in that described be adjusted by described threedimensional model according to the transformation matrix comprising described spin matrix and described translation vector, it is thus achieved that second image corresponding with described first image, specifically includes:
Determine face three-dimensional point cloud corresponding with described facial image in described threedimensional model and reference three-dimensional point cloud corresponding with user people's eye portion in described face three-dimensional point cloud；
Based on described with reference to three-dimensional point cloud, described face three-dimensional point cloud is adjusted according to the transformation matrix comprising described spin matrix and described translation vector, it is thus achieved that the three-dimensional correction point cloud corresponding with described facial image；
To map on two-dimensional coordinate system with described three-dimensional correction point cloud, it is thus achieved that the face correcting image corresponding with described facial image；
Described second image corresponding with described first image is obtained according to described face correcting image.
6. method as claimed in claim 5, it is characterised in that described according to described face correcting image described second image of acquisition, including:
Described face correcting image is made to replace the facial image in described first image, it is thus achieved that described second image.
7. the method as described in claim as arbitrary in claim 1-6, it is characterised in that described collecting unit is binocular camera.
8. an electronic equipment, including:
First operation module, for being comprised the first image of the facial image of active user by described collecting unit collection, and in described facial image, the human eye of user is partially in and non-faces state；
Processing module, for processing described first image according to adjustment parameter, it is thus achieved that the second corresponding with described first image, to comprise the face correcting image corresponding with described facial image image；Wherein, in described face correcting image, the human eye of user is partially in the state of facing；
Sending module, displays to described display unit for being sent by described second image.
9. electronic equipment as claimed in claim 8, it is characterised in that described electronic equipment also includes:
Second operation module, for in preset duration, gather described people's eye portion respectively by described collecting unit and watch the first reference picture of described collecting unit attentively, and described people's eye portion watches the second reference picture of video display window corresponding with current video call in described display unit attentively；
Determine module, for determining the adjustment parameter corresponding with described facial image according to described first reference picture with described second reference picture.
10. electronic equipment as claimed in claim 9, it is characterised in that described determine module specifically for:
Detection obtains the three-dimensional feature point set one corresponding with face characteristic in described first reference picture and the three-dimensional feature point set two corresponding with face characteristic in the second reference picture；
The correlation matrix between described three-dimensional feature point set one and described three-dimensional feature point set two is determined by singular value decomposition method；
Using the spin matrix determined according to described correlation matrix and translation vector as the adjustment parameter corresponding with described facial image.
11. electronic equipment as claimed in claim 10, it is characterised in that described processing module specifically for:
Obtain the depth information that described first image is corresponding, set up the threedimensional model corresponding with described first image based on described collecting unit；
According to the transformation matrix comprising described spin matrix and described translation vector, described threedimensional model is adjusted, it is thus achieved that second image corresponding with described first image.
12. electronic equipment as claimed in claim 11, it is characterised in that described processing module is specifically additionally operable to:
Determine face three-dimensional point cloud corresponding with described facial image in described threedimensional model and reference three-dimensional point cloud corresponding with user people's eye portion in described face three-dimensional point cloud；
Based on described with reference to three-dimensional point cloud, described face three-dimensional point cloud is adjusted according to the transformation matrix comprising described spin matrix and described translation vector, it is thus achieved that the three-dimensional correction point cloud corresponding with described facial image；
To map on two-dimensional coordinate system with described three-dimensional correction point cloud, it is thus achieved that the face correcting image corresponding with described facial image；
Described second image corresponding with described first image is obtained according to described face correcting image.
13. electronic equipment as claimed in claim 12, it is characterised in that described processing module is specifically additionally operable to:
Described face correcting image is made to replace the facial image in described first image, it is thus achieved that described second image.
14. the electronic equipment as described in claim as arbitrary in claim 8-13, it is characterised in that described collecting unit is binocular camera.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410796708.2A CN105763829A (en) | 2014-12-18 | 2014-12-18 | Image processing method and electronic device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410796708.2A CN105763829A (en) | 2014-12-18 | 2014-12-18 | Image processing method and electronic device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN105763829A true CN105763829A (en) | 2016-07-13 |
Family
ID=56340856
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410796708.2A Pending CN105763829A (en) | 2014-12-18 | 2014-12-18 | Image processing method and electronic device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105763829A (en) |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106960448A (en) * | 2017-04-19 | 2017-07-18 | 浙江大学 | A kind of device and method for carrying out quickly positioning and following the trail of to object in video |
| CN106981078A (en) * | 2017-04-14 | 2017-07-25 | 广州视源电子科技股份有限公司 | Sight bearing calibration, device, intelligent meeting terminal and storage medium |
| CN107622248A (en) * | 2017-09-27 | 2018-01-23 | 威盛电子股份有限公司 | One kind watches identification and interactive approach and device attentively |
| CN108965767A (en) * | 2018-07-26 | 2018-12-07 | 吴铁 | A kind of method for processing video frequency and system improving the experience of person to person's video interactive |
| CN109285135A (en) * | 2018-12-04 | 2019-01-29 | 厦门美图之家科技有限公司 | Face image processing process and device |
| WO2019041651A1 (en) * | 2017-08-30 | 2019-03-07 | 广州视源电子科技股份有限公司 | Image correction method, apparatus and device, and computer readable storage medium |
| CN109697749A (en) * | 2017-10-20 | 2019-04-30 | 虹软科技股份有限公司 | A kind of method and apparatus for three-dimensional modeling |
| CN109716781A (en) * | 2016-07-18 | 2019-05-03 | 斯纳普公司 | The real-time rendering of video flowing |
| CN110544317A (en) * | 2019-08-29 | 2019-12-06 | 联想(北京)有限公司 | Image processing method, image processing device, electronic equipment and readable storage medium |
| CN111737842A (en) * | 2019-03-19 | 2020-10-02 | 新奥数能科技有限公司 | Method and device for optimizing emission of nitrogen oxide of combustor |
| CN112085647A (en) * | 2019-06-14 | 2020-12-15 | 华为技术有限公司 | Face correction method and electronic equipment |
| WO2021238351A1 (en) * | 2020-05-29 | 2021-12-02 | 华为技术有限公司 | Image correction method and electronic apparatus |
| WO2022261856A1 (en) * | 2021-06-16 | 2022-12-22 | 华为技术有限公司 | Image processing method and apparatus, and storage medium |
| WO2024099319A1 (en) * | 2022-11-11 | 2024-05-16 | 广州视源电子科技股份有限公司 | Virtual video image generation method and apparatus, and device and medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080037836A1 (en) * | 2006-08-09 | 2008-02-14 | Arcsoft, Inc. | Method for driving virtual facial expressions by automatically detecting facial expressions of a face image |
| CN101593365A (en) * | 2009-06-19 | 2009-12-02 | 电子科技大学 | A kind of method of adjustment of universal three-dimensional human face model |
| CN103310186A (en) * | 2012-02-29 | 2013-09-18 | 三星电子株式会社 | Method for correcting user's gaze direction in image, machine-readable storage medium and communication terminal |
| CN103345619A (en) * | 2013-06-26 | 2013-10-09 | 上海永畅信息科技有限公司 | Self-adaption correcting method of human eye natural contact in video chat |
-
2014
- 2014-12-18 CN CN201410796708.2A patent/CN105763829A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080037836A1 (en) * | 2006-08-09 | 2008-02-14 | Arcsoft, Inc. | Method for driving virtual facial expressions by automatically detecting facial expressions of a face image |
| CN101593365A (en) * | 2009-06-19 | 2009-12-02 | 电子科技大学 | A kind of method of adjustment of universal three-dimensional human face model |
| CN103310186A (en) * | 2012-02-29 | 2013-09-18 | 三星电子株式会社 | Method for correcting user's gaze direction in image, machine-readable storage medium and communication terminal |
| CN103345619A (en) * | 2013-06-26 | 2013-10-09 | 上海永畅信息科技有限公司 | Self-adaption correcting method of human eye natural contact in video chat |
Cited By (23)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109716781B (en) * | 2016-07-18 | 2021-12-21 | 斯纳普公司 | System, apparatus, medium, and method for real-time rendering of video streams |
| CN109716781A (en) * | 2016-07-18 | 2019-05-03 | 斯纳普公司 | The real-time rendering of video flowing |
| US11750770B2 (en) | 2016-07-18 | 2023-09-05 | Snap Inc. | Real time painting of a video stream |
| US11212482B2 (en) | 2016-07-18 | 2021-12-28 | Snap Inc. | Real time painting of a video stream |
| CN106981078A (en) * | 2017-04-14 | 2017-07-25 | 广州视源电子科技股份有限公司 | Sight bearing calibration, device, intelligent meeting terminal and storage medium |
| WO2018188277A1 (en) * | 2017-04-14 | 2018-10-18 | 广州视源电子科技股份有限公司 | Sight correction method and device, intelligent conference terminal and storage medium |
| CN106981078B (en) * | 2017-04-14 | 2019-12-31 | 广州视源电子科技股份有限公司 | Sight line correction method and device, intelligent conference terminal and storage medium |
| CN106960448A (en) * | 2017-04-19 | 2017-07-18 | 浙江大学 | A kind of device and method for carrying out quickly positioning and following the trail of to object in video |
| WO2019041651A1 (en) * | 2017-08-30 | 2019-03-07 | 广州视源电子科技股份有限公司 | Image correction method, apparatus and device, and computer readable storage medium |
| CN107622248A (en) * | 2017-09-27 | 2018-01-23 | 威盛电子股份有限公司 | One kind watches identification and interactive approach and device attentively |
| CN109697749A (en) * | 2017-10-20 | 2019-04-30 | 虹软科技股份有限公司 | A kind of method and apparatus for three-dimensional modeling |
| CN108965767A (en) * | 2018-07-26 | 2018-12-07 | 吴铁 | A kind of method for processing video frequency and system improving the experience of person to person's video interactive |
| CN109285135A (en) * | 2018-12-04 | 2019-01-29 | 厦门美图之家科技有限公司 | Face image processing process and device |
| CN111737842A (en) * | 2019-03-19 | 2020-10-02 | 新奥数能科技有限公司 | Method and device for optimizing emission of nitrogen oxide of combustor |
| WO2020249076A1 (en) * | 2019-06-14 | 2020-12-17 | 华为技术有限公司 | Face calibration method and electronic device |
| CN112085647A (en) * | 2019-06-14 | 2020-12-15 | 华为技术有限公司 | Face correction method and electronic equipment |
| CN112085647B (en) * | 2019-06-14 | 2024-01-19 | 华为技术有限公司 | Face correction method and electronic equipment |
| CN110544317A (en) * | 2019-08-29 | 2019-12-06 | 联想(北京)有限公司 | Image processing method, image processing device, electronic equipment and readable storage medium |
| WO2021238351A1 (en) * | 2020-05-29 | 2021-12-02 | 华为技术有限公司 | Image correction method and electronic apparatus |
| CN113741681A (en) * | 2020-05-29 | 2021-12-03 | 华为技术有限公司 | Image correction method and electronic equipment |
| CN113741681B (en) * | 2020-05-29 | 2024-04-26 | 华为技术有限公司 | Image correction method and electronic equipment |
| WO2022261856A1 (en) * | 2021-06-16 | 2022-12-22 | 华为技术有限公司 | Image processing method and apparatus, and storage medium |
| WO2024099319A1 (en) * | 2022-11-11 | 2024-05-16 | 广州视源电子科技股份有限公司 | Virtual video image generation method and apparatus, and device and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105763829A (en) | Image processing method and electronic device | |
| US11632537B2 (en) | Method and apparatus for obtaining binocular panoramic image, and storage medium | |
| US10297076B2 (en) | Apparatus and method for generating 3D face model using mobile device | |
| CN103207664B (en) | A kind of image processing method and equipment | |
| Upenik et al. | A simple method to obtain visual attention data in head mounted virtual reality | |
| CN105787884A (en) | Image processing method and electronic device | |
| CN113168231A (en) | Enhanced techniques for tracking movement of real world objects to improve virtual object positioning | |
| CN101180653A (en) | Method and device for three-dimensional rendering | |
| CN108596106B (en) | Visual fatigue recognition method and device based on VR equipment and VR equipment | |
| US9001115B2 (en) | System and method for three-dimensional visualization of geographical data | |
| WO2020020022A1 (en) | Method for visual recognition and system thereof | |
| WO2019062056A1 (en) | Smart projection method and system, and smart terminal | |
| CN109982036A (en) | A kind of method, terminal and the storage medium of panoramic video data processing | |
| CN109002248B (en) | VR scene screenshot method, equipment and storage medium | |
| US10885651B2 (en) | Information processing method, wearable electronic device, and processing apparatus and system | |
| CN108124150A (en) | Virtual reality wears display device and observes the method for real scene by it | |
| US20190281280A1 (en) | Parallax Display using Head-Tracking and Light-Field Display | |
| CN106851240A (en) | The method and device of image real time transfer | |
| CN105812709A (en) | Method for realizing virtual camera by using cameras | |
| CN117333644A (en) | Virtual reality display picture generation method, device, equipment and medium | |
| CN107659772A (en) | 3D rendering generation method, device and electronic equipment | |
| KR101820503B1 (en) | Service systembased on face recognition inference, and face recognition inference method and storage medium thereof | |
| CN110675413A (en) | Three-dimensional face model construction method and device, computer equipment and storage medium | |
| CN114895790A (en) | Man-machine interaction method and device, electronic equipment and storage medium | |
| CN111967436B (en) | Image processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20160713 |