CN108174141A - A kind of method of video communication and a kind of mobile device - Google Patents

A kind of method of video communication and a kind of mobile device Download PDF

Info

Publication number
CN108174141A
CN108174141A CN201711241079.7A CN201711241079A CN108174141A CN 108174141 A CN108174141 A CN 108174141A CN 201711241079 A CN201711241079 A CN 201711241079A CN 108174141 A CN108174141 A CN 108174141A
Authority
CN
China
Prior art keywords
video communication
image
end subscriber
head portrait
opposite equip
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201711241079.7A
Other languages
Chinese (zh)
Other versions
CN108174141B (en
Inventor
张恒莉
金鑫
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Vivo Mobile Communication Co Ltd
Original Assignee
Vivo Mobile Communication Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Vivo Mobile Communication Co Ltd filed Critical Vivo Mobile Communication Co Ltd
Priority to CN201711241079.7A priority Critical patent/CN108174141B/en
Publication of CN108174141A publication Critical patent/CN108174141A/en
Application granted granted Critical
Publication of CN108174141B publication Critical patent/CN108174141B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/14—Systems for two-way working
    • H04N7/141—Systems for two-way working between two video terminals, e.g. videophone
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/172—Classification, e.g. identification

Abstract

An embodiment of the present invention provides the method and mobile device of a kind of video communication, the method includes：When local device opens video communication, local device acquires the first video communication image；First video communication image includes the current face image of this end subscriber；Local device generates confirmation request for the first video communication image, and confirmation request is to confirm whether opposite equip. is stored with the request of this end subscriber head portrait modeling data；User's head portrait model database includes the original facial image of user and first facial characteristic value；The current face image and confirmation request of this end subscriber are sent to opposite equip. by local device；Confirm for first as a result, local device extracts the second facial characteristics value of this end subscriber from the current face image of this end subscriber；Second facial characteristics value is sent to opposite equip. by local device.The embodiment of the present invention can or network bandwidth bad in network quality it is inadequate when, can still carry out the video communication of high quality.

Description

A kind of method of video communication and a kind of mobile device
Technical field
The present invention relates to technical field of electronic communication, method and a kind of video communication more particularly to a kind of video communication Mobile device.
Background technology
The development of science and technology brings more and more facilities to people’s lives.For example, it can only be face to be met from preceding people The meeting on opposite, but people are by intelligent terminal, such as mobile phone, computer now, with regard to remote video communication can be carried out, even if not It is that " can also meet " face-to-face.
But video communication is needed by network, in general, network quality is better, and the effect of video communication is also better, The video image that user receives is more clear.
In practical applications, when user needs to carry out video communication using intelligent terminal, it is bad to unintentionally encounter network quality, Or network bandwidth is inadequate, can not just carry out the video communication of high quality at this time.
Invention content
In view of the above problems, the embodiment of the present invention proposes a kind of method of video communication and a kind of corresponding video communication Mobile device, with solve the problems, such as that network quality is bad or network bandwidth it is insufficient caused by video communication quality it is poor.
To solve the above-mentioned problems, the embodiment of the invention discloses a kind of method of video communication, applied to mobile device, The mobile device includes local device and opposite equip., and the method is applied between local device and opposite equip.；It is described Method include：
When the local device opens video communication, the local device acquires the first video communication image；Described One video communication image includes the current face image of this end subscriber；
The local device generates confirmation request for first video communication image, and the confirmation request is confirms institute State the request whether opposite equip. is stored with this end subscriber head portrait modeling data；User's head portrait model database includes user Original facial image and first facial characteristic value；
The current face image of described end subscriber and confirmation request are sent to the opposite equip. by the local device； The opposite equip., which is used to return for the confirmation request, to be confirmed as a result, the confirmation result includes being stored with this end subscriber head First as modeling data confirms result；
Confirmed for described first as a result, the local device extracts this from the current face image of described end subscriber Second facial characteristics value of end subscriber；
The second facial characteristics value is sent to the opposite equip. by the local device；The opposite equip. is used to adopt The first facial characteristic value is updated with the second facial characteristics value, and with reference to the original facial image of described end subscriber, It generates and plays the second video communication image.
The embodiment of the invention also discloses a kind of methods of video communication, applied to mobile device, the mobile device packet Local device and opposite equip. are included, the method is applied between local device and opposite equip.；The method includes：
When the opposite equip. opens video communication state, the first video communication image that the local terminal is sent is received, And whether the opposite equip. is stored with the confirmation request of this end subscriber head portrait modeling data；User's head portrait modeling data Original facial image and first facial characteristic value including this end subscriber；
The opposite equip. is matched based on the confirmation request in user's head portrait model database；It is described to use account As model database includes user's head portrait modeling data；
If successful match, the opposite equip. is returned to the local device to be confirmed as a result, the confirmation result includes Be stored with this end subscriber head portrait modeling data first confirms result；
Receive the second facial characteristics value that the local device is sent；
The first facial characteristic value is updated using the second facial characteristics value, and with reference to the original of described end subscriber Face-image generates and plays the second video communication image.
Correspondingly, the embodiment of the invention discloses a kind of mobile device of video communication, the mobile device includes local terminal Equipment and opposite equip., the video communication applications are between local device and opposite equip.；The mobile device includes：
Acquisition module, for when the local device opens video communication, acquiring the first video communication image；Described One video communication image includes the current face image of this end subscriber；
Confirmation request generation module, for being directed to the first video communication image generation confirmation request, the confirmation please It asks to confirm whether the opposite equip. is stored with the request of this end subscriber head portrait modeling data；User's head portrait modeling data Library includes the original facial image of user and first facial characteristic value；
First sending module, for the current face image of described end subscriber and confirmation request to be sent to the opposite end Equipment；The opposite equip., which is used to return for the confirmation request, to be confirmed as a result, the confirmation result includes being stored with local terminal The first of user's head portrait modeling data confirms result；
First extraction module is confirmed for being directed to described first as a result, from the current face image of described end subscriber Extract the second facial characteristics value of this end subscriber；
Second sending module, for the second facial characteristics value to be sent to the opposite equip.；The opposite equip. For updating the first facial characteristic value using the second facial characteristics value, and with reference to the original facial of described end subscriber Image generates and plays the second video communication image.
Correspondingly, the embodiment of the invention also discloses a kind of mobile device of video communication, the mobile device includes this End equipment and opposite equip., the video communication applications are between local device and opposite equip.；The mobile device includes：
First receiving module, for when the opposite equip. opens video communication state, receiving the local device hair The confirmation whether the first video communication image and the opposite equip. sent is stored with this end subscriber head portrait modeling data please It asks；User's head portrait modeling data includes the original facial image of this end subscriber and first facial characteristic value；
Matching module is matched for being based on the confirmation request in user's head portrait model database；The user Head portrait model database includes user's head portrait modeling data；
Confirm module, if for successful match, return and confirmed as a result, the confirmation result includes to the local device Be stored with this end subscriber head portrait modeling data first confirms result；
Second receiving module, for receiving the second facial characteristics value that the local device is sent；
First playing module for updating the first facial characteristic value using the second facial characteristics value, and combines The original facial image of described end subscriber generates and plays the second video communication image.
The embodiment of the present invention includes advantages below：
The video communication of the embodiment of the present invention should be between local device and opposite equip., and the opposite equip. has user Head portrait model database, user's head portrait model database include the face-image of user and first facial characteristic value.When this When end equipment opens video communication, local device acquires the first video communication image, wherein, the first video communication image includes this The current face image of end subscriber；Then, local device generates confirmation request for the first video communication image, wherein, confirm Ask for confirm opposite equip. user's head portrait model database in whether be stored with the request of this end subscriber head portrait modeling data, Head portrait modeling data includes the original facial image of this end subscriber and first facial characteristic value；Then, local device uses local terminal The current face image and confirmation request at family are sent to opposite equip., wherein, opposite equip. is used to return for confirmation request true Recognize as a result, confirming that result includes being stored with the first of this end subscriber head portrait modeling data and confirms result；Confirm for described first As a result, local device extracts the second facial characteristics value of this end subscriber, finally, local terminal from the current face image of this end subscriber Second facial characteristics value is sent to opposite equip. by equipment；Opposite equip. is used to update first facial using the second facial characteristics value Characteristic value, and with reference to the original facial image of described end subscriber, generate and play the second video communication image.In this way, when this End subscriber, in video communication, can not have to the collected intact video images of local device being sent to opposite end with peer user Equipment, but the characteristic value of local terminal user's face in video image is sent to opposite equip., opposite equip. is according to this end subscriber The characteristic value of face, with reference to user's head portrait modeling data, simulates the collected video image of local device, so as in network matter Measure bad or network bandwidth it is inadequate when, can still carry out the video communication of high quality.
Description of the drawings
Fig. 1 is a kind of step flow chart one of the embodiment of the method for video communication of the present invention；
Fig. 2 is a kind of step flowchart 2 of the embodiment of the method for video communication of the present invention；
Fig. 3 is a kind of structure diagram one of the mobile device embodiment of video communication of the present invention；
Fig. 4 is a kind of structure diagram two of the mobile device embodiment of video communication of the present invention.
Specific embodiment
In order to make the foregoing objectives, features and advantages of the present invention clearer and more comprehensible, it is below in conjunction with the accompanying drawings and specific real Applying mode, the present invention is described in further detail.
With reference to Fig. 1, a kind of step flow chart one of the embodiment of the method for video communication of the present invention, the method are shown Applied between local device and opposite equip..
In embodiments of the present invention, local device and opposite equip. can have following features：
(1) on hardware systems, equipment has central processing unit, memory, input part and output block, that is, It says, equipment is often the microcomputer devices for having communication function.Furthermore it is also possible to multiple input modes, such as key Disk, mouse, touch screen, transmitter and camera etc., and it can be adjusted input as needed.Meanwhile equipment often has A variety of way of outputs, such as receiver, display screen, can also be adjusted as needed；
(2) on software architecture, equipment must have operating system, as Windows Mobile, Symbian, Palm, Android, iOS etc..Meanwhile these operating systems are more and more open, of the operating system platform exploitation based on these openings Property application program emerge in an endless stream, such as address book, schedule, notepad, calculator and various types of games, it is dramatically full The foot demand of personalized user；
(3) in communication capacity, equipment has flexible access way and high-bandwidth communication performance, and can be selected by The business selected and residing environment, the communication mode selected by adjust automatically, so as to user-friendly.Equipment can support GSM, WCDMA, CDMA2000, TDSCDMA, Wi-Fi and WiMAX etc. so as to adapt to multiple types network, not only support voice industry Business, more supports a variety of wireless data services；
(4) in function using upper, equipment more focuses on hommization, personalization and multifunction.With computer technology Development, equipment enter the pattern of " human-centred " from the pattern of " facility center management ", are integrated with embedding assembly, control skill Art, artificial intelligence technology and biometrics etc. have fully demonstrated people-oriented objective.Due to the hair of software technology Exhibition, equipment can be adjusted according to demands of individuals and be set, more personalized.Meanwhile equipment is integrated with numerous software and hardwares in itself, Function is also stronger and stronger.
The method specifically may include steps of：
Step 101, when the local device opens video communication, the local device acquires the first video communication figure Picture；First video communication image includes the current face image of this end subscriber；
Specifically, local device can carry out video communication by third party software, it is common such as having QQ, wechat i.e. When communication software, the video communication software built in equipment can also be passed through and carry out video communication, such as Apple Inc. iOS and Mac A video calling software FaceTime built in OS X, it is, of course, also possible to be other manner carry out video communication, the application This is not restricted.
By taking QQ as an example, after unlatching video is clicked in chat interface of this end subscriber in QQ with good friend, QQ can send one The request of a video communication can also send an interrupt signal to local device to opposite equip. in local device CPU, which represents that QQ needs the acquisition that camera is called to carry out video image, after CPU receives the interrupt signal, The interface of camera is called to enable camera.After peer user agrees to the request of video communication in opposite equip., local terminal The camera of equipment starts to acquire the first video communication image.Wherein, first video communication image includes this end subscriber Current face image, that is, currently open video communication at the time of user face-image.
Need what is illustrated, in this application, local terminal and opposite end are opposite, for example, having device A and equipment B, if station exists For the angle of device A, then device A is exactly local device, and equipment B is exactly opposite equip.；If come from the angle of equipment B It says, then equipment B is exactly local device, and device A is exactly opposite equip..Because video communication is that both devices is needed to open simultaneously Camera, so, when local device opens video communication, and acquires video communication image, opposite equip. is also opened in fact Video communication, and video communication image is acquired, the local terminal and opposite end in the application be intended merely to facilitate description technique scheme, It is not the limitation to the application.
Step 102, the local device generates confirmation request, the confirmation request for first video communication image Whether the request of this end subscriber head portrait modeling data is stored in user's head portrait model database to confirm the opposite equip.； The head portrait modeling data includes the original facial image of this end subscriber and first facial characteristic value；
The essence of video image is exactly animation in fact, so-called animation, exactly using reference object frame by frame and continuously play and Form the image technology of movement.Animation be by be drawn as after the decomposition such as the expression of personage, action, variation it is many action moments Picture, then a series of pictures are continuously shot into video camera, the picture of consecutive variations is caused to vision.Its basic principle and electricity Shadow, TV are the same, are all persistence of vision principles.The medical evidence mankind have the characteristic of " persistence of vision ", and the eyes of people see one After width picture or an object, it will not disappear in 0.34 second.Using this principle, played before a width is drawn and disappears not yet next Width is drawn, and will be given artificial into a kind of visual variation effects of smoothness.
The application is exactly using this principle, and the face-image of user and first facial feature are obtained from video image Value.Specifically, usually, the animation of 1 second includes 24 images, therefore, can obtain user's respectively from 24 images Face-image and facial characteristic value, then therefrom select one as the original facial image of user and first facial characteristic value. In addition, because user when carrying out video communication, the movement range of usual body is not too large, and 1 second time also very It is short, so, the position of the face of user hardly changes in 24 images, and what is changed is expression substantially, that is, The position of face, such as face, eyes etc., therefore, using this feature, will can also respectively be obtained from 24 images 24 The face-image for opening user synthesizes original facial image of the image as user, then extracts and uses from the image after synthesis The first facial characteristic value at family.
And the extraction of facial characteristics value then depends on face recognition technology, recognition of face is substantially three-dimensional plastic objects two The matching problem of projected image is tieed up, its difficulty is embodied in：(1) uncertainty of face plastic deformation (such as expression)；(2) people The diversity (such as beard, hair style, glasses, makeup) of face pattern；(3) uncertainty in image acquisition procedures is (such as illumination Intensity, light source direction etc.).Identification face relies primarily on the feature on face.That is it is deposited in Different Individual according to those Larger difference and to same person then more stable measurement.Since face changes complicated, feature statement and feature Extraction is very difficult.
It is generally required before feature extraction and classification is carried out to facial image and does geometrical normalization and gray scale normalization.It is several What normalization refers to face in image is transformed to same position and onesize according to Face detection result, and gray scale normalization is The influence of illumination variation can be overcome and improve identification to a certain extent to processing, illumination compensations such as image progress illumination compensations by referring to Rate.
The method of extraction face features value can have following several：
(1) method based on geometric properties
Face is made of components such as eyes, nose, face, chins, just because of in the shape of these components, size and structure Each species diversity just so that each face does poor ten thousand not in the world, therefore shape to these components and the geometry of structural relation are retouched It states, it can be as the important feature of recognition of face.Geometric properties are the description and identification for face side profile earliest, first Several significant points are determined according to side profile curve, and by these significant points export one group of characteristic measure for identification such as away from From, angle etc..Jia etc. is a kind of side for having very much new meaning by the integral projection simulation side profile figure near the gray-scale map center line of front Method.
Position of the front face identification generally by Important Characteristic Points such as extraction human eye, mouth, noses is carried out using geometric properties The geometry with the vitals such as eyes is put as characteristic of division, but Roder carries out the accuracy of Extraction of Geometrical Features Experimental research, as a result allows of no optimist.Deformable template method can be considered as a kind of improvement of geometric properties method, think substantially Think be：The organ model of a Parameter adjustable is designed, an energy function is defined, makes energy function most by adjusting model parameter Smallization, model parameter at this time is i.e. as the geometric properties of the organ.This method and thought is fine, but there are problems that two, First, the weighting coefficient of various costs can only be by empirically determined in energy function, it is difficult to promote；Second is that energy function optimization process It is quite time-consuming, it is difficult to practical application.
Face representation based on parameter, which can be realized, efficiently describes, but it is needed largely one of face notable feature Pre-treatment and fine parameter selection.Meanwhile the basic configuration and structural relation of component are only described using general geometric properties, Local fine feature is had ignored, causes the loss of partial information, more suitable for doing rough sort, and existing characteristic point inspection at present For survey technology also far from meeting the requirements on accurate rate, calculation amount is also larger.
(2) method of feature based face
The method that Turk and Pentland proposes eigenface, it constructs principal component subspace according to lineup's face training image, Since pivot has the shape of face, also referred to as eigenface.Test image is projected on principal component subspace during identification, obtains one group Projection coefficient and each known facial image are relatively identified.Pentland etc. reports fairly good as a result, 200 95% correct recognition rata is obtained in 3000 personal width images, to 150 width front faces as there was only one on FERET databases A misrecognition.But system needs to make a large amount of pretreatment works, such as normalization before eigenface method is carried out.
On the basis of traditional characteristic face, researcher notices that the big eigenface of characteristic value knows vectorial (i.e. eigenface) simultaneously The good direction of classification performance is not necessarily, has developed various features (subspace) selection method accordingly, such as the Shuangzi space of Peng Method, the linear ambiguity analysis method of Weng, FisherFace methods of Belhumeur etc..In fact, eigenface method is one The explicit pivot analysis face modeling of kind, some linear association, linear compression type BP nets certainly are then implicit pca method.It Be all the weighted sums of face representation for some vectors, these vectors are the main feature vectors of training set cross product battle array, Valetin This is discussed in detail.In short, eigenface method is a kind of simple, quick, practical algorithm based on transformation coefficient feature, But since it is in itself dependent on training set and the Gray Correlation of test set image, so also there is significant limitation.
(3) Local Features Analysis LFA methods
The expression of principal component subspace is compact, and intrinsic dimensionality substantially reduces, but it is delocalization, kernel function Support is extended in entire coordinate space, while it is non-topological, after some axial projection in neighbouring point and artwork image space The propinquity of point does not have any relationship, and locality and topological are ideal characteristics to pattern analysis and segmentation, it appears that this is more Meet the mechanism of Neural information processing, therefore it is particularly significant to find the expression with this characteristic.In view of this consideration, Atick It is proposed face characteristic extraction and recognition methods based on local feature.This method achieves good effect in practical application, It constitutes the basis of Facelt softwares.
Local Features Analysis (Local Feature Analysis, LFA) is the face recognizing that a kind of feature based represents Technology, derived from the principle of the similar partial statistics for building building blocks.LFA based on all image surfaces (style for including various complexity) all It can be integrated from the structural unit subset by cannot much simplify.These units shape using complicated statistical technique Into, they represent entire image surface, usually across multiple pixels (in regional area) and represent universal face shape, but It is not the facial characteristics on ordinary meaning.In fact, the position of face structure unit specific surface picture is much more.
However, to integrate to form an accurate image surface true to nature, it is only necessary to seldom subset of cells in entire usable set (12~40 feature unit).Determine that identity depends not only on characteristics unit, further depend on they geometry (such as it Relevant position).In this way, LFA by personal characteristic correspondence into a kind of Chinese real number expression of complexity, Ke Yijin Row comparison and identification." face line " coding mode be according to the substantive characteristics and shape of face come work, it can resist light, The variation of skin color, facial hair, hair style, glasses, expression and posture has powerful reliability, allows it from million A people is accurately recognized in people.Silver-colored morning face recognizing system is exactly this method.
(4) method based on elastic model
Lades et al. proposes dynamic link model (DLA) for the object identification of distortion invariant, by object with sparse Figure describes, and the multiple dimensioned description of vertex local energy marks, side then represent topological connection relation and with geometry away from From marking, nearest oneself is then found using plasticity Graphics matching and knows figure.Wiscott et al. makees on this basis It improves, is tested, made comparisons with 300 width facial images and other 300 width image, rate of accuracy reached is arrived with image libraries such as FERET 97.3%；The shortcomings that the method is that calculation amount is very huge.
Facial image I (x, y) is modeled as deformable 3D surface meshes (x, y, I (x, y)) by Nastar, thus by face Matching problem is converted into the Elastic Matching problem of deformable surface.Using the method march facial disfigurement of finite element analysis, and root Judge whether two pictures are same person according to the situation of deformation.The characteristics of this method, is space (x, y) and gray scale I (x, y) has been placed in a 3d space while has considered, experiment shows that recognition result is substantially better than eigenface method.
The flexible presentation model method of the propositions such as Lanitis, by being automatically positioned the notable feature of face, face is encoded to 83 model parameters, and carry out the recognition of face based on shape using the method for discrimination analysis.
(5) neural network method
Research of the neural network method in recognition of face at present is in the ascendant.Valentin proposes a kind of method, first 50 pivots of face are extracted, are then mapped it in 5 dimension spaces with auto-correlation neural network, then with a common multilayer Perceptron is differentiated, preferable to some simple test image effects；Intrator etc. proposes a kind of mixed type nerve net Network carries out recognition of face, wherein non-supervisory neural network is used for feature extraction, and supervises neural network for classifying.Lee etc. It will be described the characteristics of face with six rules, the positioning of face then carried out according to this six rule, by the geometry between face Distance input fuzzy neural network is identified, and the more general method based on Euclidean distance of effect has larger improvement； Laurence etc. carries out recognition of face using convolutional neural networks methods, due to be integrated in convolutional neural networks adjacent pixel it Between related sex knowledge, so as to being obtained to a certain extent to image translation, rotation and the invariance of local deformation, thus To ideal recognition result；Lin etc. proposes the neural network method (PDBNN) based on Probabilistic Decision-making, main thought It is that reinforcing and anti-intensified learning are carried out using virtual (positive counter-example) sample, so as to obtain ideal probability Estimation as a result, simultaneously Accelerate the study of network using modular network structure (OCON).This method is known in Face datection, Face detection and face Preferable application is obtained in other each step.Other researchs also have：The it is proposeds such as Dai carry out low point with Hopfield networks Resolution face associates and identification；Gutta etc. proposes to combine RBF into the hybrid classification for carrying out recognition of face with tree-structure network Device model；Matching Pursuit wave filters are used for recognition of face by Phillips et al.；Somebody's Statistical Learning Theory In SVM (SVM) carry out face classification.
Application of the neural network method in recognition of face has certain advantage compared with aforementioned a few class methods, because to people It is extremely difficult that many rules or rule of face identification, which carry out dominant description, and neural network method can then pass through study Process obtain to these rules and rule covert expression, its adaptability is stronger, is generally also easier to realize.
(6) other methods
Brunelli etc. has made many experiments to template matching method, the results showed that each in scale, illumination, rotation angle etc. In the case of kind conditional stability, the effect of template matches is better than other methods, but it is quicker to illumination, rotation and expression shape change Sense, affects its direct use.Using basis for estimation of the local autocorrelation as recognition of face, it has Goudail et al. There is translation invariance, it is more stable when countenance changes.
Certainly, the above method is only merely illustrative, in addition to the above method, other to can be used for extracting user's face The method of characteristic value is suitable for the application, and the application is not restricted this.
In embodiments of the present invention, opposite equip. has user's head portrait model database, user's head portrait modeling data Library includes the original facial image of this end subscriber and first facial characteristic value.For example, it is assumed that local device A and opposite equip. B, setting Standby C carries out video communication, user's head portrait model database is respectively provided in opposite equip. B, C, wherein being just stored with this end subscriber A Original facial image and first facial characteristic value.Certainly, if angle from equipment B, then local device is exactly B, right Also the original facial image of this end subscriber B and first facial feature are all respectively provided in the head portrait model database of end equipment A, C Value, the rest may be inferred by equipment C.Because local device is merely not only to carry out one-to-one video communication, institute with an opposite equip. With either local device or opposite equip. are required for the original facial image of multiple users and first facial characteristic value It is stored, and the position stored is exactly user's head portrait model database.
Wherein, the original facial image of user is when local device is with opposite equip. video communication before, opposite equip. is deposited The face-image of this end subscriber of storage.For example, if current time is on April 10th, 2017, and this end subscriber A and peer user The time of B first time video communications is on April 9th, 2017, and in this video communication, local device A stores peer user B Face-image, opposite equip. B also stores the face-image of this end subscriber A, then the peer user in local device A at this time The face-image of B is exactly the original facial image of peer user B, the face-image of this end subscriber A in opposite equip. B is exactly this The original facial image of end subscriber A.In current time, this end subscriber A carries out video communication with peer user B again, at this point, this End equipment A can be directed to current collected first video communication image generation confirmation request, and the confirmation request is described in confirmation Whether the request of this end subscriber head portrait modeling data is stored in user's head portrait model database of opposite equip..
It should be noted that, although this end subscriber is with peer user, the opposite equip. in video communication before just stores The head portrait modeling data of this end subscriber it is, however, possible to situations such as artificially clearing up, corrupted data occur, causes in opposite equip. The head portrait modeling data of this end subscriber of storage can not use, and therefore, each local device carries out video communication with opposite equip. When, it can first check whether opposite equip. is stored with this end subscriber head portrait modeling data.
Step 103, the current face image of described end subscriber and confirmation request are sent to described by the local device Opposite equip.；The opposite equip., which is used to return for the confirmation request, to be confirmed as a result, the confirmation result includes being stored with The first of this end subscriber head portrait modeling data confirms result；
The current face image that local device obtains this end subscriber from the first video communication image can be in local device It establishes after video communication success and obtains with opposite equip..Specifically, the current face figure of this end subscriber that local device is sent Seem to be extracted from the first video communication image that local device acquires, it can be directly from the first video communication image Current face image of first image as this end subscriber is extracted, multiple images can also be extracted and therefrom select one as local terminal The current face image of user can also extract multiple images and be then combined with into current face of the image as this end subscriber Then image is sent to opposite equip. together together with confirmation request.
The current face image that local device obtains this end subscriber from the first video communication image can also be set in local terminal It is standby to establish video communication success acquisition before with opposite equip..Specifically, when this end subscriber initiates to carry out video with opposite equip. During communication, the camera of local device is working, before peer user is agreed to establish the request of connection, this end subscriber It is that the video image of oneself can be seen in local device, then, local device can also obtain this end subscriber at this time Current face image, in this way, when peer user is agreed to establish the request of connection, sheet that local device will can directly be got The current face image of end subscriber is sent to opposite equip. together together with confirmation request.
It, can be by the current of this end subscriber after opposite equip. receives the current face image and confirmation request of this end subscriber Face-image is matched with user's original facial image in user's icon database, specifically, the current face by user User's original facial image in user's head portrait model database of image and opposite equip. carries out comparing calculation similarity one by one, When similarity is more than some threshold value, then it is assumed that the original of this end subscriber is stored in user's head portrait model database of opposite equip. Beginning face-image, then generation confirm result and return to local device.
Step 104, confirm for described first as a result, the local device is from the current face image of described end subscriber Middle the second facial characteristics value for extracting this end subscriber；
When local device receives the confirmation of opposite equip. return as a result, learning user's head portrait modeling data of opposite equip. After the original facial image of this end subscriber and first facial characteristic value are stored in library, local device can be acquired from camera Video in extract this end subscriber the second facial characteristics value, i.e. this end subscriber of current time facial characteristics value.It specifically, can To extract the second facial characteristics value of this current end subscriber from each frame image of the first video communication image.
Step 105, the second facial characteristics value is sent to the opposite equip. by the local device；The opposite end is set It is ready for use on and the first facial characteristic value is updated using the second facial characteristics value, and with reference to the original face of described end subscriber Portion's image generates and plays the second video communication image.
The second facial characteristics value is sent to by local device after the second facial characteristics value of this end subscriber is obtained Opposite equip., the current face characteristic value of the second facial characteristics value, i.e. user that opposite equip. receives, while from user's head portrait The original facial image of user and first facial characteristic value are obtained in model database, is then updated using the second facial characteristics value First facial characteristic value with reference to the original facial image of user, so as to generate the second video communication image, and plays out.
Because stored in user's head portrait model database of opposite end be an anticipatory remark end subscriber original facial image and The first facial characteristic value of user in the image, so, when opposite equip. is by the facial characteristics value of each this end subscriber of Zhang Dangqian When updating the first facial characteristic value in this end subscriber original facial image, it is possible on the basis of original facial image, knot The second facial characteristics value is closed, generates the current face-image of this end subscriber, is then continuously played each image, so as to be formed Video image.
Certainly, because the similarity of the original facial image of the current face image of this end subscriber and this end subscriber very Height, so, in embodiments of the present invention, opposite equip. is in addition to may be used the second facial characteristics value update first facial feature Value then in conjunction with the original facial image of this end subscriber, is generated outside the second video communication image, and the second face can also be used special Value indicative updates first facial characteristic value, then in conjunction with the current face image of this end subscriber that local device sends over, generation Second video communication image.In this way, local device can be in the case where not sending complete video communication image so that opposite end Equipment can also simulate the current face image of this end subscriber according to the facial characteristics value of this current end subscriber.
In a kind of preferred embodiment of the present invention, the confirmation result, which further includes, is not stored with this end subscriber head portrait modeling number According to second confirm result；
Confirm for described second as a result, the method further includes：
Preceding n second video communication images of the local device based on first video communication image generate this end subscriber Head portrait modeling data；The n is the integer not less than 1；
The local device models the head portrait of the preceding n seconds video communication image and described end subscriber of generation Data are sent to the opposite equip.；The opposite equip. is for playing the preceding n seconds video communication images and by the local terminal The head portrait modeling data of user is stored to user's head portrait model database；
The local device extracted second face of this end subscriber since (n+1)th second of first video communication image Characteristic value；
The second facial characteristics value is sent to the opposite equip. by the local device；The opposite equip. is used to adopt The first facial characteristic value is updated with the second facial characteristics value, and with reference to the current face image of described end subscriber, It generates and plays the second video communication image.
Specifically, when the head portrait modeling data that this end subscriber is not stored in user's head portrait model database of opposite equip. When, it is necessary to the head portrait modeling data of this end subscriber of in-time generatin.Can be that local device is carried from the first video communication image The current face image of this end subscriber and facial characteristic value are extracted in the complete video communication image of n seconds before taking, then together with institute The complete video communication image of n seconds, is sent to opposite equip. together before stating.Because the face of useless end subscriber in opposite equip. Image and facial characteristic value, so, opposite equip. the complete video of preceding n second for receiving local device transmission communicate image, After the current face image of this end subscriber and facial characteristic value, the communication image of the complete video of preceding n seconds received is played on one side, The current face image of this end subscriber and facial characteristic value are stored into user's head portrait model database on one side.Then from n-th+ Start within 1 second, local device does not just retransmit complete video communication image, but directly extracts the second face in video image Portion's characteristic value is sent to opposite equip., and opposite equip. is because have received the current face image of local terminal user in n seconds first With first facial characteristic value, so the second facial characteristics value can be used to update first facial characteristic value, and combine local terminal and use The current face image at family generates the second video communication image, then plays out.
In a kind of preferred embodiment of the present invention, confirm for described second as a result, the method further includes：
The preceding n seconds video communication image is sent to the opposite equip. by the local device；The opposite equip. is used In playing the preceding n second video communication image, and the head portrait for being generated based on the preceding n seconds video communication image this end subscriber is modeled Data, and the head portrait modeling data of this end subscriber of generation is stored to user's head portrait model database；
The second facial characteristics value is sent to the opposite equip. by the local device；The opposite equip. is used to adopt The first facial characteristic value is updated with the second facial characteristics value, and with reference to the current face image of described end subscriber, It generates and plays the second video communication image.
Specifically, in addition to generating the current face image of this end subscriber and first facial characteristic value in local device, so After be sent to outside opposite equip., the current face image of this end subscriber and first facial feature can also be generated in opposite equip. Value.
It, can be directly by preceding n before local device is extracted from the first video communication image after the complete video image of n seconds The complete video image of second is sent to opposite equip., can also after opposite equip. receives complete video image n seconds first The current face image of this end subscriber and first facial characteristic value are extracted from complete video image n seconds first, and will be raw Into this end subscriber current face image and first facial characteristic value store to user's head portrait model database.When from (n+1)th Second starts, and opposite equip. just receives the second facial characteristics value of local device transmission, updates first using the second facial characteristics value Facial characteristics value, and the face-image of this current end subscriber of generation is combined, the second video communication image is generated, is then broadcast It puts.
In a kind of preferred embodiment of the present invention, it is described generate this end subscriber head portrait modeling data the step of include：
Obtain each frame image of the preceding n seconds video images；The n is the integer not less than 1；
Based on each frame image, the current face image of this end subscriber and first facial characteristic value are generated.
For example, usually the video image of 1 second includes 24 still images, it, can be from 24 static maps when n takes 1 It extracts the face-image of this end subscriber and facial characteristic value respectively as in, that is, the face-image of 24 this end subscribers can be extracted With 24 facial characteristics values, a local terminal is then selected from the face-image of 24 this end subscribers and 24 facial characteristics values and is used The face-image at family and facial characteristic value, as the current face image of this end subscriber and first facial characteristic value.It can also incite somebody to action The face-image of 24 this end subscribers is merged into current face image of the image as this end subscriber, and by the face of the image Portion's characteristic value is as first facial characteristic value.
In practical applications, because the algorithm of recognition of face is highly developed, by several or more than ten images The first facial characteristic value of this end subscriber, that is, the zero time of several seconds can be extracted, so, n is in addition to being no less than 1 Integer outside, decimal, such as 0.5 second, 0.8 second etc. can also be taken, further, other than n+1 or n+0.1, n+0.5 Only be intended merely to clearly demonstrate the technical solution of the application Deng, n and n+1, specific numerical value can according to actual demand into Row setting.
In addition, local device can also be sent simultaneously other than it can the second facial characteristics value be sent to opposite equip. Light filling parameter.Specifically, when under the poor environment of local device leisure light, local device is sending the second facial characteristics value When, light filling parameter can also be sent simultaneously to opposite equip..Wherein, light filling parameter may include whether to need light filling, light filling Program etc..Opposite equip. can pass through algorithm when receiving the second facial characteristics and after light filling parameter according to light filling parameter Simulation light filling is carried out to the second video communication image, in this way, even if when under the poor environment of local device board light, opposite end is set It is standby to restore the preferable video communication image of light.
It should be noted that for embodiment of the method, in order to be briefly described, therefore it is all expressed as to a series of action group It closes, but those skilled in the art should know, the embodiment of the present invention is not limited by described sequence of movement, because according to According to the embodiment of the present invention, certain steps may be used other sequences or be carried out at the same time.Secondly, those skilled in the art also should Know, embodiment described in this description belongs to preferred embodiment, and the involved action not necessarily present invention is implemented Necessary to example.
With reference to Fig. 2, a kind of step flowchart 2 of the embodiment of the method for video communication of the present invention, the method are shown Applied between local device and opposite equip..
The method specifically may include steps of：
Step 201, it when the opposite equip. opens video communication state, receives the first video that the local terminal is sent and leads to Whether letter image and the opposite equip. are stored with the confirmation request of this end subscriber head portrait modeling data；The head portrait modeling Data include the original facial image of local terminal user and first facial characteristic value；
Step 202, the opposite equip. is matched based on the confirmation request in user's head portrait model database；Institute It states user's head portrait model database and includes user's head portrait modeling data；
Step 203, if successful match, the opposite equip. is returned to the local device to be confirmed as a result, the confirmation As a result include being stored with the first confirmation result of this end subscriber head portrait modeling data；
Step 204, the second facial characteristics value that the local device is sent is received；
Step 205, the first facial characteristic value is updated, and use with reference to the local terminal using the second facial characteristics value The original facial image at family generates and plays the second video communication image.
In a kind of preferred embodiment of the present invention, the confirmation result, which further includes, is not stored with this end subscriber head portrait modeling number According to second confirm result；
For the described second confirmation as a result, then local device is after the opposite equip. sends confirmation result, the side Method further includes：
Receive preceding n seconds video communication images of first video communication image that the local device is sent, Yi Jiji In the head portrait modeling data of this end subscriber of the preceding n seconds video communication image generation；The n is the integer not less than 1；
It plays the preceding n seconds video communication images and stores the head portrait modeling data of described end subscriber to the user Head portrait model database；
The opposite equip. received the second facial characteristics value that the local device is sent since (n+1)th second；
The first facial characteristic value is updated using the second facial characteristics value, and with reference to the current of described end subscriber Face-image generates and plays the second video communication image.
In a kind of preferred embodiment of the present invention, confirm for described second as a result, then local device is set to the opposite end After preparation send confirmation result, the method further includes：
Receive preceding n seconds video communication images of the first video communication image that the local device is sent；
The head portrait modeling data of this end subscriber is generated based on the preceding n seconds video communication image；The n is not less than 1 Integer；
It plays the preceding n seconds video communication images and stores the head portrait modeling data of this end subscriber of generation to the use Account is as model database；
The opposite equip. received the second facial characteristics value that the local device is sent since (n+1)th second；
The first facial characteristic value is updated using the second facial characteristics value, and with reference to the current of described end subscriber Face-image generates and plays the second video communication image.
In a kind of preferred embodiment of the present invention, described the step of generating this end subscriber head portrait modeling data, includes：
Obtain each frame image of the preceding n seconds video images；The n is the integer not less than 1；
Based on each frame image, the current face image of this end subscriber and first facial characteristic value are generated.
In addition, local device can also be sent simultaneously other than it can the second facial characteristics value be sent to opposite equip. Light filling parameter.Specifically, when under the poor environment of local device leisure light, local device is sending the second facial characteristics value When, light filling parameter can also be sent simultaneously to opposite equip..Wherein, light filling parameter may include whether to need light filling, light filling Program etc..Opposite equip. can pass through algorithm when receiving the second facial characteristics and after light filling parameter according to light filling parameter Simulation light filling is carried out to the second video communication image, in this way, even if when under the poor environment of local device board light, opposite end is set It is standby to restore the preferable video communication image of light.
For the embodiment of the method for opposite equip., since its embodiment of the method to local device is substantially similar, institute With the fairly simple of description, the relevent part can refer to the partial explaination of embodiments of method
With reference to Fig. 3, show a kind of structure diagram one of the mobile device embodiment of video communication of the present invention, specifically may be used To include following module：
Acquisition module 301, for when the local device opens video communication, acquiring the first video communication image；Institute State the current face image that the first video communication image includes this end subscriber；
Confirmation request generation module 302, for being directed to the first video communication image generation confirmation request, the confirmation It asks to confirm whether the opposite equip. is stored with the request of this end subscriber head portrait modeling data；User's head portrait models number Include the original facial image of user and first facial characteristic value according to library；
First sending module 303, it is described for the current face image of described end subscriber and confirmation request to be sent to Opposite equip.；The opposite equip., which is used to return for the confirmation request, to be confirmed as a result, the confirmation result includes being stored with The first of this end subscriber head portrait modeling data confirms result；
First extraction module 304 confirms the current face image as a result, from described end subscriber for being directed to described first Middle the second facial characteristics value for extracting this end subscriber；
Second sending module 305, for the second facial characteristics value to be sent to the opposite equip.；The opposite end is set It is ready for use on and the first facial characteristic value is updated using the second facial characteristics value, and with reference to the original face of described end subscriber Portion's image generates and plays the second video communication image.
In a kind of preferred embodiment of the present invention, the confirmation result, which further includes, is not stored with this end subscriber head portrait modeling number According to second confirm result；
Confirm for described second as a result, the device further includes：
Head portrait modeling data generation module, for the preceding n seconds video communication images based on first video communication image Generate the head portrait modeling data of this end subscriber；The n is the integer not less than 1；
Third sending module, for by the preceding n seconds video communication image and generation described end subscriber head portrait Modeling data is sent to the opposite equip.；The opposite equip. is used to play the preceding n seconds video communication images and by described in The head portrait modeling data of this end subscriber is stored to user's head portrait model database；
Second extraction module, for extracting the of this end subscriber since (n+1)th second of first video communication image Two facial characteristics values；
4th sending module, for the second facial characteristics value to be sent to the opposite equip.；The opposite equip. For updating the first facial characteristic value using the second facial characteristics value, and with reference to the current face of described end subscriber Image generates and plays the second video communication image.
In a kind of preferred embodiment of the present invention, confirm for described second as a result, the device further includes：
5th sending module, for the preceding n seconds video communication image to be sent to the opposite equip.；The opposite end is set It is ready for use on and plays the preceding n seconds video communication image, and the head portrait of this end subscriber is generated based on the preceding n seconds video communication image Modeling data, and the head portrait modeling data of this end subscriber of generation is stored to user's head portrait model database；
6th sending module, for the second facial characteristics value to be sent to the opposite equip.；The opposite equip. For updating the first facial characteristic value using the second facial characteristics value, and with reference to the current face of described end subscriber Image generates and plays the second video communication image.
In a kind of preferred embodiment of the present invention, the head portrait modeling data generation module includes：
First acquisition submodule, for obtaining each frame image of the preceding n seconds video images；The n is not less than 1 Integer；
First data generate submodule, for being based on each frame image, generate the current face image of this end subscriber With first facial characteristic value.
For device embodiment one, since it is substantially similar to embodiment of the method one, so description is fairly simple, The relevent part can refer to the partial explaination of embodiments of method.
With reference to Fig. 4, show a kind of structure diagram two of the device embodiment of video communication of the present invention, can specifically wrap Include following module：
First receiving module 401, for when the opposite equip. opens video communication state, receiving the local device The confirmation whether the first video communication image and the opposite equip. sent is stored with this end subscriber head portrait modeling data please It asks；User's head portrait modeling data includes the original facial image of this end subscriber and first facial characteristic value；
Matching module 402 is matched for being based on the confirmation request in user's head portrait model database；The use Account includes user's head portrait modeling data as model database；
Confirm module 403, if for successful match, return and confirmed as a result, the confirmation result to the local device First including being stored with this end subscriber head portrait modeling data confirms result；
Second receiving module 404, for receiving the second facial characteristics value that the local device is sent；
First playing module 405 for updating the first facial characteristic value using the second facial characteristics value, and is tied The original facial image of described end subscriber is closed, generate and plays the second video communication image.
In a kind of preferred embodiment of the present invention, the confirmation result, which further includes, is not stored with this end subscriber head portrait modeling number According to second confirm result；
Confirm for described second as a result, the device further includes：
Third receiving module, for receiving regarding for preceding n seconds for first video communication image of the local device transmission Frequency communication image and the head portrait modeling data of this end subscriber generated based on the preceding n seconds video communication image；The n is Integer not less than 1；
Second playing module, for playing the preceding n seconds video communication images and modeling the head portrait of described end subscriber Data are stored to user's head portrait model database；
4th receiving module, for since (n+1)th second, receiving the second facial characteristics value that the local device is sent；
Third playing module for updating the first facial characteristic value using the second facial characteristics value, and combines The current face image of described end subscriber generates and plays the second video communication image.
In a kind of preferred embodiment of the present invention, confirm for described second as a result, the device further includes：
5th receiving module, the preceding n seconds videos for receiving the first video communication image that the local device is sent lead to Believe image；
Generation module, for generating the head portrait modeling data of this end subscriber based on the preceding n seconds video communication image；It is described N is the integer not less than 1；
4th playing module, for playing the preceding n seconds video communication images and building the head portrait of this end subscriber of generation Modulus is according to storing to user's head portrait model database；
6th receiving module, for since (n+1)th second, receiving the second facial characteristics value that the local device is sent；
5th playing module for updating the first facial characteristic value using the second facial characteristics value, and combines The current face image of described end subscriber generates and plays the second video communication image.
In a kind of preferred embodiment of the present invention, the generation module includes：
Second acquisition submodule, for obtaining each frame image of the preceding n seconds video images；The n is not less than 1 Integer；
Second data generate submodule, for being based on each frame image, generate the current face image of this end subscriber With first facial characteristic value.
For device embodiment two, since it is substantially similar to embodiment of the method two, so description is fairly simple, The relevent part can refer to the partial explaination of embodiments of method.
Each embodiment in this specification is described by the way of progressive, the highlights of each of the examples are with The difference of other embodiment, just to refer each other for identical similar part between each embodiment.
It should be understood by those skilled in the art that, the embodiment of the embodiment of the present invention can be provided as method, apparatus or calculate Machine program product.Therefore, the embodiment of the present invention can be used complete hardware embodiment, complete software embodiment or combine software and The form of the embodiment of hardware aspect.Moreover, the embodiment of the present invention can be used one or more wherein include computer can With in the computer-usable storage medium (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.) of program code The form of the computer program product of implementation.
The embodiment of the present invention be with reference to according to the method for the embodiment of the present invention, terminal device (system) and computer program The flowchart and/or the block diagram of product describes.It should be understood that flowchart and/or the block diagram can be realized by computer program instructions In each flow and/or block and flowchart and/or the block diagram in flow and/or box combination.These can be provided Computer program instructions are set to all-purpose computer, special purpose computer, Embedded Processor or other programmable data processing terminals Standby processor is to generate a machine so that is held by the processor of computer or other programmable data processing terminal equipments Capable instruction generation is used to implement in one flow of flow chart or multiple flows and/or one box of block diagram or multiple boxes The device for the function of specifying.
These computer program instructions, which may also be stored in, can guide computer or other programmable data processing terminal equipments In the computer-readable memory to work in a specific way so that the instruction being stored in the computer-readable memory generates packet The manufacture of command device is included, which realizes in one flow of flow chart or multiple flows and/or one side of block diagram The function of being specified in frame or multiple boxes.
These computer program instructions can be also loaded into computer or other programmable data processing terminal equipments so that Series of operation steps are performed on computer or other programmable terminal equipments to generate computer implemented processing, thus The instruction offer performed on computer or other programmable terminal equipments is used to implement in one flow of flow chart or multiple flows And/or specified in one box of block diagram or multiple boxes function the step of.
Although the preferred embodiment of the embodiment of the present invention has been described, those skilled in the art once know base This creative concept can then make these embodiments other change and modification.So appended claims are intended to be construed to Including preferred embodiment and fall into all change and modification of range of embodiment of the invention.
Finally, it is to be noted that, herein, relational terms such as first and second and the like be used merely to by One entity or operation are distinguished with another entity or operation, without necessarily requiring or implying these entities or operation Between there are any actual relationship or orders.Moreover, term " comprising ", "comprising" or its any other variant meaning Covering non-exclusive inclusion, so that process, method, article or terminal device including a series of elements are not only wrapped Those elements are included, but also including other elements that are not explicitly listed or are further included as this process, method, article Or the element that terminal device is intrinsic.In the absence of more restrictions, it is wanted by what sentence "including a ..." limited Element, it is not excluded that also there are other identical elements in the process including the element, method, article or terminal device.
Method to a kind of video communication provided by the present invention and a kind of mobile device of video communication above carry out It is discussed in detail, specific case used herein is expounded the principle of the present invention and embodiment, above example Illustrate to be merely used to help understand method and its core concept of the invention；Meanwhile for those of ordinary skill in the art, according to According to the thought of the present invention, there will be changes in specific embodiments and applications, in conclusion the content of the present specification It should not be construed as limiting the invention.

Claims (16)

1. a kind of method of video communication, applied to mobile device, which is characterized in that the mobile device include local device and Opposite equip., the method are applied between local device and opposite equip.；The method includes：
When the local device opens video communication, the local device acquires the first video communication image；Described first regards Frequency communication image includes the current face image of this end subscriber；
The local device generates confirmation request for first video communication image, and the confirmation request is described right to confirm Whether end equipment is stored with the request of this end subscriber head portrait modeling data；User's head portrait model database includes the original of user Beginning face-image and first facial characteristic value；
The current face image of described end subscriber and confirmation request are sent to the opposite equip. by the local device；It is described Opposite equip. is confirmed for being returned for the confirmation request as a result, the confirmation result is built including being stored with this end subscriber head portrait The first of modulus evidence confirms result；
Confirmed for described first as a result, the local device extracts local terminal use from the current face image of described end subscriber The second facial characteristics value at family；
The second facial characteristics value is sent to the opposite equip. by the local device；The opposite equip. is used for using institute It states the second facial characteristics value and updates the first facial characteristic value, and with reference to the original facial image of described end subscriber, generation And play the second video communication image.
2. according to the method described in claim 1, it is characterized in that, the confirmation result further includes and is not stored with this end subscriber head Second as modeling data confirms result；
Confirm for described second as a result, the method further includes：
Preceding n second video communication images of the local device based on first video communication image generate the head portrait of this end subscriber Modeling data；The n is the integer not less than 1；
The local device by the preceding n seconds video communication image and generation described end subscriber head portrait modeling data It is sent to the opposite equip.；The opposite equip. is for playing the preceding n seconds video communication images and by described end subscriber Head portrait modeling data store to user's head portrait model database；
The local device extracted second facial characteristics of this end subscriber since (n+1)th second of first video communication image Value；
The second facial characteristics value is sent to the opposite equip. by the local device；The opposite equip. is used for using institute It states the second facial characteristics value and updates the first facial characteristic value, and with reference to the current face image of described end subscriber, generation And play the second video communication image.
3. according to the method described in claim 2, it is characterized in that, confirm for described second as a result, the method is also wrapped It includes：
The preceding n seconds video communication image is sent to the opposite equip. by the local device；The opposite equip. is used to broadcast The preceding n seconds video communication images are put, and the head portrait modeling number of this end subscriber is generated based on the preceding n seconds video communication image According to, and the head portrait modeling data of this end subscriber of generation is stored to user's head portrait model database；
The second facial characteristics value is sent to the opposite equip. by the local device；The opposite equip. is used for using institute It states the second facial characteristics value and updates the first facial characteristic value, and with reference to the current face image of described end subscriber, generation And play the second video communication image.
4. according to the method described in claim 2,3, which is characterized in that the step of the head portrait modeling data for generating this end subscriber Suddenly include：
Obtain each frame image of the preceding n seconds video images；The n is the integer not less than 1；
Based on each frame image, the current face image of this end subscriber and first facial characteristic value are generated.
5. a kind of method of video communication, applied to mobile device, which is characterized in that the mobile device include local device and Opposite equip., the method are applied between local device and opposite equip.；The method includes：
When the opposite equip. opens video communication state, receive the first video communication image that the local terminal sends and Whether the opposite equip. is stored with the confirmation request of this end subscriber head portrait modeling data；User's head portrait modeling data includes The original facial image of this end subscriber and first facial characteristic value；
The opposite equip. is matched based on the confirmation request in user's head portrait model database；User's head portrait is built Mould database includes user's head portrait modeling data；
If successful match, the opposite equip. is returned to the local device to be confirmed as a result, the confirmation result includes storage Have this end subscriber head portrait modeling data first confirms result；
Receive the second facial characteristics value that the local device is sent；
The first facial characteristic value is updated, and with reference to the original facial of described end subscriber using the second facial characteristics value Image generates and plays the second video communication image.
6. according to the method described in claim 5, it is characterized in that, the confirmation result further includes and is not stored with this end subscriber head Second as modeling data confirms result；
Confirm that the method is also as a result, then local device is after the opposite equip. sends confirmation result for described second Including：
Receive preceding n seconds video communication images of first video communication image that the local device is sent and based on institute The head portrait modeling data of this end subscriber of video communication image generation in n seconds before stating；The n is the integer not less than 1；
It plays the preceding n seconds video communication images and stores the head portrait modeling data of described end subscriber to user's head portrait Model database；
The opposite equip. received the second facial characteristics value that the local device is sent since (n+1)th second；
The first facial characteristic value is updated, and with reference to the current face of described end subscriber using the second facial characteristics value Image generates and plays the second video communication image.
7. according to the method described in claim 6, it is characterized in that, confirmed for described second as a result, then local device is to institute After stating opposite equip. transmission confirmation result, the method further includes：
Receive preceding n seconds video communication images of the first video communication image that the local device is sent；
The head portrait modeling data of this end subscriber is generated based on the preceding n seconds video communication image；The n is the integer not less than 1；
It plays the preceding n seconds video communication images and stores the head portrait modeling data of this end subscriber of generation to described and use account As model database；
The opposite equip. received the second facial characteristics value that the local device is sent since (n+1)th second；
The first facial characteristic value is updated, and with reference to the current face of described end subscriber using the second facial characteristics value Image generates and plays the second video communication image.
8. according to the method described in claim 6,7, which is characterized in that described the step of generating this end subscriber head portrait modeling data Including：
Obtain each frame image of the preceding n seconds video images；The n is the integer not less than 1；
Based on each frame image, the current face image of this end subscriber and first facial characteristic value are generated.
9. a kind of mobile device of video communication, which is characterized in that the mobile device includes local device and opposite equip., institute Video communication applications are stated between local device and opposite equip.；The mobile device includes：
Acquisition module, for when the local device opens video communication, acquiring the first video communication image；Described first regards Frequency communication image includes the current face image of this end subscriber；
Confirmation request generation module, for being directed to the first video communication image generation confirmation request, the confirmation request is Confirm whether the opposite equip. is stored with the request of this end subscriber head portrait modeling data；User's head portrait model database packet Include the original facial image of user and first facial characteristic value；
First sending module, for the current face image of described end subscriber and confirmation request to be sent to the opposite end and set It is standby；The opposite equip., which is used to return for the confirmation request, to be confirmed as a result, the confirmation result includes being stored with local terminal use Account confirms result as the first of modeling data；
First extraction module is confirmed for being directed to described first as a result, being extracted from the current face image of described end subscriber The second facial characteristics value of this end subscriber；
Second sending module, for the second facial characteristics value to be sent to the opposite equip.；The opposite equip. is used for The first facial characteristic value is updated, and with reference to the original facial figure of described end subscriber using the second facial characteristics value Picture generates and plays the second video communication image.
10. mobile device according to claim 9, which is characterized in that the confirmation result, which further includes, is not stored with local terminal The second of user's head portrait modeling data confirms result；
Confirm for described second as a result, the device further includes：
Head portrait modeling data generation module, for the preceding n seconds video communication images generation based on first video communication image The head portrait modeling data of this end subscriber；The n is the integer not less than 1；
Third sending module, for the head portrait of the preceding n seconds video communication image and described end subscriber of generation to be modeled Data are sent to the opposite equip.；The opposite equip. is for playing the preceding n seconds video communication images and by the local terminal The head portrait modeling data of user is stored to user's head portrait model database；
Second extraction module, for extracting second face of this end subscriber since (n+1)th second of first video communication image Portion's characteristic value；
4th sending module, for the second facial characteristics value to be sent to the opposite equip.；The opposite equip. is used for The first facial characteristic value is updated, and with reference to the current face figure of described end subscriber using the second facial characteristics value Picture generates and plays the second video communication image.
11. mobile device according to claim 10, which is characterized in that confirm the dress as a result, described for described second It puts and further includes：
5th sending module, for the preceding n seconds video communication image to be sent to the opposite equip.；The opposite equip. is used In playing the preceding n second video communication image, and the head portrait for being generated based on the preceding n seconds video communication image this end subscriber is modeled Data, and the head portrait modeling data of this end subscriber of generation is stored to user's head portrait model database；
6th sending module, for the second facial characteristics value to be sent to the opposite equip.；The opposite equip. is used for The first facial characteristic value is updated, and with reference to the current face figure of described end subscriber using the second facial characteristics value Picture generates and plays the second video communication image.
12. mobile device according to claim 10, which is characterized in that the head portrait modeling data generation module includes：
First acquisition submodule, for obtaining each frame image of the preceding n seconds video images；The n is whole not less than 1 Number；
First data generate submodule, for being based on each frame image, generate the current face image and the of this end subscriber One facial characteristic value.
13. a kind of mobile device of video communication, which is characterized in that the mobile device includes local device and opposite equip., The video communication applications are between local device and opposite equip.；The mobile device includes：
First receiving module, for when the opposite equip. opens video communication state, receiving what the local device was sent Whether the first video communication image and the opposite equip. are stored with the confirmation request of this end subscriber head portrait modeling data；Institute It states user's head portrait modeling data and includes the original facial image of this end subscriber and first facial characteristic value；
Matching module is matched for being based on the confirmation request in user's head portrait model database；User's head portrait Model database includes user's head portrait modeling data；
Confirm module, if for successful match, return and confirmed as a result, the confirmation result includes storage to the local device Have this end subscriber head portrait modeling data first confirms result；
Second receiving module, for receiving the second facial characteristics value that the local device is sent；
First playing module, for updating the first facial characteristic value using the second facial characteristics value, and with reference to described The original facial image of this end subscriber generates and plays the second video communication image.
14. mobile device according to claim 13, which is characterized in that the confirmation result, which further includes, is not stored with local terminal The second of user's head portrait modeling data confirms result；
Confirm for described second as a result, the device further includes：
Third receiving module, the preceding n seconds videos for receiving first video communication image that the local device is sent lead to Believe the head portrait modeling data of image and this end subscriber based on the preceding n seconds video communication image generation；The n is not small In 1 integer；
Second playing module, for playing the preceding n seconds video communication images and by the head portrait modeling data of described end subscriber It stores to user's head portrait model database；
4th receiving module, for since (n+1)th second, receiving the second facial characteristics value that the local device is sent；
Third playing module, for updating the first facial characteristic value using the second facial characteristics value, and with reference to described The current face image of this end subscriber generates and plays the second video communication image.
15. mobile device according to claim 14, which is characterized in that confirm the dress as a result, described for described second It puts and further includes：
5th receiving module, for receiving preceding n seconds video communication figures of the first video communication image that the local device is sent Picture；
Generation module, for generating the head portrait modeling data of this end subscriber based on the preceding n seconds video communication image；The n is Integer not less than 1；
4th playing module, for playing the preceding n seconds video communication images and the head portrait of this end subscriber of generation being modeled number According to storing to user's head portrait model database；
6th receiving module, for since (n+1)th second, receiving the second facial characteristics value that the local device is sent；
5th playing module, for updating the first facial characteristic value using the second facial characteristics value, and with reference to described The current face image of this end subscriber generates and plays the second video communication image.
16. mobile device according to claim 15, which is characterized in that the generation module includes：
Second acquisition submodule, for obtaining each frame image of the preceding n seconds video images；The n is whole not less than 1 Number；
Second data generate submodule, for being based on each frame image, generate the current face image and the of this end subscriber One facial characteristic value.
CN201711241079.7A 2017-11-30 2017-11-30 Video communication method and mobile device Active CN108174141B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201711241079.7A CN108174141B (en) 2017-11-30 2017-11-30 Video communication method and mobile device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201711241079.7A CN108174141B (en) 2017-11-30 2017-11-30 Video communication method and mobile device

Publications (2)

Publication Number Publication Date
CN108174141A true CN108174141A (en) 2018-06-15
CN108174141B CN108174141B (en) 2019-12-31

Family

ID=62524802

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201711241079.7A Active CN108174141B (en) 2017-11-30 2017-11-30 Video communication method and mobile device

Country Status (1)

Country Link
CN (1) CN108174141B (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110266994A (en) * 2019-06-26 2019-09-20 广东小天才科技有限公司 A kind of video call method, video conversation apparatus and terminal
CN110769186A (en) * 2019-10-28 2020-02-07 维沃移动通信有限公司 Video call method, first electronic device and second electronic device
CN112218034A (en) * 2020-10-13 2021-01-12 北京字节跳动网络技术有限公司 Video processing method, system, terminal and storage medium
CN113727062A (en) * 2021-11-01 2021-11-30 深圳云集智能信息有限公司 Video conference system and method for processing image data

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110122219A1 (en) * 2009-11-23 2011-05-26 Samsung Electronics Co. Ltd. Method and apparatus for video call in a mobile terminal
CN102970510A (en) * 2012-11-23 2013-03-13 清华大学 Method for transmitting human face video
CN103647922A (en) * 2013-12-20 2014-03-19 百度在线网络技术（北京）有限公司 Virtual video call method and terminals
CN104782120A (en) * 2012-12-17 2015-07-15 英特尔公司 Facial movement based avatar animation
CN105704419A (en) * 2014-11-27 2016-06-22 程超 Method for human-human interaction based on adjustable template profile photos

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110122219A1 (en) * 2009-11-23 2011-05-26 Samsung Electronics Co. Ltd. Method and apparatus for video call in a mobile terminal
CN102970510A (en) * 2012-11-23 2013-03-13 清华大学 Method for transmitting human face video
CN104782120A (en) * 2012-12-17 2015-07-15 英特尔公司 Facial movement based avatar animation
CN103647922A (en) * 2013-12-20 2014-03-19 百度在线网络技术（北京）有限公司 Virtual video call method and terminals
CN105704419A (en) * 2014-11-27 2016-06-22 程超 Method for human-human interaction based on adjustable template profile photos

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110266994A (en) * 2019-06-26 2019-09-20 广东小天才科技有限公司 A kind of video call method, video conversation apparatus and terminal
CN110266994B (en) * 2019-06-26 2021-03-26 广东小天才科技有限公司 Video call method, video call device and terminal
CN110769186A (en) * 2019-10-28 2020-02-07 维沃移动通信有限公司 Video call method, first electronic device and second electronic device
CN112218034A (en) * 2020-10-13 2021-01-12 北京字节跳动网络技术有限公司 Video processing method, system, terminal and storage medium
WO2022078066A1 (en) * 2020-10-13 2022-04-21 北京字节跳动网络技术有限公司 Video processing method and system, terminal, and storage medium
CN113727062A (en) * 2021-11-01 2021-11-30 深圳云集智能信息有限公司 Video conference system and method for processing image data

Also Published As

Publication number Publication date
CN108174141B (en) 2019-12-31

Similar Documents

Publication Publication Date Title
Zhang et al. Facial: Synthesizing dynamic talking face with implicit attribute learning
CN109359538B (en) Training method of convolutional neural network, gesture recognition method, device and equipment
CN107911643B (en) Method and device for showing scene special effect in video communication
KR102174595B1 (en) System and method for identifying faces in unconstrained media
CN109657554B (en) Image identification method and device based on micro expression and related equipment
CN109978754A (en) Image processing method, device, storage medium and electronic equipment
CN108174141A (en) A kind of method of video communication and a kind of mobile device
Tariq et al. Recognizing emotions from an ensemble of features
CN113362263B (en) Method, apparatus, medium and program product for transforming an image of a virtual idol
CN108198130A (en) Image processing method, device, storage medium and electronic equipment
CN111108508B (en) Face emotion recognition method, intelligent device and computer readable storage medium
CN110555896A (en) Image generation method and device and storage medium
CN102567716A (en) Face synthetic system and implementation method
CN112183198A (en) Gesture recognition method for fusing body skeleton and head and hand part profiles
CN108537109B (en) OpenPose-based monocular camera sign language identification method
Hajarolasvadi et al. Generative adversarial networks in human emotion synthesis: A review
CN112613416A (en) Facial expression recognition method and related device
CN113177892A (en) Method, apparatus, medium, and program product for generating image inpainting model
CN109218615A (en) Image taking householder method, device, terminal and storage medium
CN104978583B (en) The recognition methods of figure action and device
CN112906520A (en) Gesture coding-based action recognition method and device
CN111476095A (en) Expression attribute identification method and device, computer equipment and storage medium
CN111768729A (en) VR scene automatic explanation method, system and storage medium
Ibikunle et al. Face recognition using line edge mapping approach
Tran et al. Significant trajectories and locality constrained linear coding for hand gesture representation

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant