CN113487745A - Method, device and system for enhancing reality - Google Patents
Method, device and system for enhancing reality Download PDFInfo
- Publication number
- CN113487745A CN113487745A CN202110804141.9A CN202110804141A CN113487745A CN 113487745 A CN113487745 A CN 113487745A CN 202110804141 A CN202110804141 A CN 202110804141A CN 113487745 A CN113487745 A CN 113487745A
- Authority
- CN
- China
- Prior art keywords
- augmented reality
- model
- person
- trained
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/006—Mixed reality
Abstract
The invention relates to the technical field of augmented reality, in particular to a method, a device and a system for augmented reality, which comprises the following steps: receiving a first augmented reality request which is sent by a first terminal logged in by a first account and corresponds to a second account; receiving a second augmented reality request which is sent by a second terminal logged in by a second account and corresponds to the first account; performing model training based on the first facial image set, the second facial image set and the augmented reality parameter set to obtain a trained first augmented reality model and a trained second augmented reality model; the augmented reality parameter set includes parameters of a plurality of dimensions, the parameters including: skin color adjustment parameters, expression adjustment parameters, face shape adjustment parameters, angle adjustment parameters and decoration adjustment parameters; and sending the trained first augmented reality model to a first terminal, and sending the trained second augmented reality model to a second terminal. The invention can reduce image distortion and adjust the face image in the training process.
Description
Technical Field
The invention relates to the technical field of augmented reality, in particular to a method, a device and a system for augmented reality.
Background
With the rapid development of network technology, network videos are widely popularized and used, and the videos can be live videos or recorded videos. In the process of recording videos, many anchor broadcasters try new interactive modes, for example, change facial images of videos, that is, change their own facial images into facial images of other people in the videos, and then upload the images to a server for playing.
The related art has a scheme of recognizing a face image in a video image using an image recognition technology, replacing the face image in the video image with a face image of a target person, and uploading the video image to a server.
In the related art, when the video image is subjected to the face image processing, the local image replacement is performed in the video image, and the situation that the shape of the face image of the target person is not matched with that of the face image of the original person is likely to occur, and the face image of the target person needs to be subjected to deformation processing, so that the image is distorted, and the effect is poor because the face image is not adjusted.
Disclosure of Invention
It is an object of the present invention to provide a method, apparatus and system for augmented reality that overcomes some or all of the disadvantages of the prior art.
A method of augmented reality according to the present invention comprises:
receiving a first augmented reality request which is sent by a first terminal logged in by a first account and corresponds to a second account, wherein the first augmented reality request carries a first facial image set of a first person;
receiving a second augmented reality request which is sent by a second terminal logged in by a second account and corresponds to the first account, wherein the second augmented reality request carries a second facial image set of a second person;
performing model training based on the first facial image set, the second facial image set and the augmented reality parameter set to obtain a trained first augmented reality model and a trained second augmented reality model; the trained first augmented reality model is used for replacing the face image of the first person with the face image of the second person after adjustment, and the trained second augmented reality model is used for replacing the face image of the second person with the face image of the first person after adjustment;
the augmented reality parameter set includes parameters of a plurality of dimensions, the parameters including: skin color adjustment parameters, expression adjustment parameters, face shape adjustment parameters, angle adjustment parameters and decoration adjustment parameters;
and sending the trained first augmented reality model to a first terminal, and sending the trained second augmented reality model to a second terminal.
Preferably, the model training is performed based on the first face image set, the second face image set, and the augmented reality parameter set, so as to obtain a trained first augmented reality model and a trained second augmented reality model, and the method includes:
alternately acquiring facial images in a facial image set of a first person and a facial image set of a second person;
every time a face image of a first person is obtained, the face image of the first person is distorted to obtain a distorted face image of the first person, the distorted face image of the first person is input into a feature extraction model to obtain a first feature image, the first feature image is input into a second reduction model to obtain a first output image, the first output image is adjusted through enhancing a reality parameter set, and the feature extraction model and the second reduction model are updated on the basis of the currently obtained face image of the first person and the adjusted first output image;
every time a face image of a second person is obtained, the face image of the second person is distorted to obtain a distorted face image of the second person, the distorted face image of the second person is input into a feature extraction model to obtain a second feature image, the second feature image is input into a first reduction model to obtain a second output image, the second output image is adjusted through enhancing a reality parameter set, and the feature extraction model and the first reduction model are updated on the basis of the currently obtained face image of the second person and the adjusted second output image;
after parameter updating is carried out on the feature extraction model, the first reduction model and the second reduction model based on the face images of all the first characters and the face images of all the second characters, the trained first augmented reality model is determined based on the feature extraction model after the parameter updating and the first reduction model after the parameter updating, and the trained second augmented reality model is determined based on the feature extraction model after the parameter updating and the second reduction model after the parameter updating.
The invention provides a device for enhancing reality, which comprises:
the first receiving module is used for receiving a first augmented reality request which is sent by a first terminal logged in by a first account and corresponds to a second account, and the first augmented reality request carries a first facial image set of a first person;
the second receiving module is used for receiving a second augmented reality request which is sent by a second terminal logged in by a second account and corresponds to the first account, and the second augmented reality request carries a second facial image set of a second person;
the training module is used for carrying out model training based on the first face image set, the second face image set and the augmented reality parameter set to obtain a trained first augmented reality model and a trained second augmented reality model; the trained first augmented reality model is used for replacing the face image of the first person with the face image of the second person after adjustment, and the trained second augmented reality model is used for replacing the face image of the second person with the face image of the first person after adjustment; the augmented reality parameter set includes parameters of a plurality of dimensions, the parameters including: skin color adjustment parameters, expression adjustment parameters, face shape adjustment parameters, angle adjustment parameters and decoration adjustment parameters;
and the sending module is used for sending the trained first augmented reality model to a first terminal and sending the trained second augmented reality model to a second terminal.
The invention provides a system for augmented reality, which comprises: first terminal, second terminal and server, wherein:
the server receives a first augmented reality request which is sent by a first terminal logged in by a first account and corresponds to a second account, wherein the first augmented reality request carries a first facial image set of a first person; receiving a second augmented reality request which is sent by a second terminal logged in by a second account and corresponds to the first account, wherein the second augmented reality request carries a second facial image set of a second person; performing model training based on the first facial image set, the second facial image set and the augmented reality parameter set to obtain a trained first augmented reality model and a trained second augmented reality model; the trained first augmented reality model is used for replacing the face image of the first person with the face image of the second person after adjustment, and the trained second augmented reality model is used for replacing the face image of the second person with the face image of the first person after adjustment; the augmented reality parameter set includes parameters of a plurality of dimensions, the parameters including: skin color adjustment parameters, expression adjustment parameters, face shape adjustment parameters, angle adjustment parameters and decoration adjustment parameters; sending the trained first augmented reality model to a first terminal, and sending the trained second augmented reality model to a second terminal;
the first terminal sends an augmented reality request corresponding to the second account to the server; receiving a trained first augmented reality model sent by a server; when an instruction corresponding to a second account is received, inputting the first image into the trained first augmented reality model to obtain a second image after augmented reality;
the second terminal sends an augmented reality request corresponding to the first account to the server; receiving a trained second augmented reality model sent by a server; and when an instruction corresponding to the first account is received, inputting the second image into the trained second augmented reality model to obtain the first image after augmented reality.
The augmented reality model generated by the invention is used for processing the face image, the replaced second image can be obtained after the first image is input into the augmented reality model, and the pre-stored face image of the original person is not needed to be used for carrying out local image replacement on the face image of the target person, so that the deformation processing of the face image is not involved, and the image distortion can be reduced; the augmented reality parameter set is added in the training process, so that the skin color adjustment, the expression adjustment, the face shape adjustment, the angle adjustment and the decoration adjustment can be performed on the face image in the training process, and the effect is better.
Drawings
Fig. 1 is a flowchart of a method for augmented reality in embodiment 1.
Detailed Description
For a further understanding of the invention, reference should be made to the following detailed description taken in conjunction with the accompanying drawings and examples. It is to be understood that the examples are illustrative of the invention and not limiting.
Example 1
As shown in fig. 1, the present embodiment provides a method for augmented reality, which includes:
receiving a first augmented reality request which is sent by a first terminal logged in by a first account and corresponds to a second account, wherein the first augmented reality request carries a first facial image set of a first person;
receiving a second augmented reality request which is sent by a second terminal logged in by a second account and corresponds to the first account, wherein the second augmented reality request carries a second facial image set of a second person;
performing model training based on the first facial image set, the second facial image set and the augmented reality parameter set to obtain a trained first augmented reality model and a trained second augmented reality model; the trained first augmented reality model is used for replacing the face image of the first person with the face image of the second person after adjustment, and the trained second augmented reality model is used for replacing the face image of the second person with the face image of the first person after adjustment;
the augmented reality parameter set includes parameters of a plurality of dimensions, the parameters including: skin color adjustment parameters, expression adjustment parameters, face shape adjustment parameters, angle adjustment parameters and decoration adjustment parameters;
and sending the trained first augmented reality model to a first terminal, and sending the trained second augmented reality model to a second terminal.
Based on first face image set, second face image set and augmented reality parameter set, carry out model training, obtain the first augmented reality model after the training and the second augmented reality model after the training, include:
alternately acquiring facial images in a facial image set of a first person and a facial image set of a second person;
every time a face image of a first person is obtained, the face image of the first person is distorted to obtain a distorted face image of the first person, the distorted face image of the first person is input into a feature extraction model to obtain a first feature image, the first feature image is input into a second reduction model to obtain a first output image, the first output image is adjusted through enhancing a reality parameter set, and the feature extraction model and the second reduction model are updated on the basis of the currently obtained face image of the first person and the adjusted first output image;
every time a face image of a second person is obtained, the face image of the second person is distorted to obtain a distorted face image of the second person, the distorted face image of the second person is input into a feature extraction model to obtain a second feature image, the second feature image is input into a first reduction model to obtain a second output image, the second output image is adjusted through enhancing a reality parameter set, and the feature extraction model and the first reduction model are updated on the basis of the currently obtained face image of the second person and the adjusted second output image;
after parameter updating is carried out on the feature extraction model, the first reduction model and the second reduction model based on the face images of all the first characters and the face images of all the second characters, the trained first augmented reality model is determined based on the feature extraction model after the parameter updating and the first reduction model after the parameter updating, and the trained second augmented reality model is determined based on the feature extraction model after the parameter updating and the second reduction model after the parameter updating.
The embodiment provides an augmented reality device, which includes:
the first receiving module is used for receiving a first augmented reality request which is sent by a first terminal logged in by a first account and corresponds to a second account, and the first augmented reality request carries a first facial image set of a first person;
the second receiving module is used for receiving a second augmented reality request which is sent by a second terminal logged in by a second account and corresponds to the first account, and the second augmented reality request carries a second facial image set of a second person;
the training module is used for carrying out model training based on the first face image set, the second face image set and the augmented reality parameter set to obtain a trained first augmented reality model and a trained second augmented reality model; the trained first augmented reality model is used for replacing the face image of the first person with the face image of the second person after adjustment, and the trained second augmented reality model is used for replacing the face image of the second person with the face image of the first person after adjustment; the augmented reality parameter set includes parameters of a plurality of dimensions, the parameters including: skin color adjustment parameters, expression adjustment parameters, face shape adjustment parameters, angle adjustment parameters and decoration adjustment parameters;
and the sending module is used for sending the trained first augmented reality model to a first terminal and sending the trained second augmented reality model to a second terminal.
The present embodiment provides a system for augmented reality, which includes: first terminal, second terminal and server, wherein:
the server receives a first augmented reality request which is sent by a first terminal logged in by a first account and corresponds to a second account, wherein the first augmented reality request carries a first facial image set of a first person; receiving a second augmented reality request which is sent by a second terminal logged in by a second account and corresponds to the first account, wherein the second augmented reality request carries a second facial image set of a second person; performing model training based on the first facial image set, the second facial image set and the augmented reality parameter set to obtain a trained first augmented reality model and a trained second augmented reality model; the trained first augmented reality model is used for replacing the face image of the first person with the face image of the second person after adjustment, and the trained second augmented reality model is used for replacing the face image of the second person with the face image of the first person after adjustment; the augmented reality parameter set includes parameters of a plurality of dimensions, the parameters including: skin color adjustment parameters, expression adjustment parameters, face shape adjustment parameters, angle adjustment parameters and decoration adjustment parameters; sending the trained first augmented reality model to a first terminal, and sending the trained second augmented reality model to a second terminal;
the first terminal sends an augmented reality request corresponding to the second account to the server; receiving a trained first augmented reality model sent by a server; when an instruction corresponding to a second account is received, inputting the first image into the trained first augmented reality model to obtain a second image after augmented reality;
the second terminal sends an augmented reality request corresponding to the first account to the server; receiving a trained second augmented reality model sent by a server; and when an instruction corresponding to the first account is received, inputting the second image into the trained second augmented reality model to obtain the first image after augmented reality.
The augmented reality model generated in the embodiment is used for processing the face image, the replaced second image can be obtained after the first image is input into the augmented reality model, and the pre-stored face image of the original person is not needed to be used for carrying out local image replacement on the face image of the target person, so that the deformation processing on the face image is not involved, and the image distortion can be reduced; the augmented reality parameter set is added in the training process, so that the skin color adjustment, the expression adjustment, the face shape adjustment, the angle adjustment and the decoration adjustment can be performed on the face image in the training process, and the effect is better.
The present invention and its embodiments have been described above schematically, without limitation, and what is shown in the drawings is only one of the embodiments of the present invention, and the actual structure is not limited thereto. Therefore, if the person skilled in the art receives the teaching, without departing from the spirit of the invention, the person skilled in the art shall not inventively design the similar structural modes and embodiments to the technical solution, but shall fall within the scope of the invention.
Claims (4)
1. A method of augmented reality, comprising: the method comprises the following steps:
receiving a first augmented reality request which is sent by a first terminal logged in by a first account and corresponds to a second account, wherein the first augmented reality request carries a first facial image set of a first person;
receiving a second augmented reality request which is sent by a second terminal logged in by a second account and corresponds to the first account, wherein the second augmented reality request carries a second facial image set of a second person;
performing model training based on the first facial image set, the second facial image set and the augmented reality parameter set to obtain a trained first augmented reality model and a trained second augmented reality model; the trained first augmented reality model is used for replacing the face image of the first person with the face image of the second person after adjustment, and the trained second augmented reality model is used for replacing the face image of the second person with the face image of the first person after adjustment;
the augmented reality parameter set includes parameters of a plurality of dimensions, the parameters including: skin color adjustment parameters, expression adjustment parameters, face shape adjustment parameters, angle adjustment parameters and decoration adjustment parameters;
and sending the trained first augmented reality model to a first terminal, and sending the trained second augmented reality model to a second terminal.
2. The method of claim 1, wherein: based on first face image set, second face image set and augmented reality parameter set, carry out model training, obtain the first augmented reality model after the training and the second augmented reality model after the training, include:
alternately acquiring facial images in a facial image set of a first person and a facial image set of a second person;
every time a face image of a first person is obtained, the face image of the first person is distorted to obtain a distorted face image of the first person, the distorted face image of the first person is input into a feature extraction model to obtain a first feature image, the first feature image is input into a second reduction model to obtain a first output image, the first output image is adjusted through enhancing a reality parameter set, and the feature extraction model and the second reduction model are updated on the basis of the currently obtained face image of the first person and the adjusted first output image;
every time a face image of a second person is obtained, the face image of the second person is distorted to obtain a distorted face image of the second person, the distorted face image of the second person is input into a feature extraction model to obtain a second feature image, the second feature image is input into a first reduction model to obtain a second output image, the second output image is adjusted through enhancing a reality parameter set, and the feature extraction model and the first reduction model are updated on the basis of the currently obtained face image of the second person and the adjusted second output image;
after parameter updating is carried out on the feature extraction model, the first reduction model and the second reduction model based on the face images of all the first characters and the face images of all the second characters, the trained first augmented reality model is determined based on the feature extraction model after the parameter updating and the first reduction model after the parameter updating, and the trained second augmented reality model is determined based on the feature extraction model after the parameter updating and the second reduction model after the parameter updating.
3. An apparatus for augmented reality, comprising: the method comprises the following steps:
the first receiving module is used for receiving a first augmented reality request which is sent by a first terminal logged in by a first account and corresponds to a second account, and the first augmented reality request carries a first facial image set of a first person;
the second receiving module is used for receiving a second augmented reality request which is sent by a second terminal logged in by a second account and corresponds to the first account, and the second augmented reality request carries a second facial image set of a second person;
the training module is used for carrying out model training based on the first face image set, the second face image set and the augmented reality parameter set to obtain a trained first augmented reality model and a trained second augmented reality model; the trained first augmented reality model is used for replacing the face image of the first person with the face image of the second person after adjustment, and the trained second augmented reality model is used for replacing the face image of the second person with the face image of the first person after adjustment; the augmented reality parameter set includes parameters of a plurality of dimensions, the parameters including: skin color adjustment parameters, expression adjustment parameters, face shape adjustment parameters, angle adjustment parameters and decoration adjustment parameters;
and the sending module is used for sending the trained first augmented reality model to a first terminal and sending the trained second augmented reality model to a second terminal.
4. A system for augmented reality, comprising: the system comprises: first terminal, second terminal and server, wherein:
the server receives a first augmented reality request which is sent by a first terminal logged in by a first account and corresponds to a second account, wherein the first augmented reality request carries a first facial image set of a first person; receiving a second augmented reality request which is sent by a second terminal logged in by a second account and corresponds to the first account, wherein the second augmented reality request carries a second facial image set of a second person; performing model training based on the first facial image set, the second facial image set and the augmented reality parameter set to obtain a trained first augmented reality model and a trained second augmented reality model; the trained first augmented reality model is used for replacing the face image of the first person with the face image of the second person after adjustment, and the trained second augmented reality model is used for replacing the face image of the second person with the face image of the first person after adjustment; the augmented reality parameter set includes parameters of a plurality of dimensions, the parameters including: skin color adjustment parameters, expression adjustment parameters, face shape adjustment parameters, angle adjustment parameters and decoration adjustment parameters; sending the trained first augmented reality model to a first terminal, and sending the trained second augmented reality model to a second terminal;
the first terminal sends an augmented reality request corresponding to the second account to the server; receiving a trained first augmented reality model sent by a server; when an instruction corresponding to a second account is received, inputting the first image into the trained first augmented reality model to obtain a second image after augmented reality;
the second terminal sends an augmented reality request corresponding to the first account to the server; receiving a trained second augmented reality model sent by a server; and when an instruction corresponding to the first account is received, inputting the second image into the trained second augmented reality model to obtain the first image after augmented reality.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110804141.9A CN113487745A (en) | 2021-07-16 | 2021-07-16 | Method, device and system for enhancing reality |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110804141.9A CN113487745A (en) | 2021-07-16 | 2021-07-16 | Method, device and system for enhancing reality |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113487745A true CN113487745A (en) | 2021-10-08 |
Family
ID=77939728
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110804141.9A Pending CN113487745A (en) | 2021-07-16 | 2021-07-16 | Method, device and system for enhancing reality |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113487745A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR101773885B1 (en) * | 2016-10-19 | 2017-09-01 | (주)잼투고 | A method and server for providing augmented reality objects using image authentication |
| CN110533585A (en) * | 2019-09-04 | 2019-12-03 | 广州华多网络科技有限公司 | A kind of method, apparatus that image is changed face, system, equipment and storage medium |
| CN110741377A (en) * | 2017-06-30 | 2020-01-31 | Oppo广东移动通信有限公司 | Face image processing method and device, storage medium and electronic equipment |
| CN110969572A (en) * | 2019-11-29 | 2020-04-07 | 广州华多网络科技有限公司 | Face changing model training method, face exchanging device and electronic equipment |
| CN112102153A (en) * | 2020-08-20 | 2020-12-18 | 北京百度网讯科技有限公司 | Cartoon processing method and device for image, electronic equipment and storage medium |
| CN112837213A (en) * | 2021-02-07 | 2021-05-25 | 北京字跳网络技术有限公司 | Face shape adjustment image generation method, model training method, device and equipment |
-
2021
- 2021-07-16 CN CN202110804141.9A patent/CN113487745A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR101773885B1 (en) * | 2016-10-19 | 2017-09-01 | (주)잼투고 | A method and server for providing augmented reality objects using image authentication |
| CN110741377A (en) * | 2017-06-30 | 2020-01-31 | Oppo广东移动通信有限公司 | Face image processing method and device, storage medium and electronic equipment |
| CN110533585A (en) * | 2019-09-04 | 2019-12-03 | 广州华多网络科技有限公司 | A kind of method, apparatus that image is changed face, system, equipment and storage medium |
| CN110969572A (en) * | 2019-11-29 | 2020-04-07 | 广州华多网络科技有限公司 | Face changing model training method, face exchanging device and electronic equipment |
| CN112102153A (en) * | 2020-08-20 | 2020-12-18 | 北京百度网讯科技有限公司 | Cartoon processing method and device for image, electronic equipment and storage medium |
| CN112837213A (en) * | 2021-02-07 | 2021-05-25 | 北京字跳网络技术有限公司 | Face shape adjustment image generation method, model training method, device and equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110503703B (en) | Method and apparatus for generating image | |
| CN107578017B (en) | Method and apparatus for generating image | |
| CN110602554B (en) | Cover image determining method, device and equipment | |
| CN108200446B (en) | On-line multimedia interaction system and method of virtual image | |
| US20180204562A1 (en) | Method and device for image recognition | |
| US20130242127A1 (en) | Image creating device and image creating method | |
| CN109784304B (en) | Method and apparatus for labeling dental images | |
| CN104410923A (en) | Animation presentation method and device based on video chat room | |
| CN107835397B (en) | Multi-lens video synchronization method | |
| CN109302628A (en) | A kind of face processing method based on live streaming, device, equipment and storage medium | |
| CN111464827A (en) | Data processing method and device, computing equipment and storage medium | |
| CN115209180A (en) | Video generation method and device | |
| CN110570862A (en) | voice recognition method and intelligent voice engine device | |
| JP7218786B2 (en) | Image processing device, image processing method and program | |
| CN113487745A (en) | Method, device and system for enhancing reality | |
| CN108320331B (en) | Method and equipment for generating augmented reality video information of user scene | |
| CN110796689B (en) | Video processing method, electronic device and storage medium | |
| CN108055423B (en) | Multi-lens video synchronization offset calculation method | |
| CN115052197A (en) | Virtual portrait video generation method and device | |
| CN116433809A (en) | Expression driving method and model training method | |
| CN114170379A (en) | Three-dimensional model reconstruction method, device and equipment | |
| Zeng et al. | Ultra-low bit rate facial coding hybrid model based on saliency detection | |
| CN111583420A (en) | Intelligent learning system and method based on augmented reality mode | |
| CN112712474B (en) | Perspective correction method and system for video streaming images | |
| CN116126150B (en) | Simulated driving system and method based on live-action interaction |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |