CN115222899B - Virtual digital human generation method, system, computer device and storage medium - Google Patents
Virtual digital human generation method, system, computer device and storage medium Download PDFInfo
- Publication number
- CN115222899B CN115222899B CN202211149657.5A CN202211149657A CN115222899B CN 115222899 B CN115222899 B CN 115222899B CN 202211149657 A CN202211149657 A CN 202211149657A CN 115222899 B CN115222899 B CN 115222899B
- Authority
- CN
- China
- Prior art keywords
- virtual
- portrait
- client
- model data
- digital
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/172—Classification, e.g. identification
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Human Computer Interaction (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- Processing Or Creating Images (AREA)
Abstract
The scheme relates to a virtual digital person generation method, a virtual digital person generation system, computer equipment and a storage medium. The method comprises the following steps: the client sends image data containing the portrait to the server through the SDK packet; a digital human virtual engine in a server segments a target portrait in image data through an image segmentation recognition algorithm, obtains 3D point cloud information corresponding to the target portrait through a mapping projection matrix and a face recognition algorithm, converts the 3D point cloud information into model data in a graphic language transmission format and sends the model data to a client; and the client analyzes the model data through the SDK packet and renders the model data into a character model to generate a virtual digital person. The client side sends image data containing the portrait to the digital human virtual engine through the SDK packet after collecting the image data, converts traditional virtual digital human portrait pictures into three-dimensional models, and transmits the three-dimensional models through a graphic language transmission format, so that the three-dimensional portrait models can be generated quickly and are integrated into a plurality of application scenes, and the user experience is improved.
Description
Technical Field
The invention relates to the technical field of virtual image construction, in particular to a virtual digital human generation method, a virtual digital human generation system, computer equipment and a storage medium.
Background
With the development of science and technology, virtual simulation technology is more mature, and cartoon figure images are generated on line according to face recognition results, so that more scene applications are obtained. However, the generation of the cartoon character image has the problems of poor interactive experience, too high technical implementation period and cost, and too low continuous interaction of the user, so that the virtual digital person can be produced at the discretion. The virtual digital human is a virtual character with a digitalized appearance, different from a robot with an entity, the virtual digital human exists depending on display equipment, and a plurality of virtual humans can be displayed through equipment such as a mobile phone, a computer or an intelligent large screen. The virtual digital human system is generally composed of modules such as a human image, voice generation, animation generation, audio-video synthesis display, interaction and the like, and the generation of a virtual digital human image picture through picture recognition is a popular use technology at present.
However, the traditional method for making the virtual digital person needs to make a character model through various software and a large amount of manual work, the making period is long, the cost is too high, the person model can be used only by debugging in a scene, and secondary editing and development are difficult to support; the traditional method for generating the virtual digital human through the picture recognition cannot be used in the three-dimensional model, and the experience and interaction effect cannot reach the expectation of a customer. Therefore, the virtual digital person generated by the traditional generation method has the problem of poor customer experience.
Disclosure of Invention
In order to solve the above technical problems, a virtual digital person generation method, a virtual digital person generation system, a computer device, and a storage medium are provided, which can improve the user experience.
A virtual digital person generation method, the method comprising:
the method comprises the steps that a client side collects image data containing human images, and the image data are sent to a server through an SDK (software development kit);
a digital human virtual engine in the server segments a target portrait in the image data through an image segmentation recognition algorithm, and processes the target portrait through a mapping projection matrix and a face recognition algorithm to obtain 3D point cloud information corresponding to the target portrait;
a digital human virtual engine in the server converts the 3D point cloud information into model data in a graphic language transmission format and sends the model data to the client;
and the client analyzes the model data through the SDK packet, renders the model data into a character model, and generates a virtual digital person.
In one embodiment, the digital human virtual engine in the server segments the target human figure in the image data through an image segmentation recognition algorithm, including:
the digital human virtual engine separates a portrait, an object and a background in the image data through an image segmentation recognition algorithm, and takes the separated portrait as a target portrait;
and when a plurality of separated portraits exist, sequentially separating each portrait by the digital human virtual engine through the image segmentation recognition algorithm, and taking each portrait as the target portrait.
In one embodiment, the method further comprises:
and the digital human virtual engine carries out enhancement and restoration processing on the separated portrait to obtain the target portrait.
In one embodiment, the client parses the model data through an SDK package and renders into a character model to generate a virtual digital person, including:
the client side obtains a user-defined instruction and obtains a basic virtual component from the SDK package according to the user-defined instruction;
the client analyzes the model data through the SDK packet to obtain analyzed model data;
and the client renders a character model according to the analyzed model data and the basic virtual component to generate the virtual digital person.
In one embodiment, the method further comprises:
when the client side obtains the user-defined instruction, obtaining a user-defined picture in real time until the virtual digital person is generated;
and the client uploads each user-defined picture and the generated virtual digital person to a copyright block chain center for storage.
In one embodiment, the basic virtual components comprise a character virtual component and an environment virtual component.
A virtual digital person generation system, the system comprising:
the client is used for collecting image data containing human images and sending the image data to the server through the SDK packet;
the server is used for segmenting a target portrait in the image data by using an image segmentation recognition algorithm through a digital human virtual engine, and processing the target portrait through a mapping projection matrix and a face recognition algorithm to obtain 3D point cloud information corresponding to the target portrait;
the server is also used for converting the 3D point cloud information into model data in a graphic language transmission format through a digital human virtual engine and sending the model data to the client;
and the client is also used for analyzing the model data through the SDK packet, rendering the model data into a character model and generating a virtual digital person.
A computer device comprising a memory storing a computer program and a processor implementing the steps of the method when executing the computer program.
A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the above-mentioned method.
According to the virtual digital human generation method, the virtual digital human generation system, the computer equipment and the storage medium, image data containing human images are collected through the client, and the image data are sent to the server through the SDK packet; a digital human virtual engine in the server segments a target portrait in the image data through an image segmentation recognition algorithm, and processes the target portrait through a mapping projection matrix and a face recognition algorithm to obtain 3D point cloud information corresponding to the target portrait; a digital human virtual engine in the server converts the 3D point cloud information into model data in a graphic language transmission format and sends the model data to the client; and the client analyzes the model data through the SDK packet, renders the model data into a character model, and generates a virtual digital person. The client side collects image data containing the portrait and then sends the image data to the server through the SDK packet, a digital human virtual engine on the server conducts portrait segmentation recognition and generates corresponding 3D point cloud information, a traditional virtual digital human figure picture is converted into a three-dimensional model, transmission is conducted through a graphic language transmission format, the three-dimensional figure model can be rapidly generated in multiple platforms or software under the condition of low codes or no codes, the three-dimensional figure model is integrated into multiple application scenes, and the user experience is improved.
Drawings
FIG. 1 is a diagram of an application environment of a virtual digital person generation method and a block diagram of a virtual digital person generation system in one embodiment;
FIG. 2 is a schematic flow chart diagram of a method for virtual digital person generation in one embodiment;
FIG. 3 is a diagram illustrating an internal structure of a computer device according to an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of and not restrictive on the broad application.
The virtual digital person generation method provided by the embodiment of the application can be applied to the application environment shown in fig. 1. As shown in fig. 1, the application environment includes a client 110 and a server 120, and the client 110 can communicate with the server 120 and transmit data. The client 110 collects image data containing a portrait and sends the image data to the server 120 through an SDK package; a digital human virtual engine in the server 120 segments a target portrait in image data through an image segmentation recognition algorithm, and processes the target portrait through a mapping projection matrix and a face recognition algorithm to obtain 3D point cloud information corresponding to the target portrait; a digital human virtual engine in the server 120 converts the 3D point cloud information into model data in a graphic language transmission format and sends the model data to the client 110; the client 110 parses the model data through the SDK package and renders it into a character model, generating a virtual digital person.
In one embodiment, as shown in fig. 2, there is provided a virtual digital person generation method, comprising the steps of:
The client can be a personal computer, a notebook computer, a smart phone, a robot, a tablet computer and other devices. The image data containing the portrait can be a photo containing the portrait selected by the user through the client, or a photo containing the portrait shot by the user through a camera on the client in real time. The image data may include one portrait or a plurality of portraits.
The SDK package can be pre-integrated, and after the client acquires image data containing the portrait, the image data can be sent to the server through the SDK package.
And 204, segmenting a target portrait in the image data by a digital human virtual engine in the server through an image segmentation and recognition algorithm, and processing the target portrait through a mapping projection matrix and a face recognition algorithm to obtain 3D point cloud information corresponding to the target portrait.
The server can be provided with a digital human virtual engine, the client can send the image data to the digital human virtual engine in the server through the SDK packet, and the digital human virtual engine divides the target human image in the image data through an image division recognition algorithm. Specifically, the digital human virtual engine can separate human images, objects, backgrounds and the like through an image segmentation recognition algorithm.
After the digital human virtual engine segments the target portrait, the 3D point cloud information of the head portrait of the target portrait can be generated by using a mapping projection matrix corresponding to the 2D image for intelligent learning and fusing an AI face recognition algorithm.
Step 206, the digital human virtual engine in the server converts the 3D point cloud information into model data in a graphic language transmission format, and sends the model data to the client.
The digital human virtual engine converts the 3D point cloud information into model data of a GLTF in a graphic language transmission format, and the converted GLTF model data can comprise character model data, skeleton data for describing a character model motion form transformation relation, skin mapping materials, animation sequences based on the character model skeleton data and the like. The digital human virtual engine can uniformly output the formatted GLTF model data to the client.
And step 208, the client analyzes the model data through the SDK packet, renders the model data into a character model, and generates a virtual digital person.
In the embodiment, the client acquires image data containing a portrait, and sends the image data to the server through the SDK packet; a digital human virtual engine in a server segments a target portrait in image data through an image segmentation recognition algorithm, and processes the target portrait through a mapping projection matrix and a face recognition algorithm to obtain 3D point cloud information corresponding to the target portrait; a digital human virtual engine in the server converts the 3D point cloud information into model data in a graphic language transmission format and sends the model data to a client; and the client analyzes the model data through the SDK packet, renders the model data into a character model, and generates a virtual digital person. The client side collects image data containing the portrait and then sends the image data to the server through the SDK packet, a digital human virtual engine on the server conducts portrait segmentation recognition and generates corresponding 3D point cloud information, a traditional virtual digital human figure picture is converted into a three-dimensional model, transmission is conducted through a graphic language transmission format, the three-dimensional figure model can be rapidly generated in multiple platforms or software under the condition of low codes or no codes, the three-dimensional figure model is integrated into multiple application scenes, and the user experience is improved.
In one embodiment, the provided method for generating a virtual digital person may further include a process of segmenting the image, where the specific process includes: the digital human virtual engine separates the human image, the object and the background in the image data through an image segmentation recognition algorithm, and takes the separated human image as a target human image; when a plurality of separated portraits exist, the digital human virtual engine sequentially separates each portrait through an image segmentation recognition algorithm, and each portrait is taken as a target portrait.
The digital human virtual engine separates the human image from the object and the background through an image segmentation recognition algorithm, so that the accuracy of subsequently generating the virtual digital human can be improved; and when a plurality of portraits exist, the digital man virtual engine can start the multitask model, sequentially separate the portraits, ensure that the portraits in the images do not have omission, and further generate a plurality of virtual digital men.
In one embodiment, the provided method for generating a virtual digital person may further include a process of processing the separated portrait, and the specific process includes: and the digital human virtual engine performs enhancement and restoration processing on the separated portrait to obtain a target portrait.
After the portrait is separated, the digital portrait virtual engine can scan, beautify, enhance and repair the separated portrait one by one, so that the target portrait is obtained.
In one embodiment, the provided method for generating a virtual digital person may further include a user-defined process, where the specific process includes: the client side obtains a custom instruction and obtains a basic virtual component from the SDK package according to the custom instruction; the client analyzes the model data through the SDK packet to obtain analyzed model data; and rendering the character model by the client according to the analyzed model data and the basic virtual component to generate a virtual digital person.
The SDK package may include basic virtual components, such as a skeleton of a virtual character model, a plurality of body movements, basic virtual components of a facial expression package, clothing, hair, and scenes, and basic virtual components of hair, scenes, and clothing.
A user can trigger a custom instruction through the client, so that custom setting is carried out on the character model. Specifically, the user can select the basic virtual component through the client, and the client can render the character model according to the analyzed model data and the basic virtual component selected by the user, so that the virtual digital person is generated. In this embodiment, the client can use the GPU resources to render the customized character model in the specified scene in real-time and quickly. The client can also generate a link URL which can be accessed according to the virtual digital person information, namely, a digital person virtual engine in the server can push the virtual digital person to the client in a video stream mode.
In this embodiment, after the virtual digital person is generated on the client, the virtual digital person can be fused in a preset scene for application.
Because the user can carry out self-defined setting on the virtual digital person, the personalized requirements of the user are met, and the experience of the user is improved.
In one embodiment, the provided method for generating a virtual digital person may further include storing a process of making the picture, where the specific process includes: when the client side obtains the user-defined instruction, obtaining a user-defined picture in real time until a virtual digital person is generated; and the client uploads each user-defined picture and the generated virtual digital person to a copyright block chain center for storage.
After the client side obtains the user-defined instruction, a user-defined picture can be obtained in real time, and the user-defined picture can contain a picture of a user-defined virtual digital person. The client can upload each generated self-defined picture and the virtual digital person to a copyright block chain center for storage through a hash algorithm.
In one embodiment, the base virtual components include a character virtual component, an environment virtual component. Wherein, the character virtual component can comprise: human skeleton, several limbs movements, facial expression bag, dress, hair, scene, etc; the environment virtual component may include: hair, scenes, clothing, etc.
In one embodiment, the digital human virtual engine in the server transmits the model data converted into the graphic language transmission format to the client side in an external interface mode.
In one embodiment, an exemplary application of a method for generating a virtual digital person is as follows:
individual users for social dawns: a user uploads or shoots photos through a mobile phone, and after a high-realistic virtual digital person is generated on line, the user uses a surface capture device or a motion capture device to perform skeleton binding with the high-realistic virtual digital person; the client platform provides a plurality of scenes and virtual component selections such as special effects, clothes, hair, ornaments and the like, personal image quality and overall atmosphere are improved, real-time rendering is unified by the server, and the effect presentation of the client is realized in a video stream pushing mode. In the live broadcast industry, by generating a dedicated man-to-man sharing link, virtual scenes, clothes, hair, ornaments and the like can be switched according to live broadcast contents, so that the live broadcast efficiency is improved, and audiences are more attractive.
For teams or individuals with technical development capabilities: after online applying for a service provider account, uploading the developed virtual asset components to a client platform, integrating the uploaded assets in an SDK (software development kit) package of the client platform to provide personalized customized services for a client user, and providing component sharing for a service provider by the platform by using the assets of the purchase service provider.
When people are repaired, people in group photo are repaired and the high-fidelity restoration of the online virtual digital people is carried out, the base image environment in the photo is separated, people in the photo are processed in batch to generate the high-reality virtual digital people through natural light and enhanced repairing technology of the scene, and the interaction of chatting, hugging, holding, calling and the like is carried out in the scene environment of the photo.
It should be understood that, although the steps in the above-described flowcharts are shown in sequence as indicated by the arrows, the steps are not necessarily performed in sequence as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a part of the steps in the above flowcharts may include multiple sub-steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of performing the sub-steps or the stages is not necessarily performed in sequence, but may be performed alternately or alternately with other steps or at least a part of the sub-steps or the stages of other steps.
In one embodiment, as shown in fig. 1, there is provided a virtual digital person generation system, including: a client 110 and a server 120, wherein:
a client 110, configured to collect image data including a portrait, and send the image data to the server 120 through an SDK package;
the server 120 is used for segmenting a target portrait in the image data by using an image segmentation and recognition algorithm through a digital human virtual engine, and processing the target portrait through a mapping projection matrix and a face recognition algorithm to obtain 3D point cloud information corresponding to the target portrait;
the server 120 is further configured to convert the 3D point cloud information into model data in a graphic language transmission format through a digital human virtual engine, and send the model data to the client 110;
the client 110 is further configured to parse the model data through the SDK package, render the model data into a character model, and generate a virtual digital person.
In one embodiment, the server 120 is further configured to separate the portrait, the object, and the background in the image data through an image segmentation recognition algorithm by the digital human virtual engine, and use the separated portrait as the target portrait; when a plurality of separated portraits exist, the digital human virtual engine sequentially separates each portrait through an image segmentation recognition algorithm, and each portrait is taken as a target portrait.
In one embodiment, the server 120 is further configured to perform an enhancement and restoration process on the separated portrait through a digital human virtual engine to obtain a target portrait.
In one embodiment, the client 110 is further configured to obtain a custom instruction, and obtain a basic virtual component from the SDK package according to the custom instruction; analyzing the model data through the SDK packet to obtain analyzed model data; rendering the character model according to the analyzed model data and the basic virtual assembly to generate a virtual digital person.
In one embodiment, the client 110 is further configured to, when obtaining the custom instruction, obtain a custom picture in real time until generating a virtual digital person; and uploading each user-defined picture and the generated virtual digital person to a copyright block chain center for storage.
In one embodiment, the base virtual components include a character virtual component, an environment virtual component.
In one embodiment, the server 120 is further configured to send the model data converted into the graphic language transmission format to the client 110 through the digital human virtual engine via an external interface.
In one embodiment, a computer device is provided, which may be a client or a server, and its internal structure diagram may be as shown in fig. 3. The computer device comprises a processor, a memory, a network interface, a display screen and an input device which are connected through a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium, an internal memory and a video memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program to run on the non-volatile storage medium. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a virtual digital person generation method. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on a shell of the computer equipment, an external keyboard, a touch pad or a mouse and the like.
It will be appreciated by those skilled in the art that the configuration shown in fig. 3 is a block diagram of only a portion of the configuration associated with the present application, and is not intended to limit the computing device to which the present application may be applied, and that a particular computing device may include more or fewer components than shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, a computer device is provided, comprising a memory having a computer program stored therein and a processor that implements the steps of the above method when executing the computer program.
In an embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which computer program, when being executed by a processor, carries out the steps of the above-mentioned method.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above may be implemented by hardware instructions of a computer program, which may be stored in a non-volatile computer-readable storage medium, and when executed, may include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), rambus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is specific and detailed, but not to be understood as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, and these are all within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (9)
1. A method of virtual digital person generation, the method comprising:
the method comprises the steps that a client side collects image data containing human images and sends the image data to a server through an SDK (software development kit) packet;
a digital human virtual engine in the server separates a portrait, an object and a background in the image data through an image segmentation recognition algorithm, and takes the separated portrait as a target portrait; when a plurality of separated portraits exist, the digital human virtual engine sequentially separates each portrait through the image segmentation recognition algorithm and takes each portrait as the target portrait;
processing the target portrait through a mapping projection matrix and a face recognition algorithm to obtain 3D point cloud information corresponding to the target portrait;
a digital human virtual engine in the server converts the 3D point cloud information into model data in a graphic language transmission format and sends the model data to the client;
and the client analyzes the model data through the SDK packet, renders the model data into a character model, and generates a virtual digital person.
2. The virtual digital person generation method of claim 1, further comprising:
and the digital human virtual engine performs enhancement and restoration processing on the separated human image to obtain the target human image.
3. The virtual digital person generation method according to claim 1, wherein the client parses the model data through an SDK package and renders into a character model to generate a virtual digital person, and comprises:
the client side obtains a user-defined instruction and obtains a basic virtual component from the SDK package according to the user-defined instruction;
the client analyzes the model data through the SDK packet to obtain analyzed model data;
and the client renders a character model according to the analyzed model data and the basic virtual assembly to generate the virtual digital person.
4. The virtual digital human generation method of claim 3, wherein the method further comprises:
when the client side obtains the user-defined instruction, obtaining a user-defined picture in real time until the virtual digital person is generated;
and the client uploads each user-defined picture and the generated virtual digital person to a copyright block chain center for storage.
5. The virtual digital person generation method of claim 3, wherein the base virtual component comprises a character virtual component, an environment virtual component.
6. The virtual digital human generation method of claim 1, wherein the digital human virtual engine in the server transmits the model data converted into the graphic language transmission format to the client by means of an external interface.
7. A virtual digital person generation system, the system comprising:
the client is used for acquiring image data containing a portrait and sending the image data to the server through the SDK packet;
the server is used for separating the portrait, the object and the background in the image data by using an image segmentation recognition algorithm through a digital human virtual engine, and taking the separated portrait as a target portrait; when a plurality of separated portraits exist, the digital human virtual engine sequentially separates each portrait through the image segmentation recognition algorithm and takes each portrait as the target portrait; processing the target portrait through a mapping projection matrix and a face recognition algorithm to obtain 3D point cloud information corresponding to the target portrait;
the server is also used for converting the 3D point cloud information into model data in a graphic language transmission format through a digital human virtual engine and sending the model data to the client;
and the client is also used for analyzing the model data through the SDK packet, rendering the model data into a character model and generating a virtual digital person.
8. A computer device comprising a memory and a processor, the memory storing a computer program, wherein the processor implements the steps of the method of any one of claims 1 to 6 when executing the computer program.
9. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211149657.5A CN115222899B (en) | 2022-09-21 | 2022-09-21 | Virtual digital human generation method, system, computer device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211149657.5A CN115222899B (en) | 2022-09-21 | 2022-09-21 | Virtual digital human generation method, system, computer device and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115222899A CN115222899A (en) | 2022-10-21 |
| CN115222899B true CN115222899B (en) | 2023-02-21 |
Family
ID=83616967
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211149657.5A Active CN115222899B (en) | 2022-09-21 | 2022-09-21 | Virtual digital human generation method, system, computer device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115222899B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115761141B (en) * | 2022-11-30 | 2023-07-07 | 灵瞳智能科技(北京)有限公司 | Virtual digital human three-dimensional model acquisition device |
| CN115880400A (en) * | 2022-12-28 | 2023-03-31 | 中国电信股份有限公司 | Cartoon digital human image generation method and device, electronic equipment and medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108510437A (en) * | 2018-04-04 | 2018-09-07 | 科大讯飞股份有限公司 | A kind of virtual image generation method, device, equipment and readable storage medium storing program for executing |
| CN112819971A (en) * | 2021-01-26 | 2021-05-18 | 北京百度网讯科技有限公司 | Method, device, equipment and medium for generating virtual image |
| CN113223137A (en) * | 2021-05-13 | 2021-08-06 | 广州虎牙科技有限公司 | Generation method of perspective projection human face point cloud graph, application program and electronic equipment |
| WO2021223724A1 (en) * | 2020-05-08 | 2021-11-11 | 维沃移动通信有限公司 | Information processing method and apparatus, and electronic device |
| CN114723860A (en) * | 2022-06-08 | 2022-07-08 | 深圳智华科技发展有限公司 | Method, device and equipment for generating virtual image and storage medium |
| CN114782645A (en) * | 2022-03-11 | 2022-07-22 | 科大讯飞(苏州)科技有限公司 | Virtual digital person making method, related equipment and readable storage medium |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP6671852B2 (en) * | 2015-03-24 | 2020-03-25 | 日本原子力防護システム株式会社 | Information setting system and simulation system |
| US10573091B2 (en) * | 2017-02-22 | 2020-02-25 | Andre R. Vincelette | Systems and methods to create a virtual object or avatar |
| JP7376496B2 (en) * | 2018-03-20 | 2023-11-08 | ピーシーエムエス ホールディングス インコーポレイテッド | System and method for optimizing dynamic point clouds based on prioritized transformations |
| CN109949422B (en) * | 2018-10-15 | 2020-12-15 | 华为技术有限公司 | Data processing method and equipment for virtual scene |
| US11380037B2 (en) * | 2019-10-30 | 2022-07-05 | Tencent Technology (Shenzhen) Company Limited | Method and apparatus for generating virtual operating object, storage medium, and electronic device |
| CN114332374A (en) * | 2021-12-30 | 2022-04-12 | 深圳市慧鲤科技有限公司 | Virtual display method, equipment and storage medium |
-
2022
- 2022-09-21 CN CN202211149657.5A patent/CN115222899B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108510437A (en) * | 2018-04-04 | 2018-09-07 | 科大讯飞股份有限公司 | A kind of virtual image generation method, device, equipment and readable storage medium storing program for executing |
| WO2021223724A1 (en) * | 2020-05-08 | 2021-11-11 | 维沃移动通信有限公司 | Information processing method and apparatus, and electronic device |
| CN112819971A (en) * | 2021-01-26 | 2021-05-18 | 北京百度网讯科技有限公司 | Method, device, equipment and medium for generating virtual image |
| CN113223137A (en) * | 2021-05-13 | 2021-08-06 | 广州虎牙科技有限公司 | Generation method of perspective projection human face point cloud graph, application program and electronic equipment |
| CN114782645A (en) * | 2022-03-11 | 2022-07-22 | 科大讯飞(苏州)科技有限公司 | Virtual digital person making method, related equipment and readable storage medium |
| CN114723860A (en) * | 2022-06-08 | 2022-07-08 | 深圳智华科技发展有限公司 | Method, device and equipment for generating virtual image and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| OpenVINO实时人脸表面3D点云提取;OpenCV学堂;《https://cloud.tencent.com/developer/article/1821004?from=article.detail.1906215》;20210507;第1页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115222899A (en) | 2022-10-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11861936B2 (en) | Face reenactment | |
| CN115222899B (en) | Virtual digital human generation method, system, computer device and storage medium | |
| US11062494B2 (en) | Electronic messaging utilizing animatable 3D models | |
| US9626788B2 (en) | Systems and methods for creating animations using human faces | |
| CN113272870A (en) | System and method for realistic real-time portrait animation | |
| CN113240782A (en) | Streaming media generation method and device based on virtual role | |
| CN109242940B (en) | Method and device for generating three-dimensional dynamic image | |
| CN111182350B (en) | Image processing method, device, terminal equipment and storage medium | |
| KR102353556B1 (en) | Apparatus for Generating Facial expressions and Poses Reappearance Avatar based in User Face | |
| CN112866741B (en) | Gift animation effect display method and system based on 3D face animation reconstruction | |
| US12112418B2 (en) | Volumetric capture and mesh-tracking based machine learning 4D face/body deformation training | |
| US20160086365A1 (en) | Systems and methods for the conversion of images into personalized animations | |
| CN113453027B (en) | Live video and virtual make-up image processing method and device and electronic equipment | |
| CN111147766A (en) | Special effect video synthesis method and device, computer equipment and storage medium | |
| CN115331265A (en) | Training method of posture detection model and driving method and device of digital person | |
| KR20160134883A (en) | Digital actor managing method for image contents | |
| CN115984447A (en) | Image rendering method, device, equipment and medium | |
| US11741650B2 (en) | Advanced electronic messaging utilizing animatable 3D models | |
| CN108876921B (en) | Three-dimensional decorating model processing method and device, computer equipment and storage medium | |
| WO2024066549A1 (en) | Data processing method and related device | |
| KR101780496B1 (en) | Method for producing 3D digital actor image based on character modelling by computer graphic tool | |
| CN114422862A (en) | Service video generation method, device, equipment, storage medium and program product | |
| Huang et al. | A process for the semi-automated generation of life-sized, interactive 3D character models for holographic projection | |
| CN113908553A (en) | Game character expression generation method and device, electronic equipment and storage medium | |
| CN117097919A (en) | Virtual character rendering method, apparatus, device, storage medium, and program product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |