WO2023241289A1

WO2023241289A1 - Method and device for generating virtual reality service video, and storage medium

Info

Publication number: WO2023241289A1
Application number: PCT/CN2023/094580
Authority: WO
Inventors: 甘仔斌
Original assignee: 中兴通讯股份有限公司
Priority date: 2022-06-13
Filing date: 2023-05-16
Publication date: 2023-12-21
Also published as: CN117274974A

Abstract

Embodiments of the present disclosure provide a method for generating a virtual reality service video, comprising: extracting 3D point cloud data in a service video, wherein the service video is a video formed by a customer service subject performing a customer service; determining, according to an optical flow parameter of the service video, the customer service subject from subjects comprised in the service video; extracting point cloud data of the customer service subject from the 3D point cloud data, and determining a subject pose parameter of the customer service subject; transforming a reference subject model according to the subject pose parameter, and substituting point cloud data of the transformed reference subject model into the 3D point cloud data to form a reference video, wherein the reference subject model is a virtual reality customer service corresponding to the customer service subject; and adjusting the reference video until an optical flow parameter of the reference video meets a preset condition, so as to obtain a virtual reality service video corresponding to the customer service subject.

Description

Virtual reality service video generation method, device and storage medium

Technical field

Embodiments of the present disclosure relate to the field of communications, and specifically, to a method, device, and storage medium for generating virtual reality service videos.

Background technique

The contact center in each system aims to allocate professionally trained customer service personnel to provide users with corresponding services, such as business inquiries, new business processing, reporting and complaints, and other services. Customer service staff usually provide corresponding services to users through audio and video. However, in this service method, like offline counters, one customer service staff can only provide services to one user at the same time, and there is still the problem of low service efficiency.

With the rapid development of virtual reality (VR) technology and 5G networks, more and more VR technologies have been applied in 5G networks. If VR technology is applied to customer service, service efficiency can be improved through VR customer service. In order to make VR customer service services more realistic, it is necessary to convert VR virtual videos based on videos of customer service personnel providing services to users.

Contents of the invention

Embodiments of the present disclosure provide a method, device, and storage medium for generating virtual reality service videos to at least solve the problem of low service efficiency of online video services in related technologies.

According to an embodiment of the present disclosure, a method for generating a virtual reality service video is provided, including: extracting 3D point cloud data in the service video, wherein the above-mentioned service video is a video formed by a customer service object performing customer service service; according to the above-mentioned service According to the optical flow parameters of the video, the above-mentioned customer service object is determined from each object included in the above-mentioned service video; the point cloud data of the above-mentioned customer service object is extracted from the above-mentioned 3D point cloud data, and the object pose parameters of the above-mentioned customer service object are determined; according to The above-mentioned object pose parameters convert the reference object model, and the point cloud data of the converted reference object model is brought into the above-mentioned 3D point cloud data to form a reference video, where the above-mentioned reference object model is the virtual reality customer service corresponding to the above-mentioned customer service object. ; Adjust the above-mentioned reference video until the optical flow parameters of the above-mentioned reference video meet the preset conditions, and obtain the virtual reality service video corresponding to the above-mentioned customer service object.

According to another embodiment of the present disclosure, a device for generating a virtual reality service video is provided, including: an extraction module configured to extract 3D point cloud data in the service video, wherein the above-mentioned service video is formed for a customer service object performing customer service. video; the determination module is configured to determine the above-mentioned customer service object from each object included in the above-mentioned service video according to the optical flow parameters of the above-mentioned service video; the pose module is configured to extract the above-mentioned customer service object from the above-mentioned 3D point cloud data point cloud data, and determine the object pose parameters of the above-mentioned customer service object; the conversion module is configured to convert the reference object model according to the above-mentioned object pose parameters, and bring the point cloud data of the converted reference object model into the above-mentioned 3D The point cloud data forms a reference video, wherein the reference object model is the virtual reality customer service corresponding to the customer service object; the adjustment module is configured to adjust the reference video until the optical flow parameters of the reference video meet the preset conditions, and obtain the same as the customer service object. Corresponding virtual reality service video.

According to yet another embodiment of the present disclosure, a computer-readable storage medium is also provided, the computer-readable storage medium A computer program is stored in the medium, wherein the computer program is configured to execute the steps in any of the above method embodiments when running.

According to yet another embodiment of the present disclosure, an electronic device is also provided, including a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to perform any of the above. Steps in method embodiments.

Description of the drawings

Figure 1 is a hardware structure block diagram of a method for generating virtual reality service videos according to an embodiment of the present disclosure;

Figure 2 is a flow chart of a method for generating a virtual reality service video according to an embodiment of the present disclosure;

Figure 3 is a flow chart of a method for generating a virtual reality service video according to an embodiment of the present disclosure;

Figure 4 is a structural block diagram of a device for generating a virtual reality service video according to an embodiment of the present disclosure.

Detailed ways

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings and embodiments.

It should be noted that the terms "first", "second", etc. in the description and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

Terminology:

VR: Virtual Reality, virtual reality technology;

GAN: Generative Adversarial Networks, Generative Adversarial Networks, a generative technology in deep learning

SLAM: Simultaneous Localization and Mapping, simultaneous positioning and mapping technology, a method used for mapping and positioning in the field of robotics;

3D pose: the position, speed and acceleration of an object in three-dimensional space;

3D model: the components and attributes of objects in virtual reality, including three-dimensional size, color, etc.;

Video service: a video used to guide users to understand a certain business or guide the business process;

VR service: dynamically provide users with business processing services through characters in a virtual reality environment.

The method embodiments provided in the embodiments of this application can be executed in a mobile terminal, a computer terminal, or a similar computing device. Taking running on a mobile terminal as an example, FIG. 1 is a hardware structure block diagram of a mobile terminal of a method for generating a virtual reality service video according to an embodiment of the present disclosure. As shown in Figure 1, the mobile terminal may include one or more (only one is shown in Figure 1) processors 102 (the processor 102 may include but is not limited to a microprocessor (Central Processing Unit, MCU) or a programmable logic device (Field Programmable Gate Array, FPGA) and other processing devices) and a memory 104 for storing data, wherein the above-mentioned mobile terminal may also include a transmission device 106 for communication functions and an input and output device 108. Persons of ordinary skill in the art can understand that the structure shown in Figure 1 is only illustrative, and it does not limit the structure of the above-mentioned mobile terminal. For example, the mobile terminal may also include more or fewer components than shown in FIG. 1 , or have a different configuration than shown in FIG. 1 .

The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the method for generating a virtual reality service video in the embodiment of the present disclosure. The processor 102 runs the computer program stored in the memory 104 , thereby executing various functional applications and data processing, that is, implementing the above method. Memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include memory located remotely relative to the processor 102, and these remote memories may be connected to the mobile terminal through a network. Examples of the above-mentioned networks include but are not limited to the Internet, intranets, local area networks, mobile communication networks and combinations thereof.

The transmission device 106 is used to receive or send data via a network. Specific examples of the above-mentioned network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC for short), which can be connected to other network devices through a base station to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (Radio Frequency, RF for short) module, which is used to communicate with the Internet wirelessly.

In this embodiment, a method for generating a virtual reality service video is provided. Figure 2 is a flow chart according to an embodiment of the present disclosure. As shown in Figure 2, the process includes the following steps:

Step S202, extract the 3D point cloud data in the service video, where the service video is a video formed by the customer service object performing customer service service;

Step S204, determine the customer service object from each object included in the service video according to the optical flow parameters of the service video;

Step S206, extract the point cloud data of the customer service object from the 3D point cloud data, and determine the object pose parameters of the customer service object;

Step S208: Convert the reference object model according to the object pose parameters, and bring the point cloud data of the converted reference object model into the 3D point cloud data to form a reference video, where the reference object model is the virtual reality customer service corresponding to the customer service object. ;

Step S210, adjust the reference video until the optical flow parameters of the reference video meet the preset conditions, and obtain the virtual reality service video corresponding to the customer service object.

Service videos include dynamic images of customer service objects performing customer service services, and are not limited to including image information such as the environment where the customer service objects are located. The specific method of extracting the 3D point cloud data in the service video is not limited here. For example, the 3D point cloud data in the service video is extracted through SLAM technology.

The optical flow parameters in the service video are not limited to the optical flow continuity determined for adjacent key frames in the service video. According to the optical flow continuity, each included object is determined from the 3D point cloud data of the service video, and then each object is Customer service objects are identified.

When the customer service object is identified, the point cloud data of the customer service object is extracted from the 3D point cloud data of the service video, and the object pose parameter of the customer service object is obtained based on the point cloud data of the customer service object. The object pose parameter indicates It is the position and posture of the customer service object, such as gestures and the degree of bending of joints. For each key frame, it is not limited to obtaining the corresponding pose parameters.

According to the pose parameters of each key frame, the reference object model is transformed into a pose, so that the point cloud data converted to the reference object model corresponding to the pose is brought into the 3D point cloud data after the customer service object is extracted from the service video. to form a reference video. The reference object model is the character model of the virtual reality customer service corresponding to the customer service object, and is not limited to the character model having the characteristics of the customer service object and the reference image.

The adjustment of the reference video including the point cloud data of the reference object model is not limited to the comparison of the optical flow parameters of the reference video and the optical flow parameters of the service video. The pose of the reference object model is adjusted to obtain the optical flow parameters. Virtual reality service videos that meet preset conditions.

Through the embodiments of the present disclosure, the reference object model is converted through the pose parameters of the customer service object in the service video, and the converted reference object model is brought into the 3D point cloud data of the service video to form a reference video, so that the reference view is The optical flow parameters of the frequency are adjusted to obtain the virtual reality service video. The corresponding virtual reality service video is generated based on the service video of the customer service object. The virtual reality service video is used for customer service, breaking the problem that the same customer service object can only serve one user at the same time. To limit the provision of customer service services, use virtual reality service videos to provide customer service services to multiple users at the same time. Therefore, the problem of low service efficiency of online video services can be solved, and the technical effect of improving the service efficiency of video services can be achieved.

As an optional implementation, before converting the reference object model according to the object pose parameters, the above further includes:

S11, obtain the facial data of the customer service object;

S12, use neural network to convert the facial data and reference image features of the customer service object into a reference object model.

The reference object model is not limited to the neural network, and the reference object model will be generated by overcoming the object's facial data and reference image features. The facial data of the customer service object is not limited to the facial feature data of the customer service object, and the reference image features are not limited to the preset body image features, including hairstyle, limbs, clothing, accessories and other character features other than the face. Through the neural network, the facial data of the customer service object and the preset reference image characteristics are used to generate a virtual reality (VR) customer service object corresponding to the customer service object, and the VR customer service object is used to further generate a VR service video, so that the VR service video can replace the customer service object. The form of video service improves the service efficiency of video services.

As an optional implementation manner, the above-mentioned S204 determines the customer service object from each object included in the service video according to the optical flow parameters of the service video, including:

S204-1, cluster the 3D point cloud data according to the optical flow parameters of the service video to obtain each object included in the service video;

S204-2: Use the key frame images in the service video to identify each object to determine the customer service object from each object.

Obtain the optical flow parameters corresponding to each key frame in the service video, and cluster the 3D point cloud data of the service video through the continuity of the optical flow parameters between adjacent key frames to obtain each object included in the service video. . Object recognition is performed on each object obtained by clustering to determine the customer service object.

As an optional implementation, the above-mentioned clustering of 3D point cloud data is performed based on the optical flow parameters of the service video to obtain various objects included in the service video, including:

S204-11, determine the optical flow parameters of adjacent key frames in the service video;

S204-12, match the optical flow parameters with the two-dimensional coordinates formed by 3D point cloud data mapping;

S204-13, utilize the continuity of optical flow parameters of adjacent key frames to obtain each object through clustering.

Match the optical flow parameters of each key frame with the two-dimensional coordinates formed by 3D point cloud data mapping, and use the continuity of the optical flow parameters of the same object in adjacent key frames to cluster each 3D object included in the service video . Map 3D point cloud data to a two-dimensional plane to determine and match the continuity of optical flow parameters on the two-dimensional plane formed by mapping. It is not limited to setting the judgment conditions for the continuity of optical flow parameters, so as to judge adjacent key points. In the frame, the optical flow of which point cloud data is continuous, so that each 3D object can be clustered according to the continuity of the optical flow.

As an optional implementation, the above method uses key frame images in the service video to identify each object, including:

S204-21, segment key frame images from the service video according to the clustering results;

S204-22, perform object recognition on the key frame image and obtain the recognition result;

S204-23: Match the recognition results with each object, and add object labels to each object according to the matching results.

The clustering result is not limited to each 3D object included in the service video obtained by clustering. The key frame image is segmented from the service video according to the clustering result. It is not limited to determining the object for each key frame image from the service video. The identified keyframe image, for example, the keyframe image that best matches the clustering result in any one or more dimensions such as the number of objects, shape, etc.

Perform object recognition on the key frame image to obtain the object recognition result, match the object recognition result with each object obtained from the clustering result one by one, and then add object labels to each object according to the matched object recognition results, and mark them with object labels Each object obtained by clustering. The object recognition results are not limited to identifying the specific classification of each object in the key frame image, such as people, microphones, exhibition boards, etc.

As an optional implementation manner, the above-mentioned S206 extracts the point cloud data of the customer service object from the 3D point cloud data, and determines the object pose parameters of the customer service object, including:

S206-1, extract the point cloud data of the customer service object from the 3D point cloud data according to the object label;

S206-2: Based on the point cloud data of the customer service object, calculate the object pose of the customer service object and the probability corresponding to the object pose.

After adding object labels to each 3D object obtained by clustering, the customer service objects included in the 3D objects are not limited to being determined through the object labels. For example, the 3D objects identified as "characters" are determined as customer service objects, and the 3D objects of the service videos are determined. Extract point cloud data of customer service objects from point cloud data.

When the point cloud data of the customer service object is extracted, the object pose of the customer service object and the probability corresponding to the object pose are calculated. The object pose of the customer service object is not limited to the object pose of the customer service object corresponding to each key frame of the service video, and the probability change interval of the object pose. The object pose is not limited to the 3D pose of the customer service object.

As an optional implementation manner, the above-mentioned S210 adjusts the reference video until the optical flow parameters of the reference video meet the preset conditions, and obtains the virtual reality service video corresponding to the customer service object, including:

S210-1, calculate the optical flow parameters of the reference key frames in the reference video;

S210-2, obtain the optical flow parameters of the original key frame corresponding to the reference key frame in the service video;

S210-3, compare the optical flow parameters of the reference key frame and the optical flow parameters of the original key frame to obtain the optical flow difference parameter;

S210-41, when the optical flow difference parameter corresponding to each reference key frame of the reference video is less than the preset threshold, determine that the virtual reality service video is obtained;

S210-42: When the optical flow difference parameter is greater than or equal to the preset threshold, adjust the pose of the reference object model according to the optical flow difference parameter until the virtual reality service video is obtained.

Perform pose transformation on the reference object model based on the 3D pose calculated from the point cloud data of the customer service object, transform the reference object model into a pose consistent with the 3D pose, and convert the point cloud of the transformed reference object model into The data is brought into the 3D point cloud data of the service video excluding the customer service object to form a reference video. That is, the reference object model after pose transformation is used to replace the customer service object in the service video, thereby obtaining a reference video including VR customer service.

Specifically, the point cloud data of the reference object model is not limited to the point cloud data corresponding to each key frame. The reference object model is brought in for each key frame to obtain all key frames. Includes reference videos for VR customer service.

After obtaining the reference video, obtain the optical flow parameters of each reference key frame in the reference video and compare it with the optical flow parameters of the original key frame of the service video to obtain the optical flow difference parameter, which is used to indicate the difference between the reference key frame and the original key frame. Optical flow difference between frames.

When the optical flow difference parameter is greater than or equal to the preset threshold, the reference video is adjusted until the adjusted optical flow difference parameter of the reference video is less than the preset threshold, and the reference video is used as the VR service video. For reference video The adjustment is not limited to adjusting the pose of the VR customer service in each reference key frame. By adjusting the pose of the VR customer service brought in, the difference between the optical flow of the formed reference key frame and the optical flow of the original key frame is reduced to the predetermined level. Set threshold.

Specifically, the generation of virtual reality VR service videos is not limited to that shown in Figure 3. Based on the video service including customer service, SLAM technology is used to extract the 3D point cloud data of the video service. At the same time, the optical flow of each key frame of the video service is obtained, and the optical flow continuity of adjacent key frames is used to perform point cloud clustering to determine the point cloud data of the customer service included in the video service. Then, each customer service point cloud data is calculated. The 3D pose of the customer service agent in key frames. After converting the VR character model corresponding to the customer service according to the 3D pose, the transformed VR character model is used to replace the customer service and bring it into the 3D point cloud data to form a reference view. By comparing the optical flow between the reference video and the video service, the VR customer service in the reference video is iterated until the VR service is obtained.

Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus the necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is Better implementation. Based on this understanding, the technical solution of the present disclosure can be embodied in the form of a software product in nature or in part that contributes to the existing technology. The computer software product is stored in a storage medium (such as read-only memory/random access memory). The memory (Read-Only Memory/Random Access Memory, ROM/RAM), magnetic disk, optical disk) includes several instructions to cause a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the disclosure Methods described in various embodiments.

This embodiment also provides a device for generating a virtual reality service video. The device is configured to implement the above embodiments and preferred implementations. What has already been explained will not be described again. As used below, the term "module" may be a combination of software and/or hardware that implements a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and contemplated.

Figure 4 is a structural block diagram of a device for generating virtual reality service videos according to an embodiment of the present disclosure. As shown in Figure 4, the device includes:

The extraction module 41 is configured to extract 3D point cloud data in the service video, where the service video is a video formed by the customer service object performing customer service service;

The determination module 42 is configured to determine the customer service object from each object included in the service video according to the optical flow parameters of the service video;

The pose module 43 is configured to extract the point cloud data of the customer service object from the 3D point cloud data, and determine the object pose parameters of the customer service object;

The conversion module 44 is configured to convert the reference object model according to the object pose parameters, and bring the point cloud data of the converted reference object model into the 3D point cloud data to form a reference video, where the reference object model is the corresponding one of the customer service object. virtual reality customer service;

The adjustment module 45 is configured to adjust the reference video until the optical flow parameters of the reference video meet the preset conditions to obtain a virtual reality service video corresponding to the customer service object.

Optionally, the above-mentioned determination module 42 includes: clustering 3D point cloud data according to the optical flow parameters of the service video to obtain each object included in the service video; using key frame images in the service video to identify each object to Determine the customer service objects from each object.

Optionally, the above-mentioned determination module 42 clusters the 3D point cloud data according to the optical flow parameters of the service video to obtain each object included in the service video, and further includes: determining the optical flow parameters of adjacent key frames in the service video; Compare the optical flow parameters with The two-dimensional coordinates formed by 3D point cloud data mapping are matched; the continuity of the optical flow parameters of adjacent key frames is used to cluster each object.

Optionally, the above-mentioned determination module 42 uses the key frame images in the service video to identify each object, which also includes: segmenting the key frame images from the service video according to the clustering results; performing object recognition on the key frame images to obtain the identification Result; match the recognition results with each object, and add object labels to each object based on the matching results.

Optionally, the above-mentioned pose module 43 also includes: extracting the point cloud data of the customer service object from the 3D point cloud data according to the object label; calculating the object pose and object pose of the customer service object based on the point cloud data of the customer service object. corresponding probability.

Optionally, the above-mentioned generating device of virtual reality service video also includes a model module, which is configured to obtain the facial data of the customer service object before converting the reference object model according to the object pose parameters; and use the neural network to convert the facial data of the customer service object into and reference image features are transformed into reference object models.

Optionally, the above-mentioned adjustment module 45 includes: calculating the optical flow parameters of the reference key frames in the reference video; obtaining the optical flow parameters of the original key frames corresponding to the reference key frames in the service video; comparing the optical flow parameters of the reference key frames with the original key frames. The optical flow parameter of the frame is obtained to obtain the optical flow difference parameter; when the optical flow difference parameter corresponding to each reference key frame of the reference video is less than the preset threshold, the virtual reality service video is determined to be obtained; when the optical flow difference parameter is greater than or When equal to the preset threshold, the pose of the reference object model is adjusted according to the optical flow difference parameter until the virtual reality service video is obtained.

Through the embodiments of the present disclosure, the reference object model is converted through the pose parameters of the customer service object in the service video, and the converted reference object model is brought into the 3D point cloud data of the service video to form a reference video, so that the reference video is The optical flow parameters are adjusted to obtain the virtual reality service video. The corresponding virtual reality service video is generated based on the service video of the customer service object. The virtual reality service video is used for customer service, breaking the problem that the same customer service object can only provide one user at the same time. Limitation of customer service, use virtual reality service video to provide customer service to multiple users at the same time. Therefore, the problem of low service efficiency of online video services can be solved, and the technical effect of improving the service efficiency of video services can be achieved.

It should be noted that each of the above modules can be implemented through software or hardware. For the latter, it can be implemented in the following ways, but is not limited to this: the above modules are all located in the same processor; or the above modules can be implemented in any combination. The forms are located in different processors.

To facilitate understanding of the technical solutions provided by the present disclosure, detailed descriptions will be given below in conjunction with embodiments of specific scenarios.

Embodiments of the present disclosure also provide a computer-readable storage medium that stores a computer program, wherein the computer program is configured to execute the steps in any of the above method embodiments when running.

In an exemplary embodiment, the computer-readable storage medium may include but is not limited to: U disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM) , mobile hard disk, magnetic disk or optical disk and other media that can store computer programs.

Embodiments of the present disclosure also provide an electronic device, including a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

In an exemplary embodiment, the above-mentioned electronic device may further include a transmission device and an input-output device, wherein the transmission device is connected to the above-mentioned processor, and the input-output device is connected to the above-mentioned processor.

For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementations. The example will not be repeated here.

Obviously, those skilled in the art should understand that each module or each step of the above-mentioned embodiments of the present disclosure can be implemented by a general computing device, and they can be concentrated on a single computing device, or distributed among multiple computing devices. over a network, they may be implemented with program code executable by a computing device, such that they may be stored in a storage device for execution by the computing device, and in some cases, may be executed in a sequence different from that described here. The steps shown or described may be implemented by fabricating them separately into individual integrated circuit modules, or by fabricating multiple modules or steps among them into a single integrated circuit module. As such, the present disclosure is not limited to any specific combination of hardware and software.

The above are only preferred embodiments of the present disclosure and are not intended to limit the present disclosure. For those skilled in the art, various modifications and changes may be made to the embodiments of the present disclosure. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the embodiments of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

A method for generating virtual reality service videos, including:

Extract 3D point cloud data in the service video, where the service video is a video formed by the customer service object performing customer service service;

Determine the customer service object from each object included in the service video according to the optical flow parameters of the service video;

Extract the point cloud data of the customer service object from the 3D point cloud data, and determine the object pose parameters of the customer service object;

The reference object model is converted according to the object pose parameters, and the point cloud data of the converted reference object model is brought into the 3D point cloud data to form a reference video, wherein the reference object model is the customer service object Corresponding virtual reality customer service;

The reference video is adjusted until the optical flow parameters of the reference video meet the preset conditions, and a virtual reality service video corresponding to the customer service object is obtained.
The method according to claim 1, wherein the customer service object is determined from various objects included in the service video according to the optical flow parameters of the service video, including:

Cluster the 3D point cloud data according to the optical flow parameters of the service video to obtain the various objects included in the service video;

The key frame images in the service video are used to identify each object, so as to determine the customer service object from each object.
The method according to claim 2, wherein the 3D point cloud data is clustered according to the optical flow parameters of the service video to obtain the various objects included in the service video, including:

Determine the optical flow parameters of adjacent key frames in the service video;

Match the optical flow parameters with the two-dimensional coordinates formed by mapping the 3D point cloud data;

The respective objects are obtained by clustering using the continuity of the optical flow parameters of the adjacent key frames.
The method according to claim 3, wherein identifying the respective objects using key frame images in the service video includes:

Segment the key frame image from the service video according to the clustering result;

Perform object recognition on the key frame image to obtain recognition results;

The recognition results are matched with the respective objects, and object tags are added to the respective objects according to the matching results.
The method according to claim 1, wherein extracting the point cloud data of the customer service object from the 3D point cloud data and determining the object pose parameters of the customer service object includes:

Extract the point cloud data of the customer service object from the 3D point cloud data according to the object tag;

According to the point cloud data of the customer service object, the object pose of the customer service object and the probability corresponding to the object pose are calculated.
The method according to claim 1, wherein before converting the reference object model according to the object pose parameters, it further includes:

Obtain the facial data of the customer service object;

Using a neural network, the facial data and reference image features of the customer service object are converted into the reference object model.
The method according to claim 1, wherein adjusting the reference video until the optical flow parameters of the reference video meet preset conditions to obtain the virtual reality service video corresponding to the customer service object includes:

Calculate optical flow parameters of reference key frames in the reference video;

Obtain the optical flow parameters of the original key frames corresponding to the reference key frames in the service video;

Compare the optical flow parameters of the reference key frame and the optical flow parameters of the original key frame to obtain optical flow difference parameters;

When the optical flow difference parameter corresponding to each reference key frame of the reference video is less than the preset threshold, it is determined that the virtual reality service video is obtained;

When the optical flow difference parameter is greater than or equal to the preset threshold, the pose of the reference object model is adjusted according to the optical flow difference parameter until the virtual reality service video is obtained.
A device for generating virtual reality service videos, including:

The extraction module is configured to extract 3D point cloud data in the service video, where the service video is a video formed by the customer service object performing customer service service;

A determination module configured to determine the customer service object from each object included in the service video according to the optical flow parameters of the service video;

A pose module configured to extract the point cloud data of the customer service object from the 3D point cloud data, and determine the object pose parameters of the customer service object;

A conversion module configured to convert the reference object model according to the object pose parameters, and bring the point cloud data of the converted reference object model into the 3D point cloud data to form a reference video, wherein the reference object model The virtual reality customer service corresponding to the customer service object;

The adjustment module is configured to adjust the reference video until the optical flow parameters of the reference video meet the preset conditions to obtain the virtual reality service video corresponding to the customer service object.
A computer-readable storage medium having a computer program stored in the computer-readable storage medium, wherein when the computer program is executed by a processor, the steps of the method described in any one of claims 1 to 7 are implemented. .
An electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that when the processor executes the computer program, claim 1 is realized to the steps of the method described in any one of 7.