WO2021109376A1

WO2021109376A1 - Method and device for producing multiple camera-angle effect, and related product

Info

Publication number: WO2021109376A1
Application number: PCT/CN2020/082545
Authority: WO
Inventors: 刘文韬; 郑佳宇; 黄展鹏; 李佳桦
Original assignee: 深圳市商汤科技有限公司
Priority date: 2019-12-03
Filing date: 2020-03-31
Publication date: 2021-06-10
Also published as: CN111080759B; JP2023501832A; JP7457806B2; KR20220093342A; TWI752502B; TW202123178A; CN111080759A

Abstract

Embodiments of the present application disclose a method and device for producing a multiple camera-angle effect, and a related product. The method comprises: acquiring a three-dimensional virtual model; and rendering the three-dimensional virtual model using at least two different camera angles, so as to obtain virtual images respectively corresponding to the at least two different camera angles.

Description

Method, device and related products for realizing split-mirror effect

Cross-references to related applications

This application is filed based on a Chinese patent application with an application number of 201911225211.4 and an application date of December 3, 2019, and claims the priority of the Chinese patent application. The entire content of the Chinese patent application is hereby incorporated into this application by way of introduction.

Technical field

This application relates to the field of virtual technology, and in particular to a method, device and related products for realizing the split-mirror effect.

Background technique

In recent years, "virtual characters" have frequently appeared in our lives, for example, the well-known applications of virtual idols such as "Hatsune Miku" and "Luo Tianyi" in the music field, or the application of virtual hosts in live news and many more. Since virtual characters can replace real characters in activities in the online world, and users can set the appearance and shape of virtual characters according to their needs, virtual characters have gradually become a way of communication between people.

At present, the virtual characters in the network generally use motion capture technology in the generation process, and the real person images obtained by the image recognition method are analyzed, so as to direct the actions and expressions of the real characters to the virtual characters, so that the virtual characters can be Reproduce the movements and expressions of real characters.

Summary of the invention

The embodiments of the present application disclose a method, device and related products for realizing the split-mirror effect.

In the first aspect, the embodiment of the present application provides a method for implementing the splitting effect, including: obtaining a three-dimensional virtual model; rendering the three-dimensional virtual model with at least two different lens angles to obtain at least two different lens angles, respectively The corresponding virtual image.

The above method obtains a three-dimensional virtual model and renders the three-dimensional virtual model with at least two different lens angles, so as to obtain virtual images corresponding to at least two different lens angles, so that the user can see the images under different lens angles. Virtual images provide users with a rich visual experience.

In some optional embodiments of the present application, the three-dimensional virtual model includes a three-dimensional virtual character model in a three-dimensional virtual scene model. Before obtaining the three-dimensional virtual model, the above method further includes: obtaining a real image, where the real image includes a real character Image; feature extraction of real person images to obtain feature information, where the feature information includes the action information of the real person; generate a three-dimensional virtual model according to the feature information, so that the action information of the three-dimensional virtual character model in the three-dimensional virtual model is the same as that of the real person Action information correspondence.

It can be seen that by extracting the features of the collected real person images, a 3D virtual model is generated, so that the 3D virtual person model in the 3D virtual model can reproduce the facial expressions and body movements of the real person, and it is convenient for the audience to watch the 3D virtual The virtual image corresponding to the model can learn the facial expressions and body movements of the real person, so that the audience can interact more flexibly with the live anchor.

In some optional embodiments of the present application, acquiring the real image includes: acquiring a video stream, and obtaining at least two frames of real images according to at least two frames of images in the video stream; performing feature extraction on real person images to obtain feature information, including: Perform feature extraction on each frame of real person images to obtain corresponding feature information.

It can be seen that the three-dimensional virtual model can be changed in real time according to the multiple frames of real images collected, so that the user can see the dynamic change process of the three-dimensional virtual model under different lens perspectives.

In some optional embodiments of the present application, the real image further includes a real scene image, and the three-dimensional virtual model also includes a three-dimensional virtual scene model; before obtaining the three-dimensional virtual model, the above method further includes: constructing a three-dimensional virtual scene based on the real scene image model.

It can be seen that the above method can also use real scene images to construct three-dimensional virtual scene images in the three-dimensional virtual model, which makes the three-dimensional virtual scene images more selective than only selecting specific three-dimensional virtual scene images.

In some optional embodiments of the present application, acquiring at least two different lens angles includes: obtaining at least two different lens angles according to at least two frames of real images.

It can be seen that each frame of real image corresponds to a lens angle, and multiple frames of real image correspond to multiple lens angles. Therefore, at least two frames of different lens angles can be obtained from at least two frames of real images, which can be used to realize the lens angle of the 3D virtual model. Rendering to provide users with a rich visual experience.

In some optional embodiments of the present application, acquiring at least two different lens angles includes: obtaining at least two different lens angles according to the action information corresponding to the at least two frames of real images.

It can be seen that determining the lens angle of view based on the action information of the real person in the real image can magnify the action of the corresponding three-dimensional virtual character model in the image, so that the user can learn the action of the real person by watching the virtual image and improve the interaction Sex and fun.

In some optional embodiments of the present application, acquiring at least two different camera angles includes: acquiring background music; determining a time collection corresponding to the background music, where the time collection includes at least two time periods; acquiring each of the time collections The lens angle of view corresponding to the time period.

It can be seen that in the above method, by analyzing the background music and determining the time collection corresponding to the background music, multiple different lens perspectives can be obtained. This method can increase the diversity of the lens perspectives, so that users can get a richer vision. Experience.

In some optional embodiments of the present application, the at least two different lens angles include a first lens angle of view and a second lens angle of view; the three-dimensional virtual model is rendered with at least two different lens angles to obtain at least two different lens angles. The virtual images corresponding to the lens perspectives respectively include: rendering the three-dimensional virtual model with the first lens perspective to obtain the first virtual image; rendering the three-dimensional virtual model with the second lens perspective to obtain the second virtual image; An image sequence formed by the virtual image and the second virtual image.

It can be seen that rendering the three-dimensional virtual model with the first lens perspective and the second lens perspective respectively allows the user to view the three-dimensional virtual model in the first lens perspective and the three-dimensional virtual model in the second lens perspective, thereby providing users with Provide a rich visual experience.

In some optional embodiments of the present application, rendering the three-dimensional virtual model in the second lens perspective to obtain the second virtual image includes: translating or rotating the three-dimensional virtual model in the first lens perspective to obtain the second lens A three-dimensional virtual model under a viewing angle; acquiring a second virtual image corresponding to the three-dimensional virtual model under a second lens perspective.

It can be seen that by translating or rotating the three-dimensional virtual model under the first lens angle of view, the three-dimensional virtual model under the second lens angle of view, that is, the second virtual image, can be quickly and accurately obtained.

In some optional embodiments of the present application, displaying the image sequence formed according to the first image and the second virtual image includes: inserting a frame of virtual image between the first virtual image and the second virtual image, so that the first virtual image The image is gently switched to the second virtual image, where a is a positive integer.

It can be seen that the a-frame virtual image is inserted between the first virtual image and the second virtual image, so that the viewer can see the entire change process from the first virtual image to the second virtual image, instead of a single two images ( The first virtual image and the second virtual image), so that the audience can adapt to the effect of the visual difference caused by the first virtual image to the second virtual image.

In some optional embodiments of the present application, the method further includes: performing beat detection on the background music to obtain a beat collection of the background music, where the beat collection includes multiple beats, and each beat of the multiple beats corresponds to a stage special effect ; Add the target stage special effects corresponding to the beat collection to the 3D virtual model.

It can be seen that according to the beat information of the music, corresponding stage effects are added to the virtual scene where the virtual character model is located, thereby presenting different stage effects to the audience and enhancing the audience's viewing experience.

In the second aspect, an embodiment of the present application also provides a device for realizing a splitting effect, including: an acquiring unit configured to acquire a three-dimensional virtual model; and a splitting unit configured to view the three-dimensional virtual model from at least two different lens angles. Perform rendering to obtain at least two virtual images respectively corresponding to different lens angles.

In some optional embodiments of the present application, the three-dimensional virtual model includes a three-dimensional virtual character model in a three-dimensional virtual scene model, and the device further includes: a feature extraction unit and a three-dimensional virtual model generation unit; wherein, the acquisition unit is also configured to Before acquiring a three-dimensional virtual model, acquire a real image, where the real image includes an image of a real person; the feature extraction unit is configured to perform feature extraction on the image of a real person to obtain feature information, where the feature information includes the action information of the real person; the three-dimensional virtual model The generating unit is configured to generate a three-dimensional virtual model according to the feature information, so that the action information of the three-dimensional virtual character model in the three-dimensional virtual model corresponds to the action information of the real character.

In some optional embodiments of the present application, the obtaining unit is configured to obtain a video stream, and obtain at least two frames of real images according to at least two frames of images in the video stream; the feature extraction unit is configured to perform processing on each frame of real person images. Feature extraction obtains corresponding feature information.

In some optional embodiments of the present application, the real image further includes a real scene image, and the three-dimensional virtual model also includes a three-dimensional virtual scene model; the device further includes: a three-dimensional virtual scene image construction unit configured to before the acquisition unit acquires the three-dimensional virtual model , According to the real scene image, construct a three-dimensional virtual scene image.

In some optional embodiments of the present application, the device further includes a lens angle acquisition unit configured to obtain at least two different lens angles according to at least two frames of real images.

In some optional embodiments of the present application, the device further includes a lens angle acquisition unit configured to obtain at least two different lens angles according to the action information corresponding to the at least two frames of real images, respectively.

In some optional embodiments of the present application, the device further includes a lens angle acquisition unit configured to acquire background music; determine a time collection corresponding to the background music, where the time collection includes at least two time periods; and acquire each time in the time collection The lens angle of view corresponding to the segment.

In some optional embodiments of the present application, at least two different lens angles include a first lens angle of view and a second lens angle of view, and the splitting unit is configured to render the three-dimensional virtual model with the first lens angle of view to obtain the first lens angle. Virtual image; Render the three-dimensional virtual model from the second lens perspective to obtain the second virtual image; display the image sequence formed according to the first virtual image and the second virtual image.

In some optional embodiments of the present application, the splitting unit is configured to translate or rotate the three-dimensional virtual model under the first lens angle of view to obtain the three-dimensional virtual model under the second lens angle of view; and obtain the three-dimensional virtual model under the second lens angle of view. The second virtual image corresponding to the three-dimensional virtual model.

In some optional embodiments of the present application, the mirror splitting unit is configured to insert a frame of virtual image between the first virtual image and the second virtual image, so that the first virtual image is gently switched to the second virtual image, wherein, a is a positive integer.

In some optional embodiments of the present application, the device further includes: a beat detection unit and a stage special effect generation unit; wherein the beat detection unit is configured to perform beat detection on the background music to obtain a beat collection of the background music, wherein the beat collection Including multiple beats, each of the multiple beats corresponds to a stage special effect; the stage special effect generation unit is configured to add the target stage special effect corresponding to the beat collection to the three-dimensional virtual model.

In the third aspect, an embodiment of the present application provides an electronic device, including: a processor, a communication interface, and a memory; the memory is used to store instructions, the processor is used to execute instructions, and the communication interface is used to communicate with other devices under the control of the processor. Communicating, wherein the processor executes the instruction to enable the electronic device to implement any one of the methods in the first aspect described above.

In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium that stores a computer program, and the computer program is executed by hardware to implement any one of the methods in the first aspect.

In the fifth aspect, the embodiments of the present application provide a computer program product. When the computer program product is read and executed by a computer, any one of the methods in the above-mentioned first aspect is executed.

Description of the drawings

In order to more clearly describe the technical solutions in the embodiments of this application or the background art, the following will briefly introduce the drawings needed in the description of the embodiments of this application. Obviously, the drawings in the following description are some of the present application. Embodiments, for those of ordinary skill in the art, without creative work, other drawings can be obtained based on these drawings.

Fig. 1 is a schematic diagram of a specific application scenario provided by an embodiment of the present application;

FIG. 2 is a schematic diagram of a possible three-dimensional virtual model provided by an embodiment of the present application;

FIG. 3 is a schematic flowchart of a method for realizing a split-mirror effect provided by an embodiment of the present application;

FIG. 4 is a schematic diagram of an interpolation curve provided by an embodiment of the present application;

FIG. 5 is a schematic flowchart of a specific embodiment provided by an embodiment of the present application;

FIG. 6 is a schematic diagram of a splitting rule provided by an embodiment of the present application;

FIG. 7A is an effect diagram of a possible virtual image provided by an embodiment of the present application;

FIG. 7B is an effect diagram of a possible virtual image provided by an embodiment of the present application;

FIG. 7C is an effect diagram of a possible virtual image provided by an embodiment of the present application;

FIG. 7D is an effect diagram of a possible virtual image provided by an embodiment of the present application;

FIG. 8 is a schematic structural diagram of a device for implementing a split-mirror effect provided by an embodiment of the present application;

FIG. 9 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.

Detailed ways

The terms used in the embodiments of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

The method, device and related products for realizing the split-mirror effect provided by the embodiments of the present application can be applied in many fields such as social interaction, entertainment, and education. For example, it can be used for virtual live broadcast, social interaction in virtual communities, or Used to hold virtual concerts, can also be used in classroom teaching and so on. In order to facilitate the understanding of the embodiments of the present application, the following takes virtual live broadcast as an example to describe the specific application scenarios of the embodiments of the present application in detail.

Virtual live broadcast is a way to use virtual characters instead of live anchors to conduct live broadcasts on a live broadcast platform. Because virtual characters have rich expressive power and are more in line with the communication environment of social networks, the virtual live broadcast industry is developing rapidly. In the process of virtual live broadcast, computer technologies such as facial expression capture, motion capture, and sound processing are usually used to apply the facial expressions and actions of the live anchor to the virtual character model, so as to realize the audience and the virtual anchor in the video website or social networking website. of interaction.

In order to save live broadcast costs and post-production costs, users usually directly use mobile phones, tablet computers and other terminal devices for live broadcasts. Please refer to FIG. 1. FIG. 1 is a schematic diagram of a specific application scenario provided by an embodiment of the present application. In the live broadcast process shown in FIG. The server 120 transmits the generated virtual image to the user terminal 130 through the network for processing, so that different viewers can watch the entire live broadcast process through the corresponding user terminal 130.

It can be seen that although the cost of virtual live broadcast in this way is relatively low, since only a single camera device 110 shoots the live anchor, the posture of the generated virtual anchor is related to the relative position between the camera device 110 and the live anchor. That is to say, the audience can only see the virtual character under a specific angle of view, and this specific angle of view depends on the relative position between the camera device 110 and the live broadcaster, so that the live broadcast effect presented is unsatisfactory. For example, in the process of virtual live broadcast, there are often problems such as stiff movements of virtual anchors, unsmooth shot switching screens, or monotonous and boring shots, which cause visual fatigue of the audience and make it impossible for the audience to experience the immersive experience.

Similarly, in other application scenarios, for example, live teaching scenes; in the teaching process, the teacher teaches students knowledge through online teaching, but this teaching method is usually boring, and the teacher in the video cannot know the students in real time For the mastery of knowledge points, students can only see the teacher or teaching handouts in a single perspective, which can easily cause students' fatigue. Compared with the teacher's on-site teaching, the teaching effect of video teaching is greatly reduced. For another example, when the concert may not be held as scheduled due to weather and venue restrictions during the concert, the singer can hold a virtual concert in the recording studio to simulate the scene of a real concert, in order to achieve a real concert It is usually necessary to set up multiple cameras to shoot the singer. This kind of virtual concert is complicated to operate and wastes costs. Moreover, the use of multiple cameras for shooting can get pictures under multiple lenses, which may cause lens switching. The problem of non-smoothness makes users unable to adapt to the visual difference caused by switching between different lenses.

In order to solve the problems of single lens angle of view and unsmooth lens switching images that often occur in the above application scenarios, an embodiment of the present application provides a method for realizing the splitting effect. The method generates a three-dimensional virtual image based on the collected real image. Model, and obtain multiple different lens perspectives according to background music or the actions of real characters, and then render the three-dimensional virtual model with multiple different lens perspectives to obtain virtual images corresponding to multiple different lens perspectives, thereby simulating In the virtual scene, there are multiple virtual cameras to shoot the three-dimensional virtual model, which improves the viewer's viewing experience. In addition, the method also analyzes the beats of the background music and adds corresponding stage effects to the three-dimensional virtual model according to the beat information to present different stage effects to the audience, which further enhances the audience's viewing experience.

In the following, the specific process of generating a three-dimensional virtual model from a real image in an embodiment of the present application is first explained.

In the embodiment of the present application, the three-dimensional virtual model includes a three-dimensional virtual character model in a three-dimensional virtual scene. Taking Figure 2 as an example, Figure 2 shows a schematic diagram of a possible three-dimensional virtual model. According to the three-dimensional virtual model shown in Figure 2, it can be seen that the hands of the three-dimensional virtual character model are raised to the chest. In order to highlight the contrast effect, the figure The upper left corner of 2 also shows the real image collected by the split-mirror effect realization device, in which the real person is also raising his hands to his chest. In other words, the three-dimensional virtual character model is consistent with the actions of the real character. It can be understood that the above-mentioned Figure 2 is only an example. In practical applications, the real image collected by the device for implementing the split-mirror effect can be a three-dimensional image or a two-dimensional image. The number of characters in the real image can be one or There are multiple. The action of the real character can be raising both hands to the chest, raising the left foot or other actions, etc. Correspondingly, the number of 3D virtual character models in the 3D virtual model generated from the real character image can be One or more than one. The action of the three-dimensional virtual character model can be raising both hands to the chest, raising the left foot or other actions, etc., which are not specifically limited here.

Embodiment, the sub-mirror effect device for implementing real people shooting, to obtain a plurality of frames real image I _1, I _2, ..., I _n-, and in chronological order I on the real image _1, I ₂ in the present application embodiment, ..., I _{Perform feature extraction on n} to obtain multiple corresponding three-dimensional virtual models M ₁ , M ₂ ,..., M _n , where n is a positive integer, and the real images I ₁ , I ₂ ,..., I _n and the three-dimensional virtual model M ₁ There is a one-to-one correspondence between ,M ₂ ,...,M _n , that is, one frame of real image is used to generate a three-dimensional virtual model. Exemplarily, taking a real image I _{i to} generate a three-dimensional virtual model M _i as an example, a three-dimensional virtual model can be obtained as follows:

Step one, the device for achieving the split-mirror effect obtains the real image I _i .

Wherein, the real image I _i includes real person images, and i is a positive integer, 1≤i≤n.

Step 2: The device for implementing the split-mirror effect performs feature extraction on the real person image in the real _{image I i to obtain feature information.} Among them, the feature information includes action information of real characters.

Wherein, obtaining a real image includes: obtaining a video stream, and obtaining at least two frames of real images according to at least two frames of images in the video stream; correspondingly, performing feature extraction on the real person image to obtain feature information includes: separately for each frame Feature extraction is performed on the real person image to obtain corresponding feature information.

It is understandable that the feature information is used to control the posture of the three-dimensional virtual character model. The action information in the feature information includes facial expression features and body action features. Facial expression features are used to describe various emotional states of the character, such as happy, sad, Surprise, fear, anger or disgust, etc., physical movement characteristics are used to describe the movement state of real characters, for example, raising the left hand, raising the right foot, or jumping. In addition, the feature information can also include character information, where the character information includes multiple key points of the human body of the real person and their corresponding position information. The key points of the human body include key points of the face and the key points of the human skeleton, and the position features include the key points of the real person The position coordinates of the key points of the human body.

Optionally, the split-mirror effect realization device _{extracts and obtains the real person image in the real image I i} by performing image segmentation on the real image I _i ; and then performs key point detection on the extracted real person image to obtain the aforementioned multiple human key points And the position information of multiple key points of the human body, where the key points of the human body include key points of the face and the key points of the bones of the human body. The key points of the human body may be located in the head area, neck area, shoulder area, spine area, and waist of the human body. Area, hip area, wrist area, arm area, knee area, leg area, ankle area, and sole area, etc.; by analyzing the key points of the face and the position information of the key points of the face, the real image I _i is obtained Facial expression characteristics of real characters; by analyzing the key points of the human bones and the position information of the key points of the human bones, _{the bone characteristics of the real characters in the real image I i} are obtained, so as to obtain the physical movement characteristics of the real characters.

Optionally, the device for realizing the split-mirror effect _{inputs the real image I i} into the neural network for feature extraction, and after calculation of multiple convolutional layers, the multiple key point information of the human body is extracted. Among them, the neural network is obtained through a large amount of training. The neural network can be a Convolution Neural Network (CNN), a Back Propagation Neural Network (BPNN), or a generated confrontation Network (Generative Adversarial Network, GAN) or Recurrent Neural Network (Recurrent Neural Network, RNN), etc., which are not specifically limited here. It should be noted that the above-mentioned extraction process of human body features can be performed in the same neural network or in different neural networks. For example, the device for implementing the split-mirror effect can use CNN to extract key points of a human face to obtain facial expression features; it can also use BPNN to extract key points of human bones to obtain human bone features and limb movement features, which are not specifically limited here. In addition, the above example of the feature information used to drive the three-dimensional virtual character model is only used as an example, and other feature information may also be included in practical applications, which is not specifically limited here.

Step 3: The split-mirror effect realization device generates the three- _{dimensional virtual character model in the three-dimensional virtual model M i} according to the characteristic information, so that the three-dimensional virtual character model in the three-dimensional virtual model M _i corresponds to the action information of the real character in the real image I _i.

Optionally, the split-mirror effect realization device establishes a mapping relationship between the key points of the human body of the real person and the key points of the human body of the virtual character model through the above-mentioned feature information; and then controls the expression and posture of the virtual character model according to the mapping relationship, thereby making the virtual The facial expressions and body movements of the character model are consistent with the facial expressions and body movements of the real characters.

Optionally, the split-mirror effect realization device respectively performs serial number labeling on the key points of the human body of the real person to obtain the label information of the key points of the human body of the real person. Among them, the key points of the human body correspond to the label information one by one; The annotation information of the key points is used to mark the key points of the human body in the virtual character model. For example, if the label information of the left wrist of the real person is No. 1, the label information of the left wrist of the three-dimensional virtual character model is also No. 1, and the label information of the left arm of the real character is No. 2, then the left wrist of the three-dimensional virtual character model The annotation information is also No. 2 and so on; then the key point annotation information of the human body of the real person is matched with the key point annotation information of the human body of the three-dimensional virtual character model, and the position information of the key point of the human body of the real character is mapped to the corresponding three-dimensional virtual character model In the key points of the human body, the three-dimensional virtual character model can reproduce the facial expressions and body movements of real characters.

In the embodiment of the present application, the real image I _i also includes a real scene image, and the three-dimensional virtual model M _i also includes a three-dimensional virtual scene model. The above-mentioned method for generating a three-dimensional virtual model M _i _{based on the real image I i} further includes: real scene image _i, M _i construct a three-dimensional virtual model of the three-dimensional virtual scene.

Optionally, the device for realizing the split-mirror effect first performs image segmentation _{on the real image I i} _{to obtain the real scene image in the real image I i} ; then extracts the scene features in the real scene image, for example, the position features of the objects in the real scene, Shape feature, size feature, etc.; construct the three- _{dimensional virtual scene model in the three-dimensional virtual model M i} according to the scene feature, so that the three-dimensional virtual scene model in the three-dimensional virtual model M _i can highly restore the real scene image in the real _{image I i.}

For the sake of simplicity, the above only illustrates the process of generating a three-dimensional virtual model M _i _{from a real image I i} . In fact, the three-dimensional virtual model M ₁ , M ₂ ,...,M _i-1 ,M _i+1 ,...,M _n the generation process of the three-dimensional virtual model generation process is similar to M _i, will not expand further described herein.

It should be noted that the 3D virtual scene model in the 3D virtual model can be constructed based on the real scene image in the real image, or it can be a user-defined 3D virtual scene model; the facial features of the 3D virtual character model in the 3D virtual model can be changed from real The facial features of the real person image in the image can also be user-defined facial features, which is not specifically limited here.

_{Next, perform lens perspective rendering for each of the three-dimensional virtual models M 1} , M ₂ ,..., M _n with multiple different lens perspectives involved in the embodiments of the present application, so that the audience can see the same one The virtual images of the three-dimensional virtual model under different lens perspectives are described in detail. I _i to generate a real image of an example three-dimensional virtual model M _i, respectively, using k different three-dimensional virtual scene rendering model M _i, to obtain the virtual image Q _i1 k at different camera angles, Q _i2, ..., Q _ik , where k≥2, so as to achieve the effect of split-mirror switching, the specific process can be expressed as follows:

As shown in FIG. 3, FIG. 3 is a schematic flowchart of a method for realizing a split-mirror effect provided by an embodiment of the present application. The method for realizing the splitting effect of this embodiment includes but not limited to the following steps:

S101. The device for achieving split-mirror effect obtains a three-dimensional virtual model.

In the embodiments of the present application, the three-dimensional virtual model is used to simulate real characters and real scenes. The three-dimensional virtual model includes a three-dimensional virtual character model in a three-dimensional virtual scene model, and the three-dimensional virtual model is generated based on a real image. Among them, the three-dimensional virtual character model is generated based on the real character image included in the real image, the three-dimensional virtual character model in the three-dimensional virtual model is used to simulate the real character in the real image, and the actions of the three-dimensional virtual character model correspond to the actions of the real character . The three-dimensional virtual scene model may be constructed based on the real scene image included in the real image, or may be a preset three-dimensional virtual scene model. When the three-dimensional virtual scene model is constructed from the real scene image, the three-dimensional virtual scene model can be used to simulate the real scene in the real image.

S102. The device for achieving a split-mirror effect obtains at least two different lens angles of view.

In the embodiments of the present application, the angle of view of the lens is used to indicate the position of the camera relative to the object when the camera is shooting the object. For example, the camera can get a top view of the object when shooting directly above the object. Assuming that the camera is located directly above the object, the corresponding lens angle of view is V, then the image captured by the camera shows the object under the lens angle of V, that is, the top view of the object.

In some optional embodiments, obtaining at least two different lens angles includes: obtaining at least two different lens angles according to at least two frames of real images. Among them, the real image can be taken by a real camera, the position of the real camera relative to the real person may be multiple, and the multiple real images taken by multiple real cameras at different positions show multiple different lens perspectives. Real people.

In some other optional embodiments, obtaining at least two different lens angles includes: obtaining at least two different lens angles according to the action information corresponding to the at least two frames of real images. Among them, the motion information includes the body motions and facial expressions of real characters in real images. Among them, the body movements include many kinds. For example, the body movements can be one or more of raising the right hand, raising the left foot, jumping, etc. The facial expressions also include many kinds. The facial expressions can be, for example, smiling, tearing, etc. One or more of facial expressions such as anger. Examples of body movements and facial expressions in this embodiment are not limited to the above description.

In the embodiment of the present application, one action or a combination of multiple actions corresponds to one lens angle of view. For example, when a real person smiles and jumps, the corresponding lens angle of view is V ₁ , when the real person only jumps, the corresponding lens angle of view can be the lens angle of view V ₁ , or the lens angle of view V _2, etc., the same, when the real person When only smiling, the corresponding lens angle of view can be the lens angle of view V ₁ , the lens angle of view V ₂ , or the lens angle of view V _3, and so on.

In still other optional embodiments, obtaining at least two different camera angles includes: obtaining background music; determining a time collection corresponding to the background music, where the time collection includes at least two time periods; obtaining each time period in the time collection Corresponding lens angle of view. The real image may be one or more frames in a video stream. The video stream includes image information and background music information, where one frame of image corresponds to one frame of music. The background music information includes background music and a corresponding time collection. The time collection includes at least two time periods, and each time period corresponds to a lens angle.

S103. The device for implementing the split-mirror effect renders the three-dimensional virtual model with at least two different lens angles to obtain virtual images corresponding to at least two different lens angles respectively.

In the embodiment of the present application, the aforementioned at least two different lens angles include a first lens angle of view and a second lens angle of view, and the three-dimensional virtual model is rendered with at least two different lens angles to obtain at least two different lenses The virtual images corresponding to the viewing angles respectively include: S1031, rendering the three-dimensional virtual model with the first lens perspective to obtain the first virtual image; S1032, rendering the three-dimensional virtual model with the second lens perspective to obtain the second virtual image.

In the embodiment of the present application, rendering the three-dimensional virtual model from the second lens perspective to obtain the second virtual image includes: translating or rotating the three-dimensional virtual model from the first lens perspective to obtain the three-dimensional virtual model from the second lens perspective. Model; Acquire the second virtual image corresponding to the three-dimensional virtual model under the second lens perspective.

It is understandable that the first lens angle can be obtained based on the real image, it can also be obtained based on the action information corresponding to the real image, or it can be obtained based on the time collection corresponding to the background music; similarly, the second lens angle can be It is obtained based on the real image, or based on the action information corresponding to the real image, or based on the time collection corresponding to the background music, which is not specifically limited in the embodiment of the present application.

S1033. Display an image sequence formed according to the first virtual image and the second virtual image.

In the embodiment of the present application, the above display of the image sequence formed according to the first image and the second virtual image includes: inserting a frame of virtual image between the first virtual image and the second virtual image, so that the first virtual image is smooth Switch to the second virtual image, where a is a positive integer.

_{Optionally, insert a frame of virtual images P 1} , P ₂ ,..., P _a between the first virtual image and the second virtual image, so that the first virtual image is gently switched to the second virtual image, where a frame The time points at which the virtual images P ₁ , P ₂ ,..., P _{a are} _{inserted are b 1} , b ₂ ,..., b _a , and the time points b ₁ , b ₂ ,..., b _a The slope value satisfies the function of monotonically decreasing first and then increasing monotonically, and a is a positive integer.

For example, FIG. 4 shows a schematic diagram of an interpolation curve. As shown in FIG. 4, the device for realizing the split-mirror effect obtains the first virtual image at the first minute, and the second virtual image at the second minute. One virtual image presents the front view of the three-dimensional virtual model, and the second virtual image presents the left view of the three-dimensional virtual model. In order to allow the audience to see a smooth shot switching screen, the split-lens effect realization device inserts multiple time points between the first minute and the second minute, and inserts a virtual image at each time point, for example, in 1.4 inserting the virtual image P is minutes _1, inserting the virtual image P in the first 1.65 minutes _2, insertion of the virtual image P at 1.8 minutes _3, insertion of the virtual image P ₄ in the first 1.85 minutes, wherein the virtual image P ₁ presented is The effect of rotating the three-dimensional virtual model to the left by 30 degrees, the virtual image P ₂ presents the effect of rotating the three-dimensional virtual model to the left by 50 degrees, the virtual image P ₃ and the virtual image P _{4 both} present the effect of rotating the three-dimensional virtual model to the left The 90-degree effect allows the audience to see the entire process of the 3D virtual model gradually changing from the front view to the left view, instead of a single two images (the front view of the 3D virtual model and the left view of the 3D virtual model), thus making The audience can adapt to the changing effect of the visual difference when switching from the front view to the left view.

In some optional embodiments of this application, the use of stage special effects mentioned in the embodiments of this application to render a three-dimensional virtual model to present different stage effects to the audience is described in detail, which specifically includes the following steps:

Step 1: The device for realizing the split-mirror effect detects the beats of the background music, and obtains a collection of beats of the background music.

Among them, the beat collection includes multiple beats, and each beat of the multiple beats corresponds to a stage special effect. Optionally, the split-mirror effect realization device can use shaders and particle special effects to respectively render the 3D virtual model. For example, the shader can be used to realize the spotlight rotation effect on the back of the virtual stage and the sound wave effect of the virtual stage itself, and particle special effects. It is used to add similar visual effects such as sparks, fallen leaves, meteors, etc. to the 3D virtual model.

Step 2: The split-mirror effect realization device adds the target stage special effects corresponding to the beat collection to the three-dimensional virtual model.

The above method generates a three-dimensional virtual model based on the collected real images, and switches the corresponding lens perspective according to the collected real images, background music, and the actions of real characters, thereby simulating that there are multiple virtual cameras in the virtual scene. The effect of shooting the virtual model improves the viewer's sense of viewing experience. In addition, the method also analyzes the beats of the background music and adds corresponding stage special effects to the virtual image according to the beat information to present different stage effects to the audience, which further enhances the audience's viewing experience.

In order to facilitate the understanding of the method for realizing the splitting effect involved in the foregoing embodiments, the method for realizing the splitting effect of the embodiment of the present application will be described in detail below by way of examples.

Please refer to FIG. 5, which shows a schematic flowchart of a specific embodiment.

S201. The device for achieving split-mirror effect obtains a real image and background music, and obtains a first lens angle of view according to the real image. Among them, when the background music sounds, the real person acts according to the background music, and the real camera shoots the real person to obtain the real image.

S202. The device for realizing split-mirror effect generates a three-dimensional virtual model according to the real image. Among them, the three-dimensional virtual model is obtained at the first moment by the device for realizing the split-mirror effect.

S203. The split-mirror effect realization device detects the beat of the background music to obtain the beat collection of the background music, and adds the target stage special effect corresponding to the beat collection to the three-dimensional virtual model.

S204. The device for implementing split-mirror effect renders the three-dimensional virtual model with the first lens angle of view to obtain a first virtual image corresponding to the first lens angle of view.

S205. The device for realizing the split-mirror effect determines the time collection corresponding to the background music.

Wherein, the time collection includes multiple time periods, and each of the multiple time periods corresponds to a lens angle.

S206. The mirroring effect realization device judges whether the action information database contains action information, executes S207-S209 if the action information database does not contain action information, and executes S210-S212 if the action information database contains action information. . Among them, the action information is the action information of the real person in the real image, and the action information database includes a plurality of action information, and each action information in the multiple action information corresponds to a lens angle of view.

S207. The device for realizing the splitting effect determines the second lens angle corresponding to the time period at the first moment according to the time collection.

S208. The device for implementing the split-mirror effect renders the three-dimensional virtual model with the second lens angle of view to obtain a second virtual image corresponding to the second lens angle of view.

S209. The device for realizing split-mirror effect displays an image sequence formed according to the first virtual image and the second virtual image.

S210. The device for achieving a split-mirror effect determines a third lens angle of view corresponding to the action information according to the action information.

S211. The device for implementing the split-mirror effect renders the three-dimensional virtual model with the third lens angle of view to obtain a third virtual image corresponding to the third lens angle of view.

S212. The device for realizing split-mirror effect displays an image sequence formed according to the first virtual image and the third virtual image.

According to the method described in FIG. 5, an embodiment of the present application provides a schematic diagram of a splitting rule as shown in FIG. 6, and performing splitting processing and stage special effects processing on a virtual image according to the splitting rule shown in FIG. The effect diagrams of the four virtual images as shown in FIGS. 7A-7D are obtained.

As shown in Fig. 7A, at the first minute, the split-lens effect realization device shoots a real person under the _{lens angle of view V 1} _{to obtain a real image I 1} (as shown in the upper left corner of Fig. 7A), and then according to the real image I ₁ Obtain a three-dimensional virtual model M ₁ . Storyboard achieve the effect means the background music for the beat detection to determine the first minute corresponding to the pulse for B _1, and ₁ was stage effects W during the first minute ₁ according to the beat B, then the stage effects W ₁ is added to the three-dimensional virtual model In M ₁ ; the split-lens effect realization device determines the lens angle corresponding to the first minute (referred to as the time-lens angle of view) as V ₁ according to the preset lens script; the split-lens effect realization device detects that the action of a real person in the first minute is The action of raising both hands to the chest and raising both hands to the chest is not in the action information database, that is, there is no lens angle of view corresponding to the action (referred to as the action lens angle of view), then the display on the device for achieving the effect of the splitter effect is shown in Figure 7A The virtual image shown in FIG. 7A and the real image I _{1 have} the same lens angle of view.

As shown in Fig. 7B, at the second minute, the split-lens effect realization device shoots a real person under the _{lens angle of view V 1} _{to obtain a real image I 2} (as shown in the upper left corner of Fig. 7B), and then according to the real image I ₂ Obtain a three-dimensional virtual model M ₂ . Storyboard achieve the effect means the background music for the beat detection to determine the first 2 minutes corresponding to the tempo B _2, and ₂ to give the stage effects W ₂ during the first 2 minutes according to the beat B, then add stage effects W in the three-dimensional virtual model of M ₂ ₂ ; The split-lens effect realization device determines the lens angle corresponding to the second minute (referred to as the time-lens angle of view) as V ₂ according to the preset lens script; the split-lens effect realization device detects that the real person’s action in the second minute is lifted up The action of raising your hands and raising your hands is not in the action information database, that is, there is no lens angle of view corresponding to the action (referred to as action lens angle of view), then the split-lens effect realization device _{rotates the three-dimensional virtual model M 2} to the upper left to obtain The lens angle of view is the virtual image corresponding _{to V 2.} It can be seen that when the stage special effect W ₂ _{is added to the three-dimensional virtual model M 2} , the virtual image shown in FIG. 7B has a lighting effect added to the virtual image shown in FIG. 7A.

As shown in Fig. 7C, at the 3rd minute, the split-lens effect realization device shoots a real person under the _{lens angle of view V 1} _{to obtain a real image I 3} (as shown in the upper left corner of Fig. 7C), and then according to the real image I ₃ Obtain a three-dimensional virtual model M ₃ . Storyboard achieve the effect means the background music for the beat detection to determine the third minute of beats corresponding to B _3, and with stage effects W ₃ at the 3rd minute according to the beat B _3, and then add stage effects W ₃ in the three-dimensional virtual model M ₃ ; The splitting effect realization device determines that the corresponding lens angle of view (referred to as the time lens angle of view) _{at the third minute is V 2} according to the preset lens script; the splitting effect realization device detects that the real person’s action in the third minute is upward Lifting the left foot and lifting the left foot corresponds to the lens angle of view (referred to as the action lens angle of view) as V ₃ , then the split-lens effect realization device _{rotates the three-dimensional virtual model M 3} to the left to obtain the lens angle corresponding _{to V 3} Virtual image. It can be seen that when the stage special effect W ₃ _{is added to the three-dimensional virtual model M 3} , the virtual image shown in FIG. 7C is different from the lighting effect in the virtual image shown in FIG. 7B, and the virtual image shown in FIG. 7C appears There is a sound wave effect.

As shown in Fig. 7D, at the 4th minute, the split-lens effect realization device shoots a real person under the _{lens angle of view V 1} _{to obtain a real image I 4} (as shown in the upper left corner of Fig. 7D), and then according to the real image I ₄ Obtain a three-dimensional virtual model M ₄ . Storyboard achieve the effect means the background music for the beat detection to determine the first four minutes corresponding to the tempo B _4, and with the stage effects W ₃ at the time of 4 minutes according to the beat B _4, and then add stage effects W in a 3D virtual model of M ₄ in ₄ ; The splitting effect realization device determines the lens angle of view corresponding to the 3rd minute (referred to as the time lens angle of view) as V ₄ according to the preset lens script; the splitting effect realization device detects that the real person’s action in the 4th minute is standing, And the lens angle of view corresponding to the action of standing (referred to as the action lens angle of view) is V ₄ , at this time, the splitting effect realization device _{rotates the three-dimensional virtual model M 4} to the right to obtain a virtual image corresponding to the _{lens angle of view V 4.} As can be seen, when the stage ₄ is added to effect three-dimensional virtual model W M ₄ in FIG. 7D so that the virtual image shown in FIG. 7C and the virtual image shown in different stage effects.

The splitting effect realization device provided in the embodiments of the present application may be a software device or a hardware device. When the splitting effect realization device is a software device, the splitting effect realization device can be separately deployed on a computing device in a cloud environment. It can be deployed separately on a terminal device. When the split-mirror effect realization device is a hardware device, the internal unit modules of the split-mirror effect realization device can also be divided into multiple types. Each module can be a software module, a hardware module, or part of it. It is a software module and the part is a hardware module, and this application does not limit it. FIG. 8 is an exemplary division method. As shown in FIG. 8, FIG. 8 is a device 800 for implementing a splitting effect provided by an embodiment of the present application, including: an obtaining unit 810 configured to obtain a three-dimensional virtual model; The mirror unit 820 is configured to render the three-dimensional virtual model with at least two different lens angles to obtain virtual images corresponding to at least two different lens angles.

In some optional embodiments of the present application, the three-dimensional virtual model includes a three-dimensional virtual character model in a three-dimensional virtual scene model, and the above-mentioned apparatus further includes: a feature extraction unit 830 and a three-dimensional virtual model generation unit 840; wherein,

The acquiring unit 810 is further configured to acquire a real image before acquiring the three-dimensional virtual model, where the real image includes a real person image; the feature extraction unit 830 is configured to perform feature extraction on the real person image to obtain feature information, where the feature information includes Action information of a real character; the three-dimensional virtual model generating unit 840 is configured to generate a three-dimensional virtual model according to the characteristic information, so that the action information of the three-dimensional virtual character model in the three-dimensional virtual model corresponds to the action information of the real character.

In some optional embodiments of the present application, the acquiring unit is configured to acquire a video stream, and obtain at least two frames of real images according to at least two frames of images in the video stream; the feature extraction unit 830 is configured to separately analyze each frame of real person images Perform feature extraction to obtain corresponding feature information.

In some optional embodiments of the present application, the real image further includes a real scene image, and the three-dimensional virtual model also includes a three-dimensional virtual scene model; the above-mentioned apparatus further includes: a three-dimensional virtual scene image construction unit 850 configured to acquire a three-dimensional virtual model in the acquiring unit Previously, a three-dimensional virtual scene image was constructed based on the real scene image.

In some optional embodiments of the present application, the above-mentioned device further includes a lens angle acquisition unit 860 configured to obtain at least two different lens angles. Specifically, in some optional embodiments, the lens angle of view acquisition unit 860 is configured to obtain at least two different lens angles according to at least two frames of real images.

In some optional embodiments of the present application, the lens angle of view acquisition unit 860 is configured to obtain at least two different lens angles of view according to the action information corresponding to the at least two frames of real images, respectively.

In some optional embodiments of the present application, the lens angle acquisition unit 860 is configured to acquire background music; determine the time collection corresponding to the background music, where the time collection includes at least two time periods; and obtain the corresponding time period in the time collection Lens angle of view.

In some optional embodiments of the present application, at least two different lens angles include a first lens angle of view and a second lens angle of view, and the splitter unit 820 is configured to render the three-dimensional virtual model with the first lens angle of view to obtain the first lens angle. Virtual image; Render the three-dimensional virtual model from the second lens perspective to obtain the second virtual image; display the image sequence formed according to the first virtual image and the second virtual image.

In some optional embodiments of the present application, the splitting unit 820 is configured to translate or rotate the three-dimensional virtual model under the first lens angle of view to obtain the three-dimensional virtual model under the second lens angle of view; and obtain the three-dimensional virtual model under the second lens angle of view. The second virtual image corresponding to the three-dimensional virtual model.

In some optional embodiments of the present application, the mirror splitting unit 820 is configured to insert a frame of virtual image between the first virtual image and the second virtual image, so that the first virtual image is gently switched to the second virtual image, wherein, a is a positive integer.

In some optional embodiments of the present application, the above-mentioned device further includes: a beat detection unit 870 configured to perform beat detection on the background music to obtain a beat collection of the background music, wherein the beat collection includes multiple beats, Each beat corresponds to a stage special effect; the stage special effect generation unit 880 is configured to add the target stage special effect corresponding to the beat collection to the three-dimensional virtual model.

The above-mentioned split-mirror effect realization device generates a three-dimensional virtual model according to the collected real image, and obtains multiple lens perspectives according to the collected real image, background music, and the actions of real characters, and uses multiple lens perspectives to perform the three-dimensional virtual model. The corresponding lens angle of view is switched to simulate the effect of multiple virtual cameras shooting the 3D virtual model in the virtual scene, so that the user can see the 3D virtual model under different lens angles, which improves the viewer’s viewing experience . In addition, the device also analyzes the beats of the background music and adds corresponding stage effects to the three-dimensional virtual model according to the beat information to present different stage effects to the audience, which further enhances the audience's live viewing experience.

Referring to FIG. 9, an embodiment of the present application provides a schematic structural diagram of an electronic device 900, and the foregoing device for implementing the split-mirror effect is applied to the electronic device 900. The electronic device 900 includes a processor 910, a communication interface 920, and a memory 930, where the processor 910, the communication interface 920, and the memory 930 can be coupled through a bus 940. among them,

The processor 910 may be a central processing unit (CPU), a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices (Programmable Logic Device, PLD), transistor logic devices, hardware components, or any combination thereof. The processor 910 may implement or execute various exemplary methods described in conjunction with the disclosure of the present application. Specifically, the processor 910 reads the program code stored in the memory 930, and cooperates with the communication interface 920 to execute part or all of the steps of the method executed by the device for implementing the split-mirror effect in the foregoing embodiment of the present application.

The communication interface 920 can be a wired interface or a wireless interface for communicating with other modules or devices. The wired interface can be an Ethernet interface, a controller area network interface, a local interconnect network (Local Interconnect Network, LIN), and a FlexRay interface. The interface can be a cellular network interface or a wireless local area network interface. Specifically, the aforementioned communication interface 920 may be connected to an input/output device 950, and the input/output device 950 may include other terminal devices such as a mouse, a keyboard, and a microphone.

The memory 930 may include a volatile memory, such as a random access memory (Random Access Memory, RAM); the memory 930 may also include a non-volatile memory (Non-Volatile Memory), such as a read-only memory (Read-Only Memory, ROM). ), flash memory, hard disk (Hard Disk Drive, HDD), or solid-state hard disk (Solid-State Drive, SSD), and the memory 930 may also include a combination of the foregoing types of memory. The memory 930 may store program codes and program data. Wherein, the program code is composed of the codes of some or all of the units in the above-mentioned mirror effect realization device 800, for example, the code of the acquisition unit 810, the code of the mirror unit 820, the code of the feature extraction unit 830, and the 3D virtual model generation unit 840 The code of the 3D virtual scene image construction unit 850, the code of the lens angle acquisition unit 860, the code of the beat detection unit 870, the code of the stage special effect generation unit 880, and so on. The program data is data generated during the operation of the split-mirror effect realization device 800, such as real image data, three-dimensional virtual model data, lens angle data, background music data, virtual image data, and so on.

The bus 940 may be a Controller Area Network (CAN) or other internal bus that implements interconnection between various systems or devices in the vehicle. The bus 940 can be divided into an address bus, a data bus, a control bus, and so on. For ease of representation, the figure is only represented by a thick line, but it does not mean that there is only one bus or one type of bus.

It should be understood that the electronic device 900 may include more or fewer components than those shown in FIG. 9, or may have different component configurations.

The embodiment of the present application also provides a computer-readable storage medium. The above-mentioned computer-readable storage medium stores a computer program, and the above-mentioned computer program is executed by hardware (such as a processor, etc.) to realize part or All steps.

The embodiment of the present application also provides a computer program product. When the computer program product runs on the above-mentioned device or electronic device for realizing the split-mirror effect, it executes part or all of the steps of the method for realizing the above-mentioned split-mirror effect.

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented by software, it can be implemented in the form of a computer program product in whole or in part. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website, computer, server, or data center. Transmission to another website site, computer, server or data center via wired (such as coaxial cable, optical fiber, digital subscriber line) or wireless (such as infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or a data center integrated with one or more available media. The usable medium may be a magnetic medium (for example, a floppy disk, a storage disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, an SSD). In the embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in an embodiment, reference may be made to related descriptions of other embodiments.

In the several embodiments provided in this application, it should be understood that the disclosed device may also be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division, and there may be other divisions in actual implementation, for example, multiple units or components can be combined or integrated. To another system, or some features can be ignored or not implemented. In addition, the displayed or discussed indirect coupling or direct coupling or communication connection between each other may be through some interfaces, indirect coupling or communication connection between devices or units, and may be in electrical or other forms.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed to multiple network units. . Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions in the embodiments of the present application.

In addition, the functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The integrated unit may be implemented in the form of hardware or software functional unit.

If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solution of the application essentially or the part that contributes to the existing technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium. A number of instructions are included to enable a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage media may include, for example, various media capable of storing program codes, such as U disk, mobile hard disk, read-only memory, random access memory, magnetic disk or optical disk.

The above are only optional implementations of the embodiments of the present application, but the protection scope of the embodiments of the present application is not limited thereto. Any person skilled in the art can easily think of within the technical scope disclosed in the present application. Various equivalent modifications or replacements shall be covered within the protection scope of this application. Therefore, the protection scope of the embodiments of the present application should be subject to the protection scope of the claims.

Claims

A method for realizing the split-mirror effect includes:

Obtain a three-dimensional virtual model;

The three-dimensional virtual model is rendered with at least two different lens angles to obtain virtual images corresponding to at least two different lens angles.
The method according to claim 1, wherein the three-dimensional virtual model comprises a three-dimensional virtual character model in a three-dimensional virtual scene model, and before the obtaining the three-dimensional virtual model, the method further comprises:

Acquiring a real image, where the real image includes an image of a real person;

Performing feature extraction on the real person image to obtain feature information, where the feature information includes action information of the real person;

The three-dimensional virtual model is generated according to the characteristic information, so that the action information of the three-dimensional virtual character model in the three-dimensional virtual model corresponds to the action information of the real character.
The method according to claim 2, wherein said obtaining a real image comprises:

Obtaining a video stream, and obtaining at least two frames of the real image according to at least two frames of images in the video stream;

The performing feature extraction on the real person image to obtain feature information includes:

Perform feature extraction on each frame of the real person image to obtain corresponding feature information.
The method according to claim 3, wherein the real image further comprises a real scene image, and the three-dimensional virtual model further comprises the three-dimensional virtual scene model; before the obtaining the three-dimensional virtual model, the method further comprises:

According to the real scene image, the three-dimensional virtual scene model is constructed.
The method according to claim 3 or 4, wherein acquiring the at least two different lens angles includes:

According to the at least two frames of the real image, the at least two different lens angles of view are obtained.
The method according to claim 3 or 4, wherein acquiring the at least two different lens angles includes:

The at least two different lens angles of view are obtained according to the action information corresponding to the at least two frames of the real images respectively.
The method according to claim 3 or 4, wherein acquiring the at least two different lens angles includes:

Get background music;

Determining a time collection corresponding to the background music, wherein the time collection includes at least two time periods;

Obtain the lens angle of view corresponding to each time period in the time collection.
The method according to claim 1, wherein the at least two different lens angles include a first lens angle of view and a second lens angle of view; the rendering of the three-dimensional virtual model with at least two different lens angles, Obtain at least two virtual images corresponding to different lens angles, including:

Rendering the three-dimensional virtual model with the first lens perspective to obtain a first virtual image;

Rendering the three-dimensional virtual model with the second lens perspective to obtain a second virtual image;

The image sequence formed according to the first virtual image and the second virtual image is displayed.
The method according to claim 8, wherein said rendering said three-dimensional virtual model with said second lens angle of view to obtain a second virtual image comprises:

Translate or rotate the three-dimensional virtual model in the first lens angle of view to obtain the three-dimensional virtual model in the second lens angle of view;

Acquiring the second virtual image corresponding to the three-dimensional virtual model in the second lens angle of view.
The method according to claim 9, wherein the presenting the image sequence formed according to the first image and the second virtual image comprises:

A frame of virtual image is inserted between the first virtual image and the second virtual image, so that the first virtual image is gently switched to the second virtual image, where a is a positive integer.
The method according to any one of claims 7 to 10, wherein the method further comprises:

Performing beat detection on the background music to obtain a beat collection of the background music, wherein the beat collection includes multiple beats, and each beat in the multiple beats corresponds to a stage special effect;

The target stage special effect corresponding to the beat collection is added to the three-dimensional virtual model.
A device for realizing split-mirror effect, including:

The obtaining unit is configured to obtain a three-dimensional virtual model;

The mirror splitting unit is configured to render the three-dimensional virtual model with at least two different lens angles to obtain virtual images corresponding to at least two different lens angles.
The device according to claim 12, wherein the three-dimensional virtual model comprises a three-dimensional virtual character model in a three-dimensional virtual scene model; the device further comprises: a feature extraction unit and a three-dimensional virtual model generation unit; wherein, the acquisition The unit is further configured to acquire a real image before acquiring the three-dimensional virtual model, wherein the real image includes an image of a real person;

The feature extraction unit is configured to perform feature extraction on the real person image to obtain feature information, where the feature information includes action information of the real person;

The three-dimensional virtual model generating unit is configured to generate the three-dimensional virtual model according to the characteristic information, so that the action information of the three-dimensional virtual character model in the three-dimensional virtual model corresponds to the action information of the real character.
The apparatus according to claim 13, wherein the acquisition unit is configured to acquire a video stream, and obtain at least two frames of the real image according to at least two frames of the image in the video stream;

The feature extraction unit is configured to perform feature extraction on each frame of the real person image to obtain corresponding feature information.
The apparatus according to claim 14, wherein the real image further comprises a real scene image, and the three-dimensional virtual model further comprises the three-dimensional virtual scene model;

The device further includes a three-dimensional virtual scene image construction unit configured to construct the three-dimensional virtual scene model according to the real scene image before the acquisition unit acquires the three-dimensional virtual model.
The device according to claim 14 or 15, wherein the device further comprises a lens angle acquisition unit configured to obtain the at least two different lens angles according to the at least two frames of the real image.
The device according to claim 14 or 15, wherein the device further comprises a lens angle acquisition unit configured to obtain the at least two different lenses according to the action information corresponding to the at least two frames of the real images. Perspective.
The device according to claim 14 or 15, wherein the device further comprises a lens angle acquisition unit configured to acquire background music; determine a time collection corresponding to the background music, wherein the time collection includes at least two times Segment; obtain the lens angle of view corresponding to each time segment in the time collection.
The device according to claim 12, wherein the at least two different lens angles include a first lens angle of view and a second lens angle of view; the splitting unit is configured to view the three-dimensional view from the first lens angle of view. The virtual model is rendered to obtain a first virtual image; the three-dimensional virtual model is rendered with the second lens angle of view to obtain a second virtual image; it is displayed based on the first virtual image and the second virtual image Image sequence.
The device according to claim 19, wherein the splitting unit is configured to translate or rotate the three-dimensional virtual model in the first lens angle of view to obtain the three-dimensional virtual model in the second lens angle of view. Virtual model; acquiring the second virtual image corresponding to the three-dimensional virtual model in the second lens angle of view.
The device according to claim 20, wherein the splitting unit is configured to insert a frame of virtual image between the first virtual image and the second virtual image, so that the first virtual image is switched smoothly To the second virtual image, where a is a positive integer.
The device according to any one of claims 18 to 21, wherein the device further comprises: a beat detection unit and a stage special effect generation unit; wherein the beat detection unit is configured to perform beat detection on the background music, Obtaining a collection of beats of the background music, wherein the collection of beats includes multiple beats, and each beat of the multiple beats corresponds to a stage special effect;

The stage special effect generating unit is configured to add the target stage special effect corresponding to the beat collection to the three-dimensional virtual model.
An electronic device comprising: a processor, a communication interface, and a memory; the memory is used to store instructions, the processor is used to execute the instructions, and the communication interface is used to control the processor Communicate with other devices under the following conditions, wherein the processor implements the method according to any one of claims 1 to 11 when the processor executes the instructions.
A computer-readable storage medium storing a computer program, and the computer program is executed by hardware to implement the method according to any one of claims 1 to 11.
A computer program product that is read and executed by a computer to implement the method described in any one of claims 1 to 11.