TW202123178A - Method for realizing lens splitting effect, device and related products thereof - Google Patents

Abstract

The embodiment of the present application discloses a method for realizing a lens splitting effect, device, and related products. The method includes: acquiring a three-dimensional virtual model; rendering the three-dimensional virtual model with at least two different lens angles to obtain at least two Virtual images corresponding to different lens angles.

Description

Method, device and related products for realizing split-mirror effect

This application is filed based on a Chinese patent application with an application number of 201911225211.4 and an application date of December 3, 2019, and claims the priority of the Chinese patent application. The entire content of the Chinese patent application is hereby incorporated into this application by way of introduction. This application relates to the field of virtual technology, and in particular to a method, device and related products for realizing the split-mirror effect.

In recent years, "virtual characters" have frequently appeared in our lives, for example, the well-known applications of virtual idols such as "Hatsune Miku" and "Luo Tianyi" in the field of music, or the application of virtual hosts in live news and many more. Since virtual characters can replace real characters in activities in the online world, and users can set their own appearance, shape, etc. according to their needs, virtual characters have gradually become a way of communication between people.

At present, the virtual characters in the Internet generally use motion capture technology during the generation process, and analyze the captured images of real people through image recognition methods, so as to direct the actions and expressions of the real characters to the virtual characters, making Virtual characters can reproduce the actions and expressions of real characters.

The embodiments of the present application disclose a method, device and related products for realizing the split-mirror effect.

In the first aspect, an embodiment of the present application provides a method for implementing the splitting effect, including: obtaining a three-dimensional virtual model; rendering the three-dimensional virtual model with at least two different lens angles to obtain at least two different lens angles corresponding to each Virtual image.

The above method obtains a three-dimensional virtual model and renders the three-dimensional virtual model with at least two different lens angles, thereby obtaining virtual images corresponding to at least two different lens angles, so that the user can see different lens angles. The virtual image below brings a rich visual experience to the user.

In some optional embodiments of the present application, the three-dimensional virtual model includes a three-dimensional virtual character model in a three-dimensional virtual scene model. Before obtaining the three-dimensional virtual model, the above method further includes: obtaining a real image, where the real image includes Real person images; feature extraction of real person images to obtain feature information, where the feature information includes the action information of the real person; generate a three-dimensional virtual model based on the feature information, so that the action information of the three-dimensional virtual character model in the three-dimensional virtual model Corresponds to the action information of real characters.

It can be seen that by extracting the features of the collected real person images, a 3D virtual model is generated, so that the 3D virtual person model in the 3D virtual model can reproduce the facial expressions and body movements of the real person, which is convenient for the audience to watch through The virtual image corresponding to the three-dimensional virtual model can learn the facial expressions and body movements of the real person, so that the audience can interact more flexibly with the live anchor.

In some optional embodiments of the present application, obtaining the real image includes: obtaining a movie stream, obtaining at least two real images from at least two frames of the movie stream; performing feature extraction on the real person image to obtain feature information , Including: extracting features of each frame of real person images to obtain corresponding feature information.

It can be seen that the three-dimensional virtual model can be changed in real time according to the multiple frames of real images acquired, so that the user can see the dynamic change process of the three-dimensional virtual model under different lens perspectives.

In some optional embodiments of the present application, the real image further includes a real scene image, and the three-dimensional virtual model also includes a three-dimensional virtual scene model; before acquiring the three-dimensional virtual model, the above method further includes: constructing the real scene image according to the real scene image. Three-dimensional virtual scene model.

It can be seen that the above method can also use real scene images to construct three-dimensional virtual scene images in the three-dimensional virtual model. Compared with the selection of specific three-dimensional virtual scene images, the choice of three-dimensional virtual scene images More sex.

In some optional embodiments of the present application, acquiring at least two different lens angles includes: obtaining at least two different lens angles according to at least two frames of real images.

It can be seen that each frame of real image corresponds to one lens angle, and multiple frames of real image correspond to multiple lens angles. Therefore, at least two frames of different lens angles can be obtained based on at least two frames of real images, which can be used to realize a three-dimensional virtual model. The lens perspective rendering provides users with a rich visual experience.

In some optional embodiments of the present application, acquiring at least two different lens angles includes: obtaining at least two different lens angles according to the action information corresponding to the at least two frames of real images.

It can be seen that determining the lens angle of view based on the action information of the real person in the real image can magnify the action of the corresponding three-dimensional virtual person model in the image, so that the user can learn the real person’s behavior by watching the virtual image. Action to improve interactivity and fun.

In some optional embodiments of the present application, acquiring at least two different camera angles includes: acquiring background music; determining a time collection corresponding to the background music, where the time collection includes at least two time periods; acquiring each of the time collections The lens angle of view corresponding to the time period.

It can be seen that in the above method, by analyzing the background music and determining the time collection corresponding to the background music, multiple different lens perspectives can be obtained. Through this method, the diversity of the lens perspective can be increased, so that the user can get more abundant Visual experience.

In some optional embodiments of the present application, the at least two different lens angles include a first lens angle of view and a second lens angle of view; the three-dimensional virtual model is rendered with at least two different lens angles to obtain at least two different lens angles. The virtual images corresponding to the lens perspectives respectively include: rendering the three-dimensional virtual model with the first lens perspective to obtain the first virtual image; rendering the three-dimensional virtual model with the second lens perspective to obtain the second virtual image; and displaying An image sequence formed according to the first virtual image and the second virtual image.

It can be seen that rendering the three-dimensional virtual model with the first lens perspective and the second lens perspective respectively allows the user to view the three-dimensional virtual model in the first lens perspective and the three-dimensional virtual model in the second lens perspective, thus The user provides a rich visual experience.

In some optional embodiments of the present application, rendering the three-dimensional virtual model in the second lens perspective to obtain the second virtual image includes: translating or rotating the three-dimensional virtual model in the first lens perspective to obtain the second The three-dimensional virtual model under the lens angle of view; the second virtual image corresponding to the three-dimensional virtual model under the second lens angle of view is acquired.

It can be seen that by translating or rotating the three-dimensional virtual model under the first lens angle of view, the three-dimensional virtual model under the second lens angle of view, that is, the second virtual image, can be quickly and accurately obtained.

In some optional embodiments of the present application, displaying the image sequence formed according to the first image and the second virtual image includes: inserting a virtual image between the first virtual image and the second virtual image Image, so that the first virtual image is gently switched to the second virtual image, where a is a positive integer.

It can be seen that the a-frame virtual image is inserted between the first virtual image and the second virtual image, so that the viewer can see the entire change process from the first virtual image to the second virtual image, rather than a single virtual image. The two images (the first virtual image and the second virtual image), so that the audience can adapt to the visual difference caused by the first virtual image to the second virtual image.

In some optional embodiments of the present application, the method further includes: performing beat detection on the background music to obtain a beat collection of the background music, where the beat collection includes multiple beats, and each beat of the multiple beats corresponds to a stage special effect ; Add the target stage special effects corresponding to the beat collection to the 3D virtual model.

It can be seen that corresponding stage effects are added to the virtual scene where the virtual character model is located according to the beat information of the music, thereby presenting different stage effects to the audience and enhancing the audience's viewing experience.

In a second aspect, an embodiment of the present application also provides a device for implementing a splitting effect, including: an acquiring unit configured to acquire a three-dimensional virtual model; and a splitting unit configured to perform a three-dimensional virtual model with at least two different lens angles. Rendering to obtain virtual images corresponding to at least two different lens angles respectively.

In some optional embodiments of the present application, the three-dimensional virtual model includes a three-dimensional virtual character model in a three-dimensional virtual scene model, and the device further includes: a feature extraction unit and a three-dimensional virtual model generation unit; wherein, the acquisition unit is also configured to Before acquiring the three-dimensional virtual model, acquire a real image, where the real image includes an image of a real person; the feature extraction unit is configured to perform feature extraction on the image of the real person to obtain feature information, where the feature information includes the action information of the real person ; The three-dimensional virtual model generating unit is configured to generate a three-dimensional virtual model according to the feature information, so that the action information of the three-dimensional virtual character model in the three-dimensional virtual model corresponds to the action information of the real character.

In some optional embodiments of the present application, the acquiring unit is configured to acquire a film stream, and obtain at least two frames of real images according to at least two frames of images in the film stream; Image feature extraction is performed to obtain corresponding feature information.

In some optional embodiments of the present application, the real image further includes a real scene image, and the three-dimensional virtual model also includes a three-dimensional virtual scene model; the device further includes: a three-dimensional virtual scene image construction unit configured to obtain a three-dimensional virtual scene image in the acquisition unit Before the virtual model, a three-dimensional virtual scene image is constructed according to the real scene image.

In some optional embodiments of the present application, the device further includes a lens angle acquisition unit configured to obtain at least two different lens angles according to at least two frames of real images.

In some optional embodiments of the present application, the device further includes a lens angle acquisition unit configured to obtain at least two different lens angles according to the action information corresponding to the at least two frames of real images.

In some optional embodiments of the present application, the device further includes a lens angle acquisition unit configured to acquire background music; determine a time collection corresponding to the background music, where the time collection includes at least two time periods; and acquire each time in the time collection The lens angle of view corresponding to the segment.

In some optional embodiments of the present application, at least two different lens angles include a first lens angle of view and a second lens angle of view, and the splitting unit is configured to render the three-dimensional virtual model with the first lens angle of view to obtain the first lens angle. Virtual image; Render the three-dimensional virtual model from the second lens perspective to obtain the second virtual image; display the image sequence formed according to the first virtual image and the second virtual image.

In some optional embodiments of the present application, the splitting unit is configured to translate or rotate the three-dimensional virtual model under the first lens angle of view to obtain the three-dimensional virtual model under the second lens angle of view; and obtain the three-dimensional virtual model under the second lens angle of view. The second virtual image corresponding to the three-dimensional virtual model.

In some optional embodiments of the present application, the mirror splitting unit is configured to insert a frame of virtual image between the first virtual image and the second virtual image, so that the first virtual image is gently switched to the second virtual image. Image, where a is a positive integer.

In some optional embodiments of the present application, the device further includes: a beat detection unit and a stage special effect generation unit; wherein the beat detection unit is configured to perform beat detection on the background music to obtain a beat collection of the background music, wherein the beat collection Including multiple beats, each of the multiple beats corresponds to a stage special effect; the stage special effect generation unit is configured to add the target stage special effect corresponding to the beat collection to the three-dimensional virtual model.

In a third aspect, an embodiment of the present application provides an electronic device, including: a processor, a communication interface, and a memory; the memory is used to store instructions, the processor is used to execute instructions, and the communication interface is used to communicate with other devices under the control of the processor. The device communicates, wherein when the processor executes the instruction, the electronic device implements any one of the methods in the first aspect.

In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium that stores a computer program, and the computer program is executed by hardware to implement any one of the methods in the first aspect.

In the fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product is read and executed by a computer, any one of the methods in the first aspect described above is executed.

The terms used in the embodiments of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

The implementation method, device, and related products of the split-mirror effect provided by the embodiments of the present application can be applied in many fields such as social interaction, entertainment, education, etc., for example, can be used for virtual live broadcast, social interaction in virtual communities, or Used to hold virtual concerts, can also be used in classroom teaching and so on. In order to facilitate the understanding of the embodiments of the present application, the following takes virtual live broadcast as an example to describe the specific application scenarios of the embodiments of the present application in detail.

Virtual live broadcast is a way to use virtual characters instead of live anchors to conduct live broadcasts on a live broadcast platform. Because virtual characters have rich expressive power and are more in line with the communication environment of social networks, the virtual live broadcast industry is developing rapidly. In the process of virtual live broadcast, computer technologies such as facial expression capture, motion capture, and sound processing are usually used to apply the facial expressions and actions of the live anchor to the virtual character model, so as to realize the audience and the virtual anchor on the movie website or social network. Interaction in the website.

In order to save live broadcast costs and post-production costs, users usually directly use mobile phones, tablet computers and other terminal devices for live broadcasts. Please refer to Figure 1. Figure 1 is a schematic diagram of a specific application scenario provided by an embodiment of the present application. The character image is sent to the server 120 for processing through the network, and the server 120 sends the generated virtual image to the user terminal 130, so that different viewers can watch the entire live broadcast process through the corresponding user terminal 130.

It can be seen that although the cost of virtual live broadcast in this way is relatively low, since only a single camera device 110 shoots the live anchor, the posture of the generated virtual anchor is related to the relative position between the camera device 110 and the live anchor. That is to say, the audience can only see the virtual character in a specific angle of view, and this specific angle of view depends on the relative position between the photographing device 110 and the live broadcaster, so that the live broadcast effect presented is unsatisfactory. For example, in the process of virtual live broadcast, there are often problems such as stiff movements of virtual anchors, unsmooth shot switching screens, or monotonous and boring shots, which cause visual fatigue of the audience and make it impossible for the audience to experience the immersive experience.

Similarly, in other application scenarios, for example, live teaching scenes; in the teaching process, teachers teach students knowledge through online teaching, but this teaching method is usually boring, and the teacher in the film cannot know how students are right. For the mastery of knowledge points, students can only see the teacher or teaching handouts in a single perspective, which can easily cause students to feel tired. Compared with the teacher’s on-site teaching, the teaching effect of film teaching is greatly reduced. For another example, when the concert may not be held as scheduled due to weather and venue restrictions during the concert, the singer can hold a virtual concert in the recording studio to simulate the scene of a real concert, in order to achieve a real concert It is usually necessary to set up multiple cameras to shoot the singer. This kind of virtual concert is complicated to operate and wastes the cost. Moreover, the use of multiple cameras to shoot can get pictures under multiple lenses, which may cause lens switching The problem of non-smoothness makes the user unable to adapt to the visual difference caused by the switching of different lenses.

In order to solve the problems of single lens angle of view and unsmooth lens switching images that often occur in the above application scenarios, an embodiment of the present application provides a method for realizing the splitting effect. The method generates a three-dimensional virtual image based on the collected real image. Model, and obtain multiple different lens perspectives according to background music or the actions of real characters, and then render the three-dimensional virtual model with multiple different lens perspectives to obtain virtual images corresponding to multiple different lens perspectives to simulate The effect of multiple virtual cameras shooting the three-dimensional virtual model in the virtual scene improves the viewing experience of the audience. In addition, the method also analyzes the beats of the background music and adds corresponding stage effects to the three-dimensional virtual model according to the beat information to present different stage effects to the audience, which further enhances the audience's viewing experience.

In the following, the specific process of generating a three-dimensional virtual model from a real image in an embodiment of the present application is first explained.

In the embodiment of the present application, the three-dimensional virtual model includes a three-dimensional virtual character model in a three-dimensional virtual scene. Taking Figure 2 as an example, Figure 2 shows a schematic diagram of a possible three-dimensional virtual model. According to the three-dimensional virtual model shown in Figure 2, it can be seen that the hands of the three-dimensional virtual character model are raised to the chest, in order to highlight the contrast The effect, the upper left corner of the second figure also shows the real image collected by the splitter effect realization device, in which the real person is also raising his hands to his chest. In other words, the three-dimensional virtual character model is consistent with the actions of the real character. It can be understood that the above-mentioned Figure 2 is only an example. In practical applications, the real image collected by the device for implementing the split-mirror effect can be a three-dimensional image or a two-dimensional image, and the number of characters in the real image can be One or more. The action of the real character can be raising both hands to the chest, or raising the left foot or other actions, etc. Correspondingly, the three-dimensional virtual model generated from the image of the real character The number of character models can be one or more. The action of the three-dimensional virtual character model can be raising both hands to the chest, raising the left foot or other actions, etc., which are not specifically limited here.

，其中n是正整數，並且真實圖像I₁ ,I₂ ,…,I_n 與三維虛擬模型

之間存在一一對應的關係，也就是說，一幀真實圖像用於生成一個三維虛擬模型。示例性的，以真實圖像

生成三維虛擬模型

為例，一個三維虛擬模型是可以這樣得到的：In the present application example, the sub-mirror to achieve the effect of real people shooting apparatus, a plurality of frames to obtain a real image _{I 1, I 2, ...,} I n, and in chronological order of the real image I _1, I _2, ... ,I _n perform feature extraction respectively to obtain multiple corresponding three-dimensional virtual models

, Where n is a positive integer, and the real images I ₁ , I ₂ ,..., I _{n are the} same as the three-dimensional virtual model

There is a one-to-one correspondence, that is, a real image is used to generate a three-dimensional virtual model. Exemplary, with real images

Generate 3D virtual model

As an example, a 3D virtual model can be obtained like this:

。Step one, the real image of the realizing device of the split-mirror effect is obtained

.

中包括真實人物圖像，並且i是正整數，

。Among them, the real image

Includes images of real people, and i is a positive integer,

.

中的真實人物圖像進行特徵提取，得到特徵資訊。其中，特徵資訊包括真實人物的動作資訊。Step two, the split mirror effect is realized by the device on the real image

Feature extraction is performed on real person images in, and feature information is obtained. Among them, the feature information includes action information of real characters.

Wherein, acquiring the real image includes: acquiring a movie stream, and obtaining at least two frames of real images according to at least two frames of images in the movie stream; correspondingly, performing feature extraction on the real person image to obtain feature information includes: respectively Perform feature extraction on each frame of the real person image to obtain corresponding feature information.

Understandably, the feature information is used to control the posture of the three-dimensional virtual character model. The action information in the feature information includes facial expression features and body motion features. Facial expression features are used to describe various emotional states of the character, such as happiness and sadness. , Surprise, fear, anger, or disgust, etc., body movement characteristics are used to describe the movement state of real characters, for example, raising the left hand, raising the right foot, or jumping, etc. In addition, the feature information can also include character information, where the character information includes multiple key points of the human body of a real person and their corresponding position information. The key points of the human body include key points of the human face and the key points of human bones, and the position characteristics include the key points of the real person. The position coordinates of the key points of the human body.

進行圖像分割，提取得到真實圖像

中的真實人物圖像；再對提取得到的真實人物圖像進行關鍵點檢測得到上述多個人體關鍵點以及多個人體關鍵點的位置資訊，其中，上述人體關鍵點包括人臉關鍵點和人體骨骼關鍵點，上述人體關鍵點具體可以位於人體的頭部區域、脖子區域、肩膀區域、脊柱區域、腰部區域、臀部區域、手腕區域、手臂區域、膝蓋區域、腿部區域、腳腕區域以及腳掌區域等等；透過對人臉關鍵點以及人臉關鍵點的位置資訊進行分析，得到真實圖像

中真實人物的臉部表情特徵；透過對人體骨骼關鍵點以及人體骨骼關鍵點的位置資訊進行分析，得到真實圖像

中真實人物的骨骼特徵，從而得到真實人物的肢體動作特徵。Optionally, the split-mirror effect realization device penetrates the real image

Perform image segmentation and extract the real image

The real person image in the image; then the key point detection is performed on the extracted real person image to obtain the above-mentioned multiple human body key points and the position information of the multiple human body key points, where the human body key points include the face key points and the human body Bone key points, the above human key points can be located in the head area, neck area, shoulder area, spine area, waist area, hip area, wrist area, arm area, knee area, leg area, ankle area, and sole of the human body. Area, etc.; through the analysis of the key points of the face and the location information of the key points of the face, the real image is obtained

Facial expression features of real people in the middle; through analyzing the key points of human bones and the position information of key points of human bones, real images are obtained

In order to obtain the characteristics of the body movements of the real person in the bone features of the real person.

輸入神經網路中進行特徵提取，經過多個卷積層的計算後，提取得到上述多個人體關鍵點資訊。其中，神經網路是透過大量的訓練得到的，神經網路可以是卷積神經網路（Convolution Neural Network, CNN），也可以是反向傳播神經網路（Back Propagation Neural Network, BPNN），還可以是生成對抗網路（Generative Adversarial Network, GAN）或者迴圈神經網路（Recurrent Neural Network, RNN）等等，此處不作具體限定。需要說明的，上述人體特徵的提取過程可以在同一個神經網路中進行，也可以在不同神經網路中進行。例如，分鏡效果實現裝置可以利用CNN提取人臉關鍵點，得到人體臉部表情特徵；也可以利用BPNN提取人體骨骼關鍵點，得到人體骨骼特徵以及肢體動作特徵，此處不作具體限定。另外，上述用於驅動三維虛擬人物模型的特徵資訊的示例僅僅用於進行舉例，在實際應用中還可以包括其他特徵資訊，此處不作具體限定。Optionally, the split-mirror effect realization device converts the real image

Input the neural network for feature extraction, and after multiple convolutional layer calculations, the multiple key point information of the human body is extracted. Among them, the neural network is obtained through a lot of training. The neural network can be a Convolution Neural Network (CNN) or a Back Propagation Neural Network (BPNN). It can be a Generative Adversarial Network (GAN) or a Recurrent Neural Network (RNN), etc., and there is no specific limitation here. It should be noted that the above-mentioned human body feature extraction process can be performed in the same neural network or in different neural networks. For example, the device for realizing the split-mirror effect can use CNN to extract key points of human faces to obtain facial expression features; it can also use BPNN to extract key points of human bones to obtain human bone features and limb movement features, which are not specifically limited here. In addition, the above examples of the feature information used to drive the three-dimensional virtual character model are only used as examples, and other feature information may also be included in practical applications, which is not specifically limited here.

中的三維虛擬人物模型，以使得三維虛擬模型

中的三維虛擬人物模型與真實圖像

中真實人物的動作資訊對應。Step 3: The device for achieving split-mirror effect generates a three-dimensional virtual model based on the feature information

3D virtual character model in the 3D virtual model to make the 3D virtual model

3D virtual character model and real image in

Corresponding to the action information of real characters in the game.

Optionally, the split-mirror effect realization device establishes the mapping relationship between the key points of the human body of the real person and the key points of the human body of the virtual character model through the above-mentioned feature information; and then controls the expression and posture of the virtual character model according to the mapping relationship, so as to make the virtual character model The facial expressions and body movements of the character model are consistent with the facial expressions and body movements of the real characters.

Optionally, the split-mirror effect realization device respectively performs serial number labeling on the key points of the human body of the real person to obtain the annotation information of the key points of the human body of the real person. Among them, the key points of the human body correspond to the annotation information one by one; The annotation information of the key points is used to mark the key points of the human body in the virtual character model. For example, if the label information of the left wrist of the real character is No. 1, the label information of the left wrist of the 3D virtual character model is also No. 1, and the label information of the left arm of the real character is No. 2, then the left wrist of the 3D virtual character model The annotation information is also No. 2 and so on; then the human key point annotation information of the real person is matched with the human key point annotation information of the three-dimensional virtual character model, and the key point position information of the real person’s human body is mapped to the corresponding three-dimensional virtual character model In the key points of the human body, the three-dimensional virtual character model can reproduce the facial expressions and body movements of real characters.

還包括真實場景圖像，三維虛擬模型

還包括三維虛擬場景模型，上述根據真實圖像

生成三維虛擬模型

的方法還包括：根據真實圖像

中的真實場景圖像，構建三維虛擬模型

中的三維虛擬場景。In the embodiment of this application, the real image

It also includes real scene images, 3D virtual models

It also includes a three-dimensional virtual scene model, the above is based on real images

Generate 3D virtual model

The method also includes: According to the real image

The real scene image in, construct a three-dimensional virtual model

In the 3D virtual scene.

進行圖像分割，得到真實圖像

中的真實場景圖像；再提取真實場景圖像中的場景特徵，例如，真實場景中物體的位置特徵、形狀特徵以及大小特徵等等；根據場景特徵構建三維虛擬模型

中的三維虛擬場景模型，使得三維虛擬模型

中的三維虛擬場景模型可以高度還原真實圖像

中的真實場景圖像。Optionally, the device for realizing the split-mirror effect first

Perform image segmentation to get real images

The real scene image in the real scene image; then extract the scene features from the real scene image, for example, the position feature, shape feature, and size feature of the object in the real scene; build a three-dimensional virtual model according to the scene feature

The three-dimensional virtual scene model in the three-dimensional virtual model, making the three-dimensional virtual model

The three-dimensional virtual scene model in can highly restore the real image

The real scene image in.

生成三維虛擬模型

的過程，實際上，三維虛擬模型

的生成過程與三維虛擬模型

的生成過程類似，此處不再展開贅述。For the sake of simplicity, the above only illustrates that the real image

Generate 3D virtual model

The process, in fact, the three-dimensional virtual model

Generation process and 3D virtual model

The generation process is similar, so I won’t go into details here.

It should be noted that the 3D virtual scene model in the 3D virtual model can be constructed based on the real scene image in the real image, or it can be a user-defined 3D virtual scene model; the facial features of the 3D virtual character model in the 3D virtual model can be It is constructed from the facial features of the real person image in the real image, or it can be a user-defined facial features, which is not specifically limited here.

中的每一個三維虛擬模型進行鏡頭視角渲染，使得觀眾可以看到同一個三維虛擬模型在不同鏡頭視角下的虛擬圖像進行詳細說明。以真實圖像

生成的三維虛擬模型

為例，分別使用k個不同的鏡頭對三維虛擬模型

進行渲染，得到k個不同鏡頭視角下的虛擬圖像

，其中

，從而實現分鏡切換的效果，其具體過程可以表述如下：Next, for the three-dimensional virtual model involved in the embodiment of this application with multiple different lens perspectives

Each of the three-dimensional virtual models is rendered from the perspective of the lens, so that the audience can see the virtual images of the same three-dimensional virtual model under different lens perspectives for detailed description. Real image

Generated 3D virtual model

As an example, use k different lenses to pair the 3D virtual model

Perform rendering to get k virtual images under different lens angles

,among them

, In order to achieve the effect of split mirror switching, the specific process can be expressed as follows:

As shown in FIG. 3, FIG. 3 is a schematic flowchart of a method for implementing a split mirror effect provided by an embodiment of the present application. The method for realizing the splitting effect of this embodiment includes but not limited to the following steps:

S101. The device for achieving split-mirror effect obtains a three-dimensional virtual model.

In the embodiment of the present application, the three-dimensional virtual model is used to compare a real person and a real scene. The three-dimensional virtual model includes a three-dimensional virtual character model in a three-dimensional virtual scene model, and the three-dimensional virtual model is generated based on a real image. Among them, the three-dimensional virtual character model is generated based on the real character image included in the real image, the three-dimensional virtual character model in the three-dimensional virtual model is used to compare the real character in the real image, and the actions of the three-dimensional virtual character model are the same as the real character Corresponding to the action. The three-dimensional virtual scene model may be constructed based on the real scene image included in the real image, or may be a preset three-dimensional virtual scene model. When the three-dimensional virtual scene model is constructed from the real scene image, the three-dimensional virtual scene model can be used to simulate the real scene in the real image.

S102. The device for achieving a split-mirror effect obtains at least two different lens angles of view.

，則利用該相機拍攝得到的圖像展示了鏡頭視角

下的物體，也就是物體的俯視圖。In the embodiments of the present application, the angle of view of the lens is used to indicate the position of the camera relative to the object when the camera is shooting the object. For example, the camera can get a top view of the object when shooting directly above the object. Assuming that the camera is located directly above the object, the corresponding lens angle of view is

, The image taken by the camera shows the lens angle of view

The object underneath is the top view of the object.

In some optional embodiments, obtaining at least two different lens angles includes: obtaining at least two different lens angles according to at least two frames of real images. Among them, the real image can be taken by a real camera, the position of the real camera relative to the real person may be multiple, and the multiple real images taken by multiple real cameras in different positions show multiple different lenses. Real people in perspective.

In other optional embodiments, acquiring at least two different lens angles includes: obtaining at least two different lens angles according to the action information corresponding to the at least two frames of real images. Among them, the action information includes the body movements and facial expressions of real characters in real images. Among them, the body movements include many kinds. For example, the body movements can be one or more of raising the right hand, raising the left foot, jumping, etc. The facial expressions also include many kinds, such as smiling, tearing, etc. One or more of facial expressions such as anger. Examples of body movements and facial expressions in this embodiment are not limited to the above description.

，當真實人物只跳躍時對應的鏡頭視角可以是鏡頭視角

，也可以是鏡頭視角

等等，同樣的，當真實人物只微笑時對應的鏡頭視角可以是鏡頭視角

，也可以是鏡頭視角

，還可以是鏡頭視角

等等。In the embodiment of the present application, one action or a combination of multiple actions corresponds to one lens angle of view. For example, when a real person smiles and jumps, the corresponding lens angle is

, When the real person only jumps, the corresponding lens angle can be the lens angle

, It can also be the lens angle of view

Wait, the same, when the real person only smiles, the corresponding lens angle can be the lens angle

, It can also be the lens angle of view

, It can also be the lens angle of view

and many more.

In still other optional embodiments, obtaining at least two different camera angles includes: obtaining background music; determining a time collection corresponding to the background music, where the time collection includes at least two time periods; obtaining each time period in the time collection Corresponding lens angle of view. The real image may be one or more frames in a movie stream. The movie stream includes image information and background music information, where one frame of image corresponds to one frame of music. The background music information includes background music and a corresponding time collection. The time collection includes at least two time periods, and each time period corresponds to a camera angle of view.

S103. The device for implementing the split-mirror effect renders the three-dimensional virtual model with at least two different lens angles to obtain virtual images corresponding to at least two different lens angles respectively.

In the embodiment of the present application, the aforementioned at least two different lens angles include a first lens angle of view and a second lens angle of view, and the three-dimensional virtual model is rendered with at least two different lens angles to obtain at least two different lenses The virtual images corresponding to the viewing angles include: S1031, rendering the three-dimensional virtual model from the first lens perspective to obtain the first virtual image; S1032, rendering the three-dimensional virtual model from the second lens perspective to obtain the second virtual image .

In the embodiment of the present application, rendering the three-dimensional virtual model from the second lens perspective to obtain the second virtual image includes: translating or rotating the three-dimensional virtual model in the first lens perspective to obtain the three-dimensional virtual model in the second lens perspective. Virtual model; acquiring a second virtual image corresponding to the three-dimensional virtual model under the second lens perspective.

It is understandable that the first lens angle can be obtained based on the real image, or based on the action information corresponding to the real image, or based on the time collection corresponding to the background music; similarly, the second lens The viewing angle may be obtained based on the real image, or based on the action information corresponding to the real image, or based on the time collection corresponding to the background music, which is not specifically limited in the embodiment of the present application.

S1033. Display an image sequence formed according to the first virtual image and the second virtual image.

In the embodiment of the present application, the above-mentioned displaying the image sequence formed according to the first image and the second virtual image includes: inserting a frame of virtual image between the first virtual image and the second virtual image, Make the first virtual image switch to the second virtual image smoothly, where a is a positive integer.

，使得第一虛擬圖像平緩切換至第二虛擬圖像，其中a幀虛擬圖像

插入的時間點為

，時間點

形成的曲線的斜率值滿足先單調遞減後單調遞增的函數，並且a是正整數。Optionally, insert a frame of virtual image between the first virtual image and the second virtual image

, So that the first virtual image is gently switched to the second virtual image, where a frame of virtual image

The time point of insertion is

, Point in time

The slope value of the formed curve satisfies the function of monotonically decreasing first and then increasing monotonically, and a is a positive integer.

For example, Fig. 4 shows a schematic diagram of an interpolation curve. As shown in Fig. 4, the segmentation effect realization device obtains the first virtual image at the first minute and the second virtual image at the second minute. , And the first virtual image presents the front view of the three-dimensional virtual model, and the second virtual image presents the left view of the three-dimensional virtual model. In order to allow the audience to see a smooth shot switching screen, the split-lens effect realization device inserts multiple time points between the first minute and the second minute, and inserts a virtual image at each time point, for example, in when the insertion of the virtual image P ₁ 1.4 minutes, inserting the virtual image P ₂ during the first 1.65 minutes, inserting the virtual image P of 1.8 minutes at _3, the virtual image P ₄ inserted in the first 1.85 minutes, wherein the virtual image P ₁ presents the effect of rotating the three-dimensional virtual model to the left by 30 degrees, the virtual image P ₂ presents the effect of rotating the three-dimensional virtual model to the left by 50 degrees, and the virtual image P ₃ and the virtual image P ₄ present the effect Both are the effect of rotating the 3D virtual model 90 degrees to the left, so that the audience can see the whole process of the 3D virtual model gradually changing from the front view to the left view, instead of a single two images (the front view and the front view of the 3D virtual model). The left view of the three-dimensional virtual model), so that the audience can adapt to the changing effect of the visual difference when switching from the front view to the left view.

In some optional embodiments of this application, the use of stage special effects mentioned in the embodiments of this application to render a three-dimensional virtual model to present different stage effects to the audience is described in detail, which specifically includes the following steps:

Step 1: The device for realizing the split-mirror effect detects the beats of the background music, and obtains the beat collection of the background music.

Among them, the beat collection includes multiple beats, and each beat of the multiple beats corresponds to a stage special effect. Optionally, the split-mirror effect realization device can use shaders and particle special effects to respectively render the 3D virtual model. For example, the shader can be used to realize the spotlight rotation effect on the back of the virtual stage and the sound wave effect of the virtual stage itself, and particle special effects. It is used to add similar visual effects such as sparks, fallen leaves, meteors, etc. to the 3D virtual model.

Step 2: The split-mirror effect realization device adds the target stage special effects corresponding to the beat collection to the three-dimensional virtual model.

The above method generates a three-dimensional virtual model according to the collected real images, and performs corresponding lens angle switching according to the collected real images, background music, and the actions of real characters, thus analogy with multiple virtual cameras in the virtual scene The effect of shooting the three-dimensional virtual model improves the viewer's viewing experience. In addition, the method also analyzes the beat of the background music, and adds corresponding stage effects to the virtual image according to the beat information, presenting different stage effects to the audience, and further enhancing the audience's viewing experience.

In order to facilitate the understanding of the method for realizing the split-lens effect involved in the foregoing embodiment, the method for realizing the split-lens effect of the embodiment of the present application will be described in detail below through an example.

Please refer to Fig. 5, which shows a schematic flowchart of a specific embodiment.

S201. The device for achieving split-mirror effect obtains a real image and background music, and obtains a first lens angle of view according to the real image. Among them, when the background music sounds, the real person acts according to the background music, and the real camera shoots the real person to obtain the real image.

S202. The device for realizing split-mirror effect generates a three-dimensional virtual model according to the real image. Among them, the three-dimensional virtual model is obtained at the first moment by the device for realizing the split mirror effect.

S203. The split-mirror effect realization device detects the beat of the background music to obtain the beat collection of the background music, and adds the target stage special effect corresponding to the beat collection to the three-dimensional virtual model.

S204. The device for implementing split-mirror effect renders the three-dimensional virtual model with the first lens angle of view to obtain a first virtual image corresponding to the first lens angle of view.

S205. The device for realizing the split-mirror effect determines the time collection corresponding to the background music.

Wherein, the time collection includes multiple time periods, and each of the multiple time periods corresponds to a lens angle.

S206. The split-mirror effect realization device judges whether the action information database contains action information, executes S207-S209 if the action information database does not contain action information, and executes S210-S212 if the action information database contains action information. . Wherein, the action information is the action information of the real person in the real image, and the action information database includes a plurality of action information, and each action information in the plurality of action information corresponds to a camera angle of view.

S207. The device for realizing the splitting effect determines the second lens angle corresponding to the time period at the first moment according to the time collection.

S208. The device for implementing the split-mirror effect renders the three-dimensional virtual model with the second lens angle of view to obtain a second virtual image corresponding to the second lens angle of view.

S209. The device for realizing split-mirror effect displays an image sequence formed according to the first virtual image and the second virtual image.

S210. The device for achieving split-lens effect determines a third lens angle corresponding to the action information according to the action information.

S211. The device for implementing the split-mirror effect renders the three-dimensional virtual model with the third lens angle of view to obtain a third virtual image corresponding to the third lens angle of view.

S212. The device for realizing split-mirror effect displays an image sequence formed according to the first virtual image and the third virtual image.

According to the method described in FIG. 5, the embodiment of the present application provides a schematic diagram of a splitting rule as shown in FIG. 6, and performing splitting processing and stage special effects on a virtual image according to the splitting rule shown in FIG. 6 Through processing, the effect diagrams of the four virtual images shown in Figures 7A-7D can be obtained.

（如第7A圖左上角所示），然後根據真實圖像

得到三維虛擬模型

。分鏡效果實現裝置對背景音樂進行節拍檢測，確定第1分鐘對應的節拍為B₁ ，並根據節拍B₁ 得到第1分鐘時的舞臺特效

，然後將舞臺特效

添加到三維虛擬模型

中；分鏡效果實現裝置根據預設的鏡頭腳本確定第1分鐘對應的鏡頭視角（簡稱為時間鏡頭視角）為V₁ ；分鏡效果實現裝置檢測到真實人物在第1分鐘的動作是雙手舉到胸前，並且雙手舉到胸前這個動作不在動作資訊庫中，即不存在動作對應的鏡頭視角（簡稱為動作鏡頭視角），則此時分鏡效果實現裝置上顯示如第7A圖所示的虛擬圖像，其中，第7A圖所示虛擬圖像和真實圖像

的鏡頭視角相同。As shown in Fig. 7A, at the first minute, the split-lens effect realization device shoots a real person under the _{lens angle of view V 1 to obtain a real image}

(As shown in the upper left corner of Figure 7A), and then according to the real image

Get 3D virtual model

. The split-mirror effect realization device detects the beat of the background music, determines that the beat corresponding to the first minute is B ₁ , and obtains the stage special effects at the first minute according to the beat B ₁

, And then put the stage effects

Add to 3D virtual model

Medium; the splitting effect realization device determines the lens angle corresponding to the first minute (referred to as the time lens angle of view) as V ₁ according to the preset lens script; the splitting effect realization device detects that the real person’s movements in the first minute are both hands The action of raising to the chest and raising both hands to the chest is not in the action information database, that is, there is no lens angle of view corresponding to the action (referred to as the action lens angle of view), then the split-lens effect realization device displays as shown in Figure 7A The virtual image shown, of which the virtual image and the real image shown in Figure 7A

The lens angle of view is the same.

（如第7B圖左上角所示），然後根據真實圖像

得到三維虛擬模型

。分鏡效果實現裝置對背景音樂進行節拍檢測，確定第2分鐘對應的節拍B₂ ，並根據節拍B₂ 得到第2分鐘時的舞臺特效

，然後在三維虛擬模型

中添加舞臺特效

；分鏡效果實現裝置根據預設的鏡頭腳本確定第2分鐘對應的鏡頭視角（簡稱為時間鏡頭視角）為V₂ ；分鏡效果實現裝置檢測到真實人物在第2分鐘的動作是向上抬起雙手，並且向上抬起雙手這個動作不在動作資訊庫中，即不存在動作對應的鏡頭視角（簡稱為動作鏡頭視角），則此時分鏡效果實現裝置將三維虛擬模型

向左上方旋轉得到鏡頭視角為V₂ 對應的虛擬圖像。可以看出，當在三維虛擬模型

中添加舞臺特效

時，第7B圖示出的虛擬圖像比第7A圖示出的虛擬圖像中增添了燈光效果。As shown in Fig. 7B, in the second minute, the split-lens effect realization device shoots a real person under the _{lens angle of view V 1 to obtain a real image}

(As shown in the upper left corner of Figure 7B), and then according to the real image

Get 3D virtual model

. The split-mirror effect realization device detects the beat of the background music, determines the beat B ₂ corresponding to the second minute, and obtains the stage special effects at the second minute according to the beat B ₂

And then in the 3D virtual model

Add stage effects

; The split-lens effect realization device determines the lens angle corresponding to the second minute (referred to as the time-lens angle) as V ₂ according to the preset lens script; the split-lens effect realization device detects that the real person’s action in the second minute is lifted up The action of raising both hands and raising both hands is not in the action information database, that is, there is no lens angle of view corresponding to the action (referred to as the action lens angle of view), then the split-lens effect realization device will use the three-dimensional virtual model

Rotate to the upper left to obtain a virtual image corresponding to the _{lens angle of view V 2.} It can be seen that when the 3D virtual model

Add stage effects

At this time, the virtual image shown in Figure 7B adds lighting effects to the virtual image shown in Figure 7A.

（如第7C圖左上角所示），然後根據真實圖像

得到三維虛擬模型

。分鏡效果實現裝置對背景音樂進行節拍檢測，確定第3分鐘對應的節拍B₃ ，並根據節拍B₃ 得到第3分鐘時的舞臺特效

，然後在三維虛擬模型

中添加舞臺特效

；分鏡效果實現裝置根據預設的鏡頭腳本確定第3分鐘對應的鏡頭視角（簡稱為時間鏡頭視角）為V₂ ；分鏡效果實現裝置檢測到真實人物在第3分鐘的動作是向上抬起左腳，並且抬起左腳這個動作對應的鏡頭視角（簡稱為動作鏡頭視角）為V₃ ，則此時分鏡效果實現裝置將三維虛擬模型

向左旋轉得到鏡頭視角為V₃ 對應的虛擬圖像。可以看出，當在三維虛擬模型

中添加舞臺特效

時，第7C圖示出的虛擬圖像與第7B圖示出的虛擬圖像中的燈光效果不同，並且第7C圖示出的虛擬圖像中呈現有音效波浪效果。As shown in Figure 7C, in the 3rd minute, the split-lens effect realization device takes a picture of a real person under the _{lens angle of view V 1 to obtain a real image}

(As shown in the upper left corner of Figure 7C), and then according to the real image

Get 3D virtual model

. The split-mirror effect realization device detects the beat of the background music, determines the beat B ₃ corresponding to the third minute, and obtains the stage special effects at the third minute according to the beat B ₃

And then in the 3D virtual model

Add stage effects

; The split-lens effect realization device determines the lens angle corresponding to the third minute (referred to as the time-lens angle) as V ₂ according to the preset lens script; the split-lens effect realization device detects that the real person’s action in the third minute is lifted up Left foot, and the lens angle of view corresponding to the action of raising the left foot (referred to as the action lens angle of view) is V ₃ , then the device for realizing the split-lens effect will use the three-dimensional virtual model

Rotate Left camera angle is obtained V ₃ corresponding to the virtual image. It can be seen that when the 3D virtual model

Add stage effects

At this time, the virtual image shown in Figure 7C has a different lighting effect from the virtual image shown in Figure 7B, and the virtual image shown in Figure 7C has a sound wave effect.

（如第7D圖左上角所示），然後根據真實圖像

得到三維虛擬模型

。分鏡效果實現裝置對背景音樂進行節拍檢測，確定第4分鐘對應的節拍B₄ ，並根據節拍B₄ 得到第4分鐘時的舞臺特效

，然後在三維虛擬模型

中添加舞臺特效

；分鏡效果實現裝置根據預設的鏡頭腳本確定第3分鐘對應的鏡頭視角（簡稱為時間鏡頭視角）為V₄ ；分鏡效果實現裝置檢測到真實人物在第4分鐘的動作是站立，並且站立這個動作對應的鏡頭視角（簡稱為動作鏡頭視角）為V₄ ，則此時分鏡效果實現裝置將三維虛擬模型

向右旋轉得到鏡頭視角為V₄ 對應的虛擬圖像。可以看出，當在三維虛擬模型

中添加舞臺特效

時，使得第7D圖示出的虛擬圖像與第7C圖示出的虛擬圖像中舞臺效果不相同。As shown in Fig. 7D, at the 4th minute, the split-lens effect realization device shoots a real person under the _{lens angle of view V 1 to obtain a real image}

(As shown in the upper left corner of Figure 7D), and then according to the real image

Get 3D virtual model

. The segmentation effect realization device detects the beat of the background music, determines the beat B ₄ corresponding to the 4th minute, and obtains the stage special effects at the 4th minute according to the beat B ₄

And then in the 3D virtual model

Add stage effects

; The splitting effect realization device determines the lens angle corresponding to the 3rd minute (referred to as the time lens angle of view) as V ₄ according to the preset lens script; the splitting effect realization device detects that the real person’s action in the 4th minute is standing, and The angle of view of the lens corresponding to the action of standing (referred to as the angle of view of the action lens) is V ₄ , at this time, the device for realizing the split-lens effect will use the three-dimensional virtual model

Rotate to the right to obtain a virtual image corresponding to the _{lens angle of view V 4.} It can be seen that when the 3D virtual model

Add stage effects

When the virtual image shown in Figure 7D is different from the virtual image shown in Figure 7C, the stage effect is different.

The splitting effect realization device provided by the embodiments of the present application can be a software device or a hardware device. When the splitting effect realization device is a software device, the splitting effect realization device can be separately deployed on a computing device in a cloud environment , It can also be deployed separately on a terminal device. When the split-lens effect realization device is a hardware device, the unit modules inside the split-lens effect realization device can also be divided into various types, and each module can be a software module or a The hardware module may also be part of the software module and part of the hardware module, and this application does not limit it. Fig. 8 is an exemplary division method. As shown in Fig. 8, Fig. 8 is an apparatus 800 for implementing a split-mirror effect provided by an embodiment of the present application, and includes: an obtaining unit 810 configured to obtain a three-dimensional virtual model; The mirror splitting unit 820 is configured to render the three-dimensional virtual model with at least two different lens angles to obtain virtual images corresponding to at least two different lens angles.

In some optional embodiments of the present application, the three-dimensional virtual model includes a three-dimensional virtual character model in a three-dimensional virtual scene model, and the above-mentioned apparatus further includes: a feature extraction unit 830 and a three-dimensional virtual model generation unit 840; wherein,

The acquiring unit 810 is further configured to acquire a real image before acquiring the three-dimensional virtual model, where the real image includes a real person image; the feature extraction unit 830 is configured to perform feature extraction on the real person image to obtain feature information, wherein The feature information includes the action information of the real person; the 3D virtual model generating unit 840 is configured to generate a 3D virtual model according to the feature information, so that the action information of the 3D virtual person model in the 3D virtual model corresponds to the action information of the real person.

In some optional embodiments of the present application, the acquiring unit is configured to acquire a film stream, and obtain at least two frames of real images according to at least two frames of images in the film stream; the feature extraction unit 830 is configured to perform real images for each frame. Character images are extracted to obtain corresponding characteristic information.

In some optional embodiments of the present application, the real image further includes a real scene image, and the three-dimensional virtual model also includes a three-dimensional virtual scene model; the above-mentioned apparatus further includes: a three-dimensional virtual scene image construction unit 850 configured to acquire at the acquisition unit Before the three-dimensional virtual model, a three-dimensional virtual scene image is constructed according to the real scene image.

In some optional embodiments of the present application, the above-mentioned device further includes a lens angle acquisition unit 860 configured to obtain at least two different lens angles. Specifically, in some optional embodiments, the lens angle of view acquisition unit 860 is configured to obtain at least two different lens angles according to at least two frames of real images.

In some optional embodiments of the present application, the lens angle acquisition unit 860 is configured to obtain at least two different lens angles according to the action information corresponding to the at least two frames of real images.

In some optional embodiments of the present application, the lens angle acquisition unit 860 is configured to acquire background music; determine the time collection corresponding to the background music, where the time collection includes at least two time periods; and obtain the corresponding time period in the time collection Lens angle of view.

In some optional embodiments of the present application, at least two different lens angles include a first lens angle of view and a second lens angle of view, and the splitter unit 820 is configured to render the three-dimensional virtual model with the first lens angle of view to obtain the first lens angle. Virtual image; Render the three-dimensional virtual model from the second lens perspective to obtain the second virtual image; display the image sequence formed according to the first virtual image and the second virtual image.

In some optional embodiments of the present application, the splitting unit 820 is configured to translate or rotate the three-dimensional virtual model under the first lens angle of view to obtain the three-dimensional virtual model under the second lens angle of view; and obtain the three-dimensional virtual model under the second lens angle of view. The second virtual image corresponding to the three-dimensional virtual model.

In some optional embodiments of the present application, the splitting unit 820 is configured to insert a frame of virtual image between the first virtual image and the second virtual image, so that the first virtual image is gently switched to the second virtual image. Image, where a is a positive integer.

In some optional embodiments of the present application, the above-mentioned device further includes: a beat detection unit 870 configured to perform beat detection on the background music to obtain a beat collection of the background music, wherein the beat collection includes multiple beats, Each beat corresponds to a stage special effect; the stage special effect generation unit 880 is configured to add the target stage special effect corresponding to the beat collection to the three-dimensional virtual model.

The aforementioned split-mirror effect realization device generates a three-dimensional virtual model based on the collected real image, and obtains multiple lens perspectives based on the collected real image, background music, and the actions of real characters, and uses multiple lens perspectives to compare the three-dimensional virtual model. The model performs the corresponding lens angle switch, which is analogous to the effect of multiple virtual cameras shooting the 3D virtual model in the virtual scene, so that the user can see the 3D virtual model under different lens angles, which improves the audience’s viewing. Sense of experience. In addition, the device also analyzes the beats of the background music and adds corresponding stage effects to the three-dimensional virtual model based on the beat information to present different stage effects to the audience, which further enhances the audience’s live viewing experience.

Referring to FIG. 9, an embodiment of the present application provides a schematic structural diagram of an electronic device 900, and the foregoing device for implementing the split-mirror effect is applied to the electronic device 900. The electronic device 900 includes a processor 910, a communication interface 920, and a memory 930. The processor 910, the communication interface 920, and the memory 930 can be coupled through a bus 940. among them,

The processor 910 can be a central processing unit (Central Processing Unit, CPU), a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), a dedicated integrated circuit (Application-Specific Integrated Circuit, ASIC), on-site programmable design Field Programmable Gate Array (FPGA) or other programmable logic devices (Programmable Logic Device, PLD), transistor logic devices, hardware components, or any combination thereof. The processor 910 may implement or execute various exemplary methods described in conjunction with the disclosure of this application. Specifically, the processor 910 reads the program code stored in the memory 930, and cooperates with the communication interface 920 to execute part or all of the steps of the method executed by the split-mirror effect realization device in the above-mentioned embodiment of the present application.

The communication interface 920 can be a wired interface or a wireless interface for communicating with other modules or devices. The wired interface can be an Ethernet interface, a controller area network interface, a local interconnect network (Local Interconnect Network, LIN), and a FlexRay interface , The wireless interface can be a cellular network interface or a wireless LAN interface. Specifically, the aforementioned communication interface 920 may be connected to an input/output device 950, and the input/output device 950 may include other terminal devices such as a mouse, a keyboard, and a microphone.

The memory 930 may include volatile memory, such as random access memory (Random Access Memory, RAM); the memory 930 may also include non-volatile memory (Non-Volatile Memory), such as read-only memory ( Read-Only Memory, ROM), flash memory, hard disk (HDD) or solid-state drive (Solid-State Drive, SSD), the memory 930 may also include a combination of the foregoing types of memory. The memory 930 can store program codes and program data. Wherein, the program code is composed of the codes of some or all of the units in the above-mentioned mirror effect realization device 800, for example, the code of the acquisition unit 810, the code of the mirror unit 820, the code of the feature extraction unit 830, and the three-dimensional virtual model generation unit 840. The code of the 3D virtual scene image construction unit 850, the code of the lens angle acquisition unit 860, the code of the beat detection unit 870, the code of the stage special effect generation unit 880, and so on. The program data is data generated during the operation of the split-mirror effect realization device 800, such as real image data, three-dimensional virtual model data, lens angle data, background music data, virtual image data, and so on.

The bus 940 may be a Controller Area Network (CAN) or other internal bus that implements interconnection between various systems or devices in the vehicle. The bus 940 can be divided into address bus, data bus, control bus and so on. For ease of representation, the figure is only represented by a thick line, but it does not mean that there is only one busbar or one type of busbar.

It should be understood that the electronic device 900 may include more or fewer components than those shown in FIG. 9, or may have different component configurations.

An embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and the computer program is executed by hardware (such as a processor, etc.) to realize part or All steps.

The embodiment of the present application also provides a computer program product, which executes part or all of the steps of the above-mentioned method for realizing the split-mirror effect when the computer program product runs on the above-mentioned device or electronic device for realizing the split-mirror effect.

In the above-mentioned embodiments, it may be implemented in whole or in part through software, hardware, firmware, or any combination thereof. When implemented by software, it can be implemented in the form of a computer program product in whole or in part. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer may be a general-purpose computer, a dedicated computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be from a website, computer, server, or The data center transmits data to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line) or wireless (such as infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, a data center, or the like integrated with one or more available media. The usable medium may be a magnetic medium, (for example, a floppy disk, a storage disk, and a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, an SSD). In the embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in an embodiment, reference may be made to related descriptions of other embodiments.

In the several embodiments provided in this application, it should be understood that the disclosed device can also be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division, and there may be other divisions in actual implementation, for example, multiple units or elements can be combined or integrated into Another system, or some features can be ignored or not implemented. In addition, the shown or discussed indirect coupling or direct coupling or communication connection between each other may be through some interfaces, indirect coupling or communication connection between devices or units, and may be in electrical or other forms.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed to multiple network units. on. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions in the embodiments of the present application.

In addition, the functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The integrated unit can be implemented in the form of hardware or software functional unit.

If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solution of this application essentially or the part that contributes to the existing technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium may include, for example, various media capable of storing program codes, such as a U disk, a removable hard disk, a read-only memory, a random access memory, a floppy disk, or an optical disk.

The above are only optional implementation manners of the embodiments of the present application, but the protection scope of the embodiments of the present application is not limited thereto. Any person skilled in the art can easily think of within the technical scope disclosed in the present application. Various equivalent modifications or replacements shall be covered within the protection scope of this application. Therefore, the protection scope of the embodiments of this application shall be subject to the protection scope of the invention application patent.

110: Photography equipment 120: server 130: user terminal S101, S102, S103: steps S201, S202, S203, S204, S205, S206, S207, S208, S209, S210, S211, S212: steps 800: Split-lens effect realization device 810: get unit 820: Splitter unit 830: Feature Extraction Unit 840: 3D virtual model generation unit 850: 3D virtual scene image construction unit 860: Lens angle acquisition unit 870: Beat detection unit 880: stage special effects generation unit 900: electronic equipment 910: processor 920: Communication interface 930: Memory 940: Bus 950: input and output devices

In order to more clearly describe the technical solutions in the embodiments of this application or the background art, the following will briefly introduce the drawings needed in the description of the embodiments of this application. Obviously, the drawings in the following description are some of the present application. Embodiments, for those of ordinary skill in the art, without making progressive labor, other drawings can be obtained based on these drawings. Figure 1 is a schematic diagram of a specific application scenario provided by an embodiment of the present application; Figure 2 is a schematic diagram of a possible three-dimensional virtual model provided by an embodiment of the present application; FIG. 3 is a schematic flowchart of a method for realizing a split-mirror effect provided by an embodiment of the present application; Figure 4 is a schematic diagram of an interpolation curve provided by an embodiment of the present application; Figure 5 is a schematic flowchart of a specific embodiment provided by an embodiment of the present application; Figure 6 is a schematic diagram of a splitting rule provided by an embodiment of the present application; Figure 7A is an effect diagram of a possible virtual image provided by an embodiment of the present application; Figure 7B is an effect diagram of a possible virtual image provided by an embodiment of the present application; Figure 7C is an effect diagram of a possible virtual image provided by an embodiment of the present application; Figure 7D is an effect diagram of a possible virtual image provided by an embodiment of the present application; FIG. 8 is a schematic structural diagram of a device for implementing a split-mirror effect provided by an embodiment of the present application; Figure 9 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.

S101, S102, S103: steps

Claims (13)

A method for realizing the split-mirror effect includes: Obtain a three-dimensional virtual model; The three-dimensional virtual model is rendered with at least two different lens angles to obtain virtual images corresponding to at least two different lens angles. The method according to claim 1, wherein the three-dimensional virtual model includes a three-dimensional virtual character model in a three-dimensional virtual scene model, and before the obtaining the three-dimensional virtual model, the method further includes: Acquiring a real image, where the real image includes an image of a real person; Perform feature extraction on the real person image to obtain feature information, where the feature information includes action information of the real person; The three-dimensional virtual model is generated according to the characteristic information, so that the action information of the three-dimensional virtual character model in the three-dimensional virtual model corresponds to the action information of the real character. The method according to claim 2, wherein the obtaining a real image includes: Acquiring a film stream, and obtaining at least two frames of the real image according to at least two frames of images in the film stream; The performing feature extraction on the real person image to obtain feature information includes: Perform feature extraction on each frame of the real person image to obtain corresponding feature information. The method according to claim 3, wherein the real image further includes a real scene image, the three-dimensional virtual model further includes the three-dimensional virtual scene model; before the acquiring the three-dimensional virtual model, the method further include: According to the real scene image, the three-dimensional virtual scene model is constructed. The method according to claim 3 or 4, wherein acquiring the at least two different lens angles includes: According to the at least two frames of the real image, the at least two different lens angles of view are obtained. The method according to claim 3 or 4, wherein acquiring the at least two different lens angles includes: The at least two different camera angles are obtained according to the action information corresponding to the at least two frames of the real images respectively. The method according to claim 3 or 4, wherein acquiring the at least two different lens angles includes: Get background music; Determining a time collection corresponding to the background music, wherein the time collection includes at least two time periods; Obtain the lens angle of view corresponding to each time period in the time collection. The method according to claim 1, wherein the at least two different lens angles include a first lens angle of view and a second lens angle of view; the rendering of the three-dimensional virtual model with at least two different lens angles, Obtain at least two virtual images corresponding to different lens angles, including: Rendering the three-dimensional virtual model with the first lens perspective to obtain a first virtual image; Rendering the three-dimensional virtual model with the second lens perspective to obtain a second virtual image; The image sequence formed according to the first virtual image and the second virtual image is displayed. The method according to claim 8, wherein the rendering the three-dimensional virtual model with the second lens angle of view to obtain a second virtual image includes: Translate or rotate the three-dimensional virtual model in the first lens angle of view to obtain the three-dimensional virtual model in the second lens angle of view; Acquiring the second virtual image corresponding to the three-dimensional virtual model in the second lens angle of view. The method according to claim 9, wherein the displaying the image sequence formed according to the first image and the second virtual image includes: A frame of virtual image is inserted between the first virtual image and the second virtual image, so that the first virtual image is gently switched to the second virtual image, where a is a positive integer. The method according to claim 7, wherein the method further includes: Performing beat detection on the background music to obtain a beat collection of the background music, wherein the beat collection includes multiple beats, and each beat in the multiple beats corresponds to a stage special effect; The target stage special effect corresponding to the beat collection is added to the three-dimensional virtual model. An electronic device comprising: a processor, a communication interface, and a memory; the memory is used to store instructions, the processor is used to execute the instructions, and the communication interface is used to connect to the processor Communicate with other devices under control, wherein the processor implements the method described in any one of the request items 1 to 11 when the processor executes the instruction. A computer-readable storage medium stores a computer program, and the computer program is executed by hardware to implement the method described in any one of claim items 1 to 11.

Images