WO2023142650A1 - Special effect rendering - Google Patents

Abstract

The present disclosure provides a method and apparatus for special effect rendering, a device, and a storage medium. The method comprises: obtaining a target image, the target image comprising a target object; obtaining a 3D object model, the 3D object model comprising a target area; and on the basis of depth information of the 3D object model and color information of a non-target area except the target area on the 3D object model, rendering the 3D object model on the target image to obtain a rendering result image, wherein the target object is displayed in the target area.

Description

special effects rendering technical field

The present disclosure relates to the technical field of special effect rendering, and in particular, to a method, device, device and storage medium for special effect rendering.

Background technique

At present, when performing special effect rendering on an image, some special effects or rendering materials are generally rendered on the image. For example, in the process of taking pictures or shooting short videos, special effects of stickers are often added to the faces in the photos or short videos, or special effects of beautification are rendered on the faces. However, this special effect rendering method is relatively simple and lacks interest.

Contents of the invention

In order to overcome the problems existing in related technologies, the present disclosure provides a method, device, device and storage medium for special effect rendering.

According to the first aspect of an embodiment of the present disclosure, there is provided a special effect rendering method, including: acquiring a target image, the target image including a target object; acquiring a 3D object model, the 3D object model including a target area; based on the 3D Depth information of the object model and color information of non-target areas on the 3D object model except the target area, rendering the 3D object model onto the target image to obtain a rendering result image, wherein the target Objects are displayed within the target area.

In some embodiments, rendering the 3D object model on the target image based on the depth information of the 3D object model and the color information of the non-target area on the 3D object model includes: converting the depth of the target area assigning information to the pixel points corresponding to the target area on the target image, and assigning the depth information and color information of the non-target area in the 3D object model to the pixel points corresponding to the non-target area in the 3D object model on the target image .

Based on this, by assigning the depth information of the target area on the 3D object model, the depth information and the color information of the non-target area on the 3D object model to each corresponding pixel on the target image, the 3D object model can be rendered on the target image, and The 3D object model does not obscure the target object. In the rendering result image, the part of the target image and the 3D object model are fused into one image, realizing the combination of virtual and reality.

In some embodiments, the 3D object model is located in the pre-acquired 3D scene model, and the method further includes: based on the depth information and color information of the non-coverage area in the 3D scene model The scene model is rendered onto the target image to obtain the rendering result image, wherein the coverage area is an area covered by the target area in the 3D object model under the perspective of the rendering camera. Based on this, the rendering result image can include both the 3D object model and the 3D scene model, making the rendering result more diverse and more interesting.

In some embodiments, the 3D object model includes a plurality of sub-object models, and at least one of the sub-object models moves along a corresponding motion path; and/or, the 3D scene model includes a plurality of sub-scene models, and at least one of the sub-object models The sub-scene model moves along the corresponding movement path.

Based on this, in the rendered image, at least one sub-scene model included in the 3D scene model can move on a corresponding path; and/or at least one sub-object model included in the 3D object model can move on a corresponding path. A sub-object model and/or a sub-scene model of dynamic motion is included in the rendered rendering result.

In some embodiments, the method further includes: acquiring the display range of the rendering result image, and rendering the sub-object model that moves beyond the display range at the start of the motion path corresponding to the sub-object model Location.

Re-rendering the sub-object model that has moved out of the display range at the starting position of the motion path corresponding to the sub-object model and maintaining the motion can make the motion of the sub-object model manifest as a circular motion.

In some embodiments, the method further includes: acquiring the display range of the rendering result image, and rendering the sub-scene model whose motion exceeds the display range at the start of the motion path corresponding to the sub-scene model Location.

The sub-scene model that has moved out of the display range is re-rendered at the start position of the motion path corresponding to the sub-scene model and maintained in motion, so that the motion of the target sub-scene model can be embodied as a circular motion.

In some embodiments, before rendering the 3D object model and/or the 3D scene model, the method further includes: acquiring the orientation of the target object; adjusting the 3D object model and/or based on the orientation of the target object Or the orientation of the 3D scene model in the rendering result image.

The orientation of the 3D object model and/or the 3D scene model may be adjusted according to the orientation of the target object, so that the orientation of the 3D object model and/or the 3D scene model in the rendering result may be different based on the orientation of the target object.

In some embodiments, before rendering the 3D object model and/or 3D scene model, the method further includes: acquiring the orientation of the target object; adjusting the angle parameter of the rendering camera based on the orientation of the target object; wherein , the 3D object model and/or the 3D scene model display different orientations on the rendering result image under different angle parameters.

The orientation of the 3D object model and/or the 3D scene model in the rendering result can be changed by changing the angle parameter of the rendering camera, and the angle parameter of the rendering camera can be changed according to the orientation of the target object, so that the 3D object model and/or The orientation of the 3D scene model changes as the angle parameter of the rendering camera changes.

In some embodiments, when the target image includes multiple target objects, the acquiring 3D object models includes: acquiring multiple 3D object models, the 3D object models and the target objects are respectively Correspondingly, each 3D object model includes a target area.

Based on this, when there are multiple target objects in the target image, multiple 3D object models can be obtained, so that the rendered result can include multiple target objects and multiple 3D object models, wherein the multiple 3D object models are related to the multiple The corresponding target objects are displayed on the rendering result image.

Further, acquiring a 3D object model includes: recognizing the target image to obtain a recognition result; and determining a 3D object model corresponding to each of the target objects from a plurality of 3D object models based on the recognition result.

The target image can be identified, and the 3D object model corresponding to the target object is determined according to the recognition result of the target image, thereby determining the 3D object model corresponding to each target object, so that the target object in the rendering result is combined with the above-mentioned corresponding 3D object model show.

Further, in some embodiments, the recognition result includes attribute information of the target object, and the 3D object model matches the corresponding attribute information of the target object.

Based on this, the 3D object model matching the attribute information can be determined according to the attribute information of the target object, so that the 3D object model in the rendering result can be displayed correspondingly to the target object with the attribute information matching the 3D object model.

In some embodiments, the attribute information of the target object may also be obtained; according to the attribute information of the target object, the attribute information of the 3D scene model is determined; the depth of the non-covered area based on the 3D scene model Information and color information, rendering the 3D scene model on the target image to obtain a rendering result image, including: based on the depth information and color information of the non-covered area in the 3D scene model and the attribute information of the 3D scene model, The 3D scene model is rendered.

The attribute information of the 3D scene model can be determined according to the attribute information of the target object, and the 3D scene model with different attribute information can be rendered for the target object with different attribute information in the target image.

According to a second aspect of an embodiment of the present disclosure, there is provided a special effect rendering device, including: a first acquisition module, configured to acquire a target image, the target image including a target object; a second acquisition module, configured to acquire a 3D object model , the 3D object model includes a target area; a rendering module, configured to render the 3D object based on the depth information of the 3D object model and the color information of non-target areas other than the target area on the 3D object model A model is rendered onto the target image to obtain a rendering result image, wherein the target object is displayed in the target area.

According to a third aspect of the embodiments of the present disclosure, there is provided a special effect rendering device, including a processor and a memory for storing computer program instructions executable by the processor. Wherein, the processor is configured to implement the method described in any of the above embodiments by executing the computer program instructions.

According to a fourth aspect of the embodiments of the present disclosure, there is provided a computer-readable medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps in the method described in any of the above-mentioned embodiments are implemented.

In the technical solution provided by the embodiments of the present disclosure, a target image including a target object is obtained, and a virtual 3D object model on the target image is obtained. Since the rendered image includes both the target object and the 3D object model, and the 3D object model cannot block the target object, the 3D object model includes a target area for displaying the target object. Furthermore, based on the depth information of the 3D object model and the color information of the non-target area except the target area on the 3D object model, the 3D object model is rendered onto the target image to obtain a rendering result image. Render the 3D object model on the target image in the above way, so that the real target object and the virtual 3D object model can be integrated in the rendering result image, improve the fun and immersion, and make the user feel more immersive .

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure.

Description of drawings

Fig. 1 is a flowchart of a method for rendering special effects provided by an exemplary embodiment.

Fig. 2 is an effect diagram of an acquired target image provided by an exemplary embodiment.

Fig. 3 is an effect diagram of an acquired partial 3D object model and partial 3D scene model provided by an exemplary embodiment.

Fig. 4 is an effect diagram of a rendering result of rendering a 3D object model and a 3D scene model on a target image provided by an exemplary embodiment.

Fig. 5 is an effect diagram of dividing a 3D scene model into multiple sub-scene models and the sub-scene models are in a moving state provided by an exemplary embodiment.

Fig. 6 is an effect diagram of a target sub-scene model moving in a rendering result provided by an exemplary embodiment.

Fig. 7 is an effect diagram of different orientations of a target object provided by an exemplary embodiment.

Fig. 8 is an effect diagram of adjusting a 3D object model and a 3D scene model according to an orientation of a target object provided by an exemplary embodiment.

Fig. 9 is an effect diagram of adjusting a rendering camera angle of view according to an orientation of a target object provided by an exemplary embodiment.

Fig. 10 is a flowchart of a special effect rendering method provided by an exemplary embodiment.

Fig. 11 is a schematic structural diagram of a special effect rendering device provided by an exemplary embodiment.

Fig. 12 is a schematic structural diagram of a special effect rendering device provided by an exemplary embodiment.

Detailed ways

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatuses and methods consistent with aspects of the present disclosure as recited in the appended claims.

The terminology used in the present disclosure is for the purpose of describing particular embodiments only, and is not intended to limit the present disclosure. As used in this disclosure and the appended claims, the singular forms "a", "the", and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It should also be understood that the term "and/or" as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

It should be understood that although the terms first, second, third, etc. may be used in the present disclosure to describe various information, the information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of the present disclosure, first information may also be called second information, and similarly, second information may also be called first information. Depending on the context, the word "if" as used herein may be interpreted as "at" or "when" or "in response to a determination."

At present, when performing special effects rendering on an image, some special effects or rendering materials are generally rendered to target objects in the image. Taking the target object as a human face as an example, you can render special beautification effects on the human face; add decorations such as hats, earrings and other sticker materials to people; enlarge, reduce, and distort the human face. The above-mentioned special effect rendering method is relatively simple and lacks interest.

In view of the above problems, an embodiment of the present disclosure provides a special effect rendering method, which is suitable for processing images or videos captured in real time, and processing images or videos captured and stored in advance. For example, for the scene of shooting images or videos in real time, when shooting, the target object (such as the user's face or body and other parts) can be displayed in a pre-made virtual 3D (three-dimensional) model, thereby improving the user experience. It enhances the sense of integration between the target object and the 3D model, making users feel more immersive. Of course, the above application scenarios are only used for illustrative illustration, and should not be understood as limiting the embodiments of the present disclosure.

Referring to FIG. 1, FIG. 1 is a flow chart of special effect rendering provided by an embodiment of the present disclosure, including the following steps:

S102. Acquire a target image, where the target image includes a target object;

S104. Acquire a 3D object model, where the 3D object model includes a target area;

S106. Render the 3D object model to the on the target image to obtain a rendering result image, wherein the target object is displayed in the target area.

In S102, the target image may be acquired during real-time shooting by the camera, or the target image may be acquired in a pre-shot video or image. Since the captured or stored video frames may or may not include the target object, it may first be determined whether each captured or stored image includes the target object, and the image including the target object is determined as the target image. Whether the target object is included in the image can be determined by performing object detection on the image, or whether the target object is included in the image can be determined based on pre-labeled information. For images that do not include the target object, special effect rendering processing is not performed. The target object can be a body part such as a human face, a human hand or a human foot, or a living body such as a person, an animal, or a plant, or a non-living body such as a building. Specifically, the type of the target object may be changed according to the requirements of the actual scene, and the embodiment of the present disclosure does not limit the type of the target object.

In S104, a prefabricated virtual 3D object model may be obtained, and the 3D object model includes the target area. In the embodiment of the present disclosure, the 3D object model can be an animal, such as a 3D model of a piglet, a little monkey, etc., or a 3D model of a mirror, a painting, etc., depending on the effect required by the actual scene Get pre-made models of different 3D objects. The target area may be a partial area in the 3D object model. For example, a target area included in a 3D object model may be a body part in the 3D object model. For example, when the 3D object model is a piggy model, the target area may be the face of the piggy model, the hands of the piggy model, or the stomach of the piggy model, and so on. The target area included in the 3D object model may also be an area within a specified range in the 3D object model. For example, when the 3D object model is a model of a painting, the target area may be a rectangular area in the middle of the painting, or a range area covered by a certain person or object in the painting. Of course, specifically, the target area in the 3D object model can be set according to the effect required by the actual scene. The embodiment of the present disclosure does not limit the 3D object model and the target area in the 3D object model.

In S106, based on the depth information of the 3D object model and the color information of the non-target area on the 3D object model, the 3D object model can be rendered onto the target image to obtain a rendering result image. In some embodiments, it may be that the depth information of the target area in the 3D object model is assigned to the pixel points corresponding to the target area on the target image, and the depth information and color information of the non-target area in the 3D object model are assigned to the target Pixels on the image that correspond to non-target areas in the 3D object model. Through the above method, the 3D object model can be rendered on the target image, and in the rendered image, since the pixels corresponding to the target area on the target image are only endowed with the depth information of the 3D object model, the rendered 3D object model does not will obscure the target object in the target image. Therefore, the final rendered effect is to integrate the real-world target object into the virtual 3D object model.

In some embodiments, when the 3D object model is rendered on the target image, each pixel point on the target image corresponds to a point in the 3D object model, and for the pixel point set P1 of the non-target area in the 3D object model, Determine the depth information and color information of each point in the pixel point set P1, and then assign the depth information and color information of each point in the pixel point set P1 to the corresponding pixel point on the target image; for the target area in the 3D object model The pixel point set P2 of the pixel point set P2, determine the depth information of each point in the pixel point point P2, and assign the depth information of each point in the pixel point set P2 to the corresponding pixel point on the target image (which may be the pixel point forming the target object). Based on the above method, the 3D object model is rendered on the target image. Of course, in the actual implementation process, it is also possible to perform a certain transformation on the depth information and/or color information of each point in the determined pixel point set P1, and transform the depth information and/or color information of each point in the pixel point set P1 Or the color information is given to the corresponding pixel on the target image; the depth information of the points in the pixel set P2 can also be transformed as above. The embodiments of the present disclosure do not limit this.

In some cases, in order to further improve the richness of special effect rendering results, in addition to obtaining the 3D object model, a pre-fabricated virtual 3D scene model may also be obtained, and the 3D object model is located in the 3D scene model. Wherein, the 3D object model is used for rendering the area around the target object, so that the target object is finally displayed in the target area in the 3D object model; region (hereinafter also referred to as the background region for short) for rendering.

Exemplarily, the manner of rendering the 3D scene model may be: rendering the 3D scene model on the target image based on the depth information and color information of the area other than the covered area (hereinafter also referred to as the non-covered area) in the 3D scene model A rendering result image is obtained, wherein the coverage area is the area covered by the target area in the 3D object model under the perspective of the rendering camera.

When rendering the 3D scene model and/or the 3D object model, there is a virtual rendering camera capable of capturing a part of the 3D object model and/or the 3D scene model.

Since the 3D scene model may overlap with the target area of the 3D object model when rendering the 3D scene model, the coincidence will cause the 3D scene model to block the target object originally intended to be displayed in the target area. Therefore, in order to prevent the target object from being occluded by the 3D scene model, when rendering the 3D scene model, only the depth information and color information of the non-covered area in the 3D scene model can be rendered, and then in the rendering result image, each pixel in the target area and The color information of the corresponding pixel in the 3D scene model is not assigned, so the target area in the rendered image still displays the target object.

Under the perspective of the rendering camera, the target area in the 3D object model will cover part of the area in the 3D scene model. In one embodiment, before rendering the 3D scene model, the coverage area in the 3D scene model that is covered by the target area in the 3D object model can be eliminated, and then the 3D scene is rendered based on the depth information and color information of the non-coverage area in the 3D scene model Model. In another embodiment, before rendering the 3D scene model, the coverage area in the 3D scene model is not removed, but the 3D scene model is rendered based on the depth information of the 3D scene model and the color information of the non-coverage area in the 3D scene model , that is, the coverage area in the 3D scene model only renders depth information, and the non-coverage area in the 3D scene model renders both depth information and color information. Through any of the above methods, the final displayed rendering effect can be: the target object is displayed in the 3D object model (specifically, the target area), and the 3D object model is displayed in the 3D scene model (specifically, the coverage area). Wherein, the position of the 3D object model in the 3D scene model can be set according to actual requirements, which is not limited in this embodiment of the present disclosure.

In some embodiments, when the 3D scene model is rendered on the target image, each pixel on the target image corresponds to a point in the 3D scene model. For the pixel point set Q1 in the non-covered area in the 3D scene model, determine the depth information and color information of each point in the pixel point set Q1, and then assign the depth information and color information of each point in the pixel point set Q1 to the target image The corresponding pixels on the above; for the pixel point set Q2 of the coverage area in the 3D scene model, determine the depth information of each point in the pixel point set Q2, and assign the depth information of each point in the pixel point set Q2 to the target image corresponding pixels. Based on the above method, the 3D scene model is rendered on the target image. Of course, in the actual implementation process, the depth information and/or color information of each point in the determined pixel point set Q1 may also be transformed to a certain extent, and the transformed depth information and/or color information of each point in the pixel point set Q1 Or color information is assigned to the corresponding pixel on the target image; the depth information of the points in the pixel set Q2 can also be transformed as above. The embodiments of the present disclosure do not limit this.

In a specific embodiment, FIG. 2 is an acquired target image, where the target object is a girl's face. Fig. 3 is a part of the acquired 3D object model and a part of the 3D scene model, wherein, the 3D object model 301 is a model of a small animal with a human face model in the figure; the target area 302 in the 3D object model is the human face model in the figure area; the 3D scene model 303 includes all other models except the 3D object model 301 . 3D object model 301 and 3D scene model 303 in Fig. 3 are rendered to the effect diagram of the rendering result image of the target image in Fig. 2 as shown in Fig. 4, and the target object (the face of the girl in Fig. 2) is displayed in the rendering result image, Other areas in the original target image are covered by the special effects of the 3D object model and the 3D scene model. It can be seen that through the above-mentioned special effect rendering method, the target object can be integrated into the pre-made 3D object model and 3D scene model, so that the user has a more immersive feeling, and a stronger sense of fun and immersion. In another embodiment, the 3D object model contained in FIG. 3 can also be obtained, and the effect of rendering the 3D object model into the target image in FIG. 2 is: the target object is displayed in the pre-made 3D object model . In this regard, the present disclosure will not repeat them any further.

After the target object is integrated into the 3D object model and/or the 3D scene model, in order to further enhance the interest of the 3D model, in some embodiments of the present disclosure, the motion animation of the 3D object model and/or the 3D scene can also be preset Motion animation of the model. During 3D modeling, the 3D object model and the 3D scene model can be divided into blocks according to actual needs, for example, the 3D object model can be divided into multiple sub-object models, and each sub-object model can be a part of the 3D object model; and/ Or divide the 3D scene model into multiple sub-scene models, and each sub-scene model may be a partial area on the 3D scene model. And set motion animation on one or more sub-object models and/or one or more sub-scene models on their respective motion paths, after rendering the models, each sub-object model and/or sub-scene model moves along their corresponding motion paths , the effect of integrating the target object into the moving 3D object model and/or 3D scene model can be achieved. Of course, according to actual application requirements, only the motion animation of the 3D object model can be set without setting the motion animation of the 3D scene model; or only the motion animation of the 3D scene model can be set without the motion animation of the 3D object model. It is not limited here.

Wherein, the movement path of the target sub-object model includes an initial position, and the initial position on the movement path of the target sub-object model can also be set according to actual requirements. When the target sub-object model moves beyond the display range of the rendered result image, the target sub-object model whose motion exceeds the display range of the rendered result image can be re-rendered at the starting position of the target sub-object model's motion path to continue to maintain the target sub-object model exercise.

There can also be multiple starting positions on the motion path of the target sub-object model. At this time, if the target sub-object model moves beyond the display range of the rendering result image, when re-rendering the position of the target sub-object model, multiple starting positions can be randomly selected. Any one of the starting positions, or one of the multiple starting positions may be selected according to preset rules. Moreover, when the target sub-object model is moving, the moving speed and/or moving path can be set according to actual needs, and when the target sub-object model is moving, the moving speed and/or moving path of the target sub-object model can also be changed according to preset rules. or motion path; moreover, the starting position on the motion path of the target sub-object model can also be changed along with the change of the motion path of the target sub-object model. Based on this, when the target sub-object model is displayed on the rendering result image, it can perform a certain circular motion to improve the interest.

The motion path of the target sub-scene model includes a starting position, and the starting position on the motion path of the target sub-scene model can also be set according to actual requirements. When the movement of the target sub-scene model exceeds the display range of the rendering result image, the target sub-scene model whose movement exceeds the display range of the rendering result image can be re-rendered at the starting position of the movement path of the target sub-scene model to maintain the target sub-scene model sports.

There can also be multiple starting positions on the motion path of the target sub-scene model. At this time, if the target sub-scene model moves beyond the display range of the rendering result image and the position of the target sub-scene model needs to be re-rendered, multiple starting positions can be randomly selected. Any one of the starting positions, or one of the multiple starting positions may be selected according to preset rules. Moreover, when the target sub-scene model is in motion, the motion speed and/or motion path can be set according to actual needs, and when the target sub-scene model is in motion, the motion speed and/or motion path of the target sub-scene model can also be changed according to preset rules. or the motion path; moreover, the starting position on the motion path of the target sub-scene model may also change as the motion path of the target sub-scene model changes. Based on this, when the target sub-scene model is displayed on the rendering result image, it can perform a certain circular motion to improve the interest. The embodiments of the present disclosure are only examples of several implementations and combinations of implementations, and the embodiments of the disclosure are not intended to be limiting.

The target sub-object model and/or the target sub-scene model exceeds the display range of the rendering result image during the movement. It may be that some blocks of the target sub-object model and/or the target sub-scene model exceed the display range of the rendering result image, and the specific setting method depends on actual requirements, which is not limited in this disclosure. In a specific embodiment, referring to FIG. 5, as shown in (5-1) in FIG. 5, the ground scene is divided into 5 sub-scene models in advance, and each block from left to right is respectively marked as block 501 and block 502 , block 502, block 504, and block 505, the state at this time is the initial state of the rendering result image after rendering the ground scene after the target image, let block 501 in the ground scene be the target sub-scene model, and the target sub-scene model 501 The current position is the starting position of the target sub-scene model 501, and the movement direction of the target sub-scene model 501 is from left to right, and the movement path of the target sub-scene model 501 is the entire displayed ground scene. Wherein, the display range of the rendering result image is the range displayed by the five sub-scene models in the ground scene. After a period of movement, the ground scene is shown as (5-2) in FIG. 5 , where the position of the target sub-scene model 501 is as shown in the figure. After a period of movement, the position of the target sub-scene model 501 is shown in (5-3) in FIG. The display range of the rendering result image; at this time, for the target sub-scene model 501 whose movement exceeds the display range of the rendering result image, quickly move the target sub-scene model 501 back to the initial position on the motion path of the target sub-scene model 501, quickly The process of moving is as shown in (5-3) in Figure 5 to (5-4) in Figure 5 to (5-1) in Figure 5 (the position of the target sub-scene model 501 is shown in (5-3) in Figure 5, (5-4) in Fig. 5 and (5-1) shown in Fig. 5), finally re-render the target sub-scene model 501 at the starting position on the motion path of the target sub-scene model 501, as shown in Fig. 5 In (5-1), where the target sub-scene model 501 is located, the effect achieved is that the entire scene is in a state of animation in a certain cyclic motion. Of course, the above-mentioned block division method is not limited to the above-mentioned sub-scene model block method. For example, according to the actual situation and actual needs, any one or more parameters such as the shape, size, quantity, and movement mode of the blocks can be compared with those in the figure. The situation shown in is different.

In an embodiment based on rendering a 3D object model and a 3D scene model, the 3D object model includes a plurality of sub-object models, the 3D scene model includes a plurality of sub-scene models, and a certain number of sub-object models and/or sub-scene models are along their respective Motion path for motion. Referring to FIG. 6, FIG. 6 is a three-frame image in a video in which a 3D object model and a 3D scene model are rendered. In the embodiment of the present disclosure, the target image may be a target image frame, which is not limited thereto. As shown in Figure 6 (6-1), there is a sun and a cloud above the 3D scene model. At this time, the sun and the cloud are generally set as the target sub-scene model 601, and the target sub-scene model 601 has been moving for a period of time The post-display position is shown as (6-2) in Figure 6. After the target sub-scene model 601 moves for a period of time, the displayed position is as shown in (6-3) in Figure 6. At this time, the entire target sub-scene model is about to move beyond the display range of the rendering result image; when the entire target sub-scene model When 601 moves beyond the display range of the rendering result image, quickly move the target sub-scene model 601 back to the initial position in (6-1) in FIG. 6 , and maintain the movement of the target sub-scene model 601, so that the displayed scene is The animation effect of infinite loop has high interest. Certainly, the present disclosure here is only for illustration, not for limitation.

In the acquired target image, there are multiple acquired images with different orientations of the target object. The orientation of the target object can be the direction the target object is facing. For example, when the target object is a human face, the orientation of the human face is the orientation of the target object; when the target object is a human hand, the orientation of the target object can be the orientation of the palm, or It can be the direction of the finger, etc., and can be preset according to the specific scene. In a specific embodiment of the present disclosure, as shown in FIG. 2 , ( 7 - 1 ) in FIG. 7 , and ( 7 - 2 ) in FIG. 7 , the orientations of the girls' faces in these three pictures are different. In order to make the rendered effect more realistic and interesting, before rendering the 3D object model and 3D scene model, the orientation of the target object can be obtained first, and the orientation of the 3D object model and/or 3D scene model can be adjusted based on the orientation of the target object . Specifically, the orientation of the target object may be determined first, and the orientation of the 3D object model and/or the 3D scene model may be determined according to the orientation of the target object, for example, the orientation of the 3D object model may be adjusted so that the orientation of the target object in the rendering result is consistent with the orientation of the 3D object model. in the same direction. For example, if the orientation of the target object is to the left, the 3D object model is adjusted so that the orientation of the 3D object model in the rendering result is also to the left.

Among them, the way to obtain the orientation of the target object can be to extract the features of the target image, and calculate the orientation of the target object based on the algorithm of orientation recognition; it can also be to pre-train the neural network used to determine the orientation of the target object, and use the neural network to determine the target object. The orientation of the object is not limited by this embodiment of the present disclosure.

In the embodiment of the present disclosure, as shown in Fig. 4, (8-1) in Fig. 8, and (8-2) in Fig. 8, the above three pictures (Fig. 2, (7-1) in Fig. 7) are respectively rendered , (7-2) in Fig. 7). Before rendering the 3D object model and the 3D scene model, adjust the orientation of the 3D object model and the 3D scene model according to the orientation of the girl's face (target object), so that the orientation of the girl's face is consistent with the orientation of the 3D object model and the 3D scene model Matching to make the final rendered effect more realistic and interesting. In one embodiment, the transform matrix of the 3D object model and the 3D scene model ( transformation matrix), and then adjust the orientation of the 3D object model and the 3D scene model. Of course, based on actual needs, only the orientation of the 3D object model can be adjusted without adjusting the orientation of the 3D scene model based on the orientation of the target object; or only the orientation of the 3D scene model can be adjusted without adjusting the orientation of the 3D object model. It is not limited here. In this embodiment, the target object is a girl's face. In an actual scene, the target object can also be a human hand, and the orientation of the target object is the orientation of the finger; the target object can also be a car, and the target object's The orientation refers to the orientation of the front of the vehicle, etc., which is not limited in the present disclosure. The angle parameters for adjusting the orientation of the 3D object model can be determined according to actual needs. It can be adjusted so that the orientation of the 3D object model is the same as the orientation of the target object, or it can be adjusted so that the orientation of the 3D object model is always perpendicular to the orientation of the target object, or adjusted to The orientation of the 3D object model is at any angle to the orientation of the target object, etc. When adjusting the orientation of the 3D scene model, it can be adjusted so that the orientation of the 3D scene model is the same as that of the target object, or it can be adjusted so that the orientation of the 3D scene model is always perpendicular to the orientation of the target object, or it can be adjusted so that the orientation of the 3D scene model is the same as that of the target The orientation of the object is at any angle, etc., which are not limited in this embodiment of the present disclosure.

In an example of adjusting the 3D object model based on the orientation of the target object, see (9-1) in Figure 9, the arrow in the figure is the perspective of the rendering camera in the figure, and the leopard model on the right is the 3D object model, assuming the target object at this time The orientation of the object is right, and the orientation of the leopard model is also to the right; when the orientation of the target object is to the left, the rendered effect is shown in Figure 9 (9-2), and the orientation of the leopard model is also adjusted at this time is facing left. In this embodiment, the viewing angle of the rendering camera does not change, and this embodiment is only used as an example rather than a limitation.

Through the above method, the orientation of the 3D object model and/or the 3D scene model is adjusted based on the orientation of the target object, thereby diversifying the display modes of the 3D object model and/or the 3D scene model in the rendering result and improving the interest.

Before rendering the 3D object model and the 3D scene model, in addition to adjusting the orientation of the 3D object model and the 3D scene model based on the orientation of the target object, an angle parameter of the rendering camera may also be adjusted based on the orientation of the target object. Wherein, the angle parameter of the rendering camera is the angle of the rendering camera relative to the 3D object model and/or the 3D scene model when the rendering camera renders the 3D object model and/or the 3D scene model. Specifically, the orientation of the target object may be determined first, and the rendering angle of the rendering camera is determined according to the orientation of the target object, for example, the rendering angle of the rendering camera is adjusted so that the orientation of the target object in the rendering result is consistent with the orientation of the 3D object model. For example, if the orientation of the target object is to the left, the rendering angle of the rendering camera is adjusted so that the orientation of the 3D object model in the rendering result is also to the left.

In a specific embodiment, the faces of girls in Fig. 2, (7-1) in Fig. 7, and (7-2) in Fig. 7 have different orientations, and the 3D object model and the 3D Instead, adjust the angle parameters of the rendering camera instead of the orientation of the scene model, and the final rendered effect is shown in Figure 4, (8-1) in Figure 8, and (8-2) in Figure 8. It can be seen that adjusting the angle parameters of the rendering camera according to the orientation of the girl's face can also make the rendering effect more realistic and interesting. Adjusting the angle parameters of the rendering camera can be determined according to actual needs. It can be adjusted so that the angle of the rendering camera is the same as or opposite to the direction of the target object; it can also be adjusted so that the angle of the rendering camera and the direction of the target object are at any angle. The embodiments are not limited here.

In an example of adjusting the angle parameters of the rendering camera based on the orientation of the target object, see (9-1) in Figure 9, the arrow in the figure is the angle of view of the rendering camera in the figure, and the leopard model on the right is the 3D object model, assuming that at this time The orientation of the target object is to the right, at this time the viewing angle of the rendering camera is oriented to the right; when the orientation of the target object is to the left, adjust the angle parameters of the rendering camera, as shown in (9-3) in Figure 9, at this time the rendering The view angle of the camera is facing left. In this embodiment, the orientation of the 3D object model does not change. This embodiment is only used as an illustration, not as a limitation.

In the above manner, adjusting the angle parameter of the rendering camera based on the orientation of the target object can also diversify the display modes of the 3D object model and/or the 3D scene model in the rendering result and improve the interest.

Of course, based on the requirements in practical applications, the orientation of the 3D object model and/or the 3D scene model and the angle parameter of the rendering camera can also be adjusted simultaneously based on the orientation of the target object. The specific adjustment manner is not limited in the embodiments of the present disclosure.

In the actual shooting process or in the selected image, there are situations where multiple target objects appear in the target image at the same time. In the case that the target image includes multiple target objects, the method for rendering the 3D object model further includes: acquiring multiple 3D object models, the 3D object models correspond to the target objects one by one, wherein each 3D object model includes the target area. It may be that when multiple target objects are detected in the target image in advance, 3D object models including target areas equal to the number of target objects are obtained, and finally each target object is displayed in the target area of a 3D object model . Enriched the rendering method when there are multiple target objects in the target image to improve the fun.

Based on this method, multiple 3D object models are obtained, and each target object is displayed in a 3D object model corresponding to the target object, and the final rendered effect is as follows: the rendering result image includes multiple 3D object models, Each 3D object model includes a target object. According to requirements of actual application scenarios, the 3D object models corresponding to different target objects may be the same 3D object model, or may be different 3D object models. For example, when the target image includes 3 target objects, in one case, obtain 3 identical piggy models (3D object models), and each target object corresponds to a piggy model, and then perform special effect rendering; In another case, obtain 3 different models, such as a pig model, a puppy model and a lamb model, and each target object corresponds to one of the 3D object models, and the specific corresponding method is not described here. Do limited. In another case, the three acquired 3D object models are two piglet models and one lamb model, and each target object corresponds to one of the 3D object models, and the specific corresponding manner is not limited here.

In some embodiments, the acquired target image can be firstly identified to determine whether the target image contains multiple target objects; if so, further identification is performed, and each target object is determined from multiple 3D object models according to the identification result The corresponding 3D object models are displayed in one-to-one matching. It may be to identify the features of the target image, determine the 3D object model corresponding to the features of the target image and match and display the target image.

In some specific embodiments, in the scene where multiple people take selfies, the target object is a human face at this time, if two human faces appear in the target image at the same time, two 3D object models can be obtained, wherein the two human faces and the two The 3D object models are in one-to-one correspondence, and both 3D object models include the target area. The final rendering effect is: there are two 3D object models in the rendering result image, and the target areas in the two 3D object models respectively display the above two human faces. Through the above method, it is possible to integrate multiple target objects into multiple 3D object models, so that multiple target objects in the target image can simultaneously experience the interest and immersion of the special effect rendering in the embodiment of the present disclosure.

In some embodiments, the target object is a human face, and the 3D object model includes a virtual human face model. It is also possible to perform face recognition on the human face in the target image to obtain a face recognition result, and determine the face recognition result based on the face recognition result. The 3D object model corresponding to the human face. In a specific embodiment, when the target image includes two human faces, two different 3D object models are obtained, such as a piglet model and a lamb model; the features of the two human faces can be obtained Information, such as the eye information, nose information, mouth information, etc. of the two faces; face recognition is performed based on the feature information of the faces, and then the identity corresponding to each face is determined, and the identity of each face is determined based on the identity of each face. The corresponding 3D object model of the face. For example, the two faces are face A and face B, face recognition is performed based on the feature information of the faces, and the identities of face A and face B are determined; and the 3D object model corresponding to face A is determined to be a pig model, and determine the 3D object model corresponding to face B as a lamb model. Based on the above method, each different human face can be made to correspond to a different 3D object model, so as to enhance the interest and authenticity of special effect rendering. For example, when user A and user B are taking pictures, the terminal device determines through face recognition that user A's face corresponds to the piggy model, and user B's face corresponds to the lamb model; The range that the camera can capture, that is, the face of user A is not displayed in the target image for a period of time, and then when user A's face appears in the target image again, the terminal device can still obtain the same information as the user A through face recognition. A's face corresponds to the piggy model, and user A's face is displayed in the piggy model. Of course, the number of human faces in the above embodiment is not limited here. When determining the correspondence between a human face and a 3D object model, it may be defined according to an actual scene, for example, a larger human face corresponds to a piggy model, etc.; this is not limited in this embodiment of the present disclosure.

In the foregoing embodiment, the target object can also be, for example, a flower, and the attribute information can be the type of flower, the color of the flower, etc.; the target object can also be, for example, a building, and the attribute information can be the height of the building, The color of the building and so on. In this regard, the embodiment of the present disclosure does not make a limitation.

During the actual shooting process of the user or in the image selected by the user, in order to further enhance the interest of the special effect rendering, the attribute information of the target object can also be identified, and the 3D object model matches the attribute information of the corresponding target object.

It may be to first detect the attribute information of the target object. When the target object is a person, the attribute information may be, for example, age, gender, etc.; when the target object is a flower, the attribute information may be, for example, color, flower type, etc. Then, according to the attribute information of the target object, a 3D object model matching the attribute information of the target object is determined for rendering. In some specific embodiments, when the target object is a human face, user attribute information such as the user's gender and age can be calculated according to the information of each feature point on the human face; Information matching 3D object model; for example, when the target object is a male face, obtain a 3D object model with short hair and more muscular body; when the target object is a female face, obtain a 3D object model with long hair and a slender figure. In another example, the user's age attribute is calculated according to the information of each feature point of the face, and a 3D object model matching the user's age attribute is obtained according to the user's age attribute; for example, when the target object is a child's face, Get 3D object models of cute baby animals; get 3D object models of ferocious beasts when the target object is an adult human face, and more. Of course, in addition to the examples of face attribute information listed above, it may also include many other face attribute information, such as whether glasses are worn or not. In addition to face attribute information, other attribute information can also be included. For example, when the target object is a human hand, the attributes of the hand such as the size of the hand, the degree of aging of the hand, etc.; when the target object is a tree, the attribute of the tree can be The type of number, the height of the tree, and so on. Regarding the attribute examples listed above, based on the attribute information of the target object, it is determined to prefabricate a 3D object model that matches the above attributes. The above-mentioned examples are all examples, and the present disclosure is not intended as a limitation.

In a specific embodiment, when the 3D object model includes a virtual human face model, not only can the 3D object model matching the attribute information of the target object be obtained according to the attribute information of the target object, but also can be determined according to the attribute information of the target object. The attributes of the 3D scene model are specifically implemented as follows: rendering the 3D scene model also includes: obtaining the attribute information of the target object; according to the attribute information of the target object, determining the attribute information of the 3D scene model, based on the depth information of the non-covered area in the 3D scene and color information and attribute information of the 3D scene model to render the 3D scene model.

It may first detect the attribute information of the target object, determine the attribute information of the 3D scene model matching the attribute information of the target object according to the attribute information of the target object, and then determine the 3D scene model to be rendered for rendering. In a specific embodiment, for example, when the target object is a human face, user attribute information such as the user's gender and age can be calculated according to the information of each feature point on the human face; The attribute information of the matched 3D scene model, for example, the target object is a child's face, then determine that the attribute of the 3D scene model is a child attribute, then render the 3D scene model including children's elements, and the children's elements can include, for example, cute cartoon characters etc.; if the target object is the face of an adult, determine that the attribute of the 3D scene model is an attribute of an adult, then render the 3D scene model including adult elements, and the adult elements may include, for example, ferocious beasts and the like. If the attribute information of the 3D scene model is determined according to the user's gender attribute, the 3D scene model including female elements can be determined according to the user's female attribute, and the 3D scene model including male elements can be determined according to the user's male attribute , wherein, the female attribute can be that the color of the 3D scene model is pink series, etc., and the male attribute can be that the color of the 3D scene model is blue series, etc. The embodiments of the present disclosure are only used for illustration and not for limitation.

In the foregoing embodiment, the target object can also be, for example, a flower, and the attribute information can be the type of flower, the color of the flower, etc.; the target object can also be, for example, a building, and the attribute information can be the height of the building, The color of the building and so on. In this regard, the embodiment of the present disclosure does not make a limitation.

In some embodiments, the attribute information of the target object may be determined first, and the first 3D scene model corresponding to the attribute information is determined according to the attribute information of the target object. When rendering the first 3D scene model, the points on the target image respectively correspond to the points on the first 3D scene model. For the pixel point set M1 corresponding to the non-target area in the first 3D scene model, determine the depth information and color information of each point in the pixel point set M1, and then calculate the depth information and color information of each point in the pixel point set M1 Assign the corresponding pixel points on the target image; for the pixel point set M2 of the coverage area in the first 3D scene model, determine the depth information of each point in the pixel point set M2, and set the depth information of each point in the pixel point set M2 The information is assigned to the corresponding pixel on the target image. Based on the above method, the first 3D scene model is rendered on the target image.

In some embodiments, the attribute information of the target object may also be determined first, and the attribute information of the 3D scene model is determined according to the attribute information of the target object. When rendering the 3D scene model, the points on the target image correspond to the points on the 3D scene model. For the pixel point set N1 corresponding to the non-target area in the 3D scene model, determine the depth information and color information of each point in the pixel point set N1, and then change the color information according to certain preset rules according to the above attribute information; The depth information and the changed color information of each point in the pixel point set N1 are assigned to the corresponding pixel points on the target image; for the pixel point set N2 in the coverage area in the 3D scene model, determine each pixel point set N2 The depth information of the point is to assign the depth information of each point in the pixel point set N2 to the pixel point corresponding to each point in the pixel point set N2 on the target image. Based on the above method, the 3D scene model is rendered on the target image.

The above embodiments are all illustrations of the embodiments of the present disclosure.

Referring to FIG. 10 , FIG. 10 is a specific flowchart of an embodiment of the present disclosure.

According to S1002, the target image is obtained, and the target image includes the target object; the target image may be a target image obtained during the user's real-time shooting through the camera, or a target obtained in a pre-shot video or image Image; the target object can be a body part such as a human face, a human hand or a human foot, or a small animal, a plant, or a building, etc., and the user can modify the type of the target object according to the actual shooting scene; the individuality of the target object Number can be one or more. According to S1004, a prefabricated virtual 3D object model and a 3D scene model are obtained, the 3D object model includes the target area; the 3D object model is located in the 3D scene model. Wherein, the obtained 3D object model may be to obtain a 3D object model for one target object, and may also be to obtain an equal amount of 3D object models for multiple target objects; wherein, the above-mentioned equivalent 3D object models may be equivalent The 3D object models may also be equivalent and different 3D object models, which is not limited in this disclosure. According to S1006, based on the depth information of the 3D scene model and the color information of the non-target area on the 3D object model except the target area, and based on the color information of the non-covered area in the 3D scene model except the covered area Depth information and color information, rendering the 3D object model and the 3D scene model on the target image to obtain a rendering result, wherein the target object is displayed in the target area, and the coverage area is rendered The area covered by the target area in the 3D object model from the perspective of the camera; wherein, for rendering the 3D scene model, the coverage area in the 3D scene model can be removed before rendering the 3D scene model, and then based on the non-coverage in the 3D scene model The depth information and color information of the region render the 3D scene model; before rendering the 3D scene model, the coverage area in the 3D scene model is not removed, but based on the depth information of the 3D scene model and the non-coverage area in the 3D scene model The color information for rendering the 3D scene model. The effect displayed on the rendering result image is: the target object is displayed in the target area in the prefabricated 3D object model, and the 3D object model is located in the 3D scene model. That is, the target object is integrated into the virtual 3D object model and the 3D scene model, so that the user has a more immersive feeling, thereby improving the user's experience.

Certainly, all the above embodiments are examples, and the algorithms therein are only examples, and are not intended to limit the embodiments of the present disclosure. In the embodiments of the present disclosure, one or more algorithms may be used to calculate the attribute information and feature information of the target object, and the embodiments of the present disclosure will not be elaborated here.

The above descriptions are only specific implementation manners of the embodiments of the present disclosure, and are not intended to limit the present solution. It should be pointed out that those skilled in the art can make some improvements and modifications without departing from the principle of the solution, and these improvements and modifications should also be regarded as the protection scope of the solution.

Referring to FIG. 11 , an embodiment of the present disclosure also provides a special effect rendering device, which includes:

A first acquisition module 1102, configured to acquire a target image, where the target image includes a target object;

A second acquiring module 1104, configured to acquire a 3D object model, where the 3D object model includes a target area;

A rendering module 1106, configured to render the 3D object model onto the target image based on the depth information of the 3D object model and the color information of non-target areas other than the target area on the 3D object model to A rendering result is obtained, wherein the target object is displayed in the target area.

In some embodiments, the target image acquired by the first acquiring module 1102 may be the target image captured by the user through a camera in real time, or the target image captured in a pre-shot video or image. Before acquiring the target image, it is first determined whether the target object is included in the image, and the target image including the target object is acquired, and special effect rendering processing may not be performed for images not including the target object. The target object can be a body part such as a human face, a human hand or a human foot, or a small animal, or a building, etc. The user can modify the type of the target object according to the actual shooting scene.

In some embodiments, the 3D object model acquired by the second acquiring module 1104 is a prefabricated virtual 3D object model, and the 3D object model includes the target area. Among them, the type of 3D object model can be a small animal, such as a piglet, a small monkey, etc., or an object, such as a mirror, a painting, etc., which can be pre-made according to the effect required by the actual scene. Nice different types of 3D object models. The target area included in the 3D object model may be a body part of the 3D object model, for example, when the 3D object model is a pig, the target area may be the face of the pig, the hands of the pig, or the feet of the pig, etc. ; The target area included in the 3D object model can also be a specified range in the 3D object model, for example, when the 3D object model is a picture, the target area can be a rectangular area in the middle of the picture, or It is the range covered by a certain person or thing in the painting. Of course, the target area in the 3D object model can be set according to the effect required by the actual scene.

In some embodiments, the second acquiring module 1104 can also acquire a pre-fabricated virtual 3D scene model, where the 3D object model is located in the 3D scene model.

In some embodiments, when the rendering module 1106 renders the 3D object model on the target image, it may assign the depth information of the target region to the pixels corresponding to the target region on the target image, and assign the non-target The depth information and color information of the region are assigned to the pixels corresponding to the non-target region in the 3D object model on the target image. The final effect is to integrate a part of the real world into the virtual 3D object model, and integrate the target object into the 3D object model.

In some embodiments, the rendering module 1106 also renders both the 3D object model and the 3D scene model on the target image. Wherein, the 3D object model is located in the 3D scene model, based on the way of rendering the 3D object model in the previous embodiment, it also includes: under the viewing angle of the rendering camera, the target area of the 3D object model will cover the coverage area in the 3D scene model, In one embodiment, before rendering the 3D scene model, the coverage area in the 3D scene model is eliminated, and then the 3D scene model is rendered based on the depth information and color information of the non-coverage area in the 3D scene model, so that the final displayed rendering effect is : The target object is displayed in the 3D object model, and the 3D object model is displayed in the 3D scene model. In another embodiment, before rendering the 3D scene model, the coverage area in the 3D scene model is not removed, but the 3D scene model is rendered based on the depth information of the 3D scene model and the color information of the non-coverage area in the 3D scene model , so that the final displayed rendering effect is: the target object is displayed in the 3D object model, and the 3D object model is displayed in the 3D scene model.

In some embodiments, the device in the embodiment of the present disclosure further includes an animation setting module, configured to preset motion animation of the 3D object model and motion animation of the 3D scene model. In a specific embodiment, during 3D modeling, the animation setting module first divides the 3D object model and/or 3D scene model into blocks according to requirements, for example, divides the 3D object model into multiple sub-object models, and divides the 3D object model into multiple sub-object models. The scene model is divided into multiple sub-scene models; after the model is rendered, the 3D sub-object model and/or the 3D sub-scene model moves along their corresponding motion paths, and the 3D object model and/or 3D object model that integrates the target object into the motion can be realized. Effects in the scene model. Of course, according to the actual application requirements, it is also possible to only set the motion animation of the 3D object model without setting the motion animation of the 3D scene model; or only set the motion animation of the 3D scene model without setting the motion animation of the 3D object model, so that Meet the individual needs of users. Since each block is in motion, the target 3D sub-object model and/or the 3D sub-scene model may move beyond the display range of the rendering result image. In a specific embodiment, the above-mentioned animation setting module further includes a moving subunit, which is used for moving the target 3D sub-object model and/or the target 3D sub-scene model whose motion exceeds the display range of the rendering result image. The target 3D sub-object model and/or the target 3D sub-scene model quickly moves back to the initial position and maintains the motion of the target 3D sub-object model and/or the target 3D sub-scene model.

In some embodiments, the first acquisition module 1102 can also be used to acquire the orientation of the target object, and the device in the embodiment of the present disclosure further includes a first adjustment module, configured to adjust the 3D model and/or the 3D scene model based on the orientation of the target object orientation. The device in the embodiment of the present disclosure further includes a second adjustment module, configured to adjust the angle parameter of the rendering camera based on the orientation of the target object.

In some embodiments, the device in the embodiment of the present disclosure further includes a corresponding module, which is used to obtain multiple 3D object models when the target image includes multiple target objects, and the 3D object models correspond to the target objects one by one, wherein , each 3D object model includes the target region.

In some embodiments, the device in the embodiment of the present disclosure further includes a recognition module, which recognizes the target image to obtain a recognition result; and further determines the 3D object model corresponding to each target object from multiple 3D object models based on the recognition result.

In some embodiments, the recognition module may obtain attribute information of the target object, and obtain a 3D object model matching the attribute information of the target object according to the attribute information of the target object.

In some embodiments, the attribute information of the 3D scene model can also be determined according to the attribute information of the target object acquired by the identification module, based on the depth information and color information of the non-coverage area in the 3D scene model except the above-mentioned coverage area. and attribute information of the 3D scene model to render the 3D scene model.

As for the device embodiment, since it basically corresponds to the method embodiment, for related parts, please refer to the part description of the method embodiment. The device embodiments described above are only illustrative, and the modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in One place, or it can be distributed to multiple network modules. Part or all of the modules can be selected according to actual needs to achieve the purpose of the disclosed solution. It can be understood and implemented by those skilled in the art without creative effort.

Embodiments of the special effect rendering apparatus of the present disclosure can be applied to computer equipment, such as servers or terminal equipment. In one embodiment, the computer device further includes a camera; where the camera is used to acquire target images captured by the user in real time. The device embodiments can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by reading the corresponding computer program instructions in the non-volatile memory into the memory for execution by its processor. From the hardware level, as shown in Figure 12, it is a hardware structure diagram of the computer equipment where the special effect rendering device of the embodiment of the present disclosure is located, except for the processor 1210, memory 1230, network interface 1220, and non-easy In addition to the volatile memory 1240, the server or electronic device where the special effect rendering device 1231 is located in the embodiment may generally include other hardware according to the actual functions of the computer device, which will not be repeated here.

An embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the method described in any one of the foregoing embodiments is implemented. Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can be implemented by any method or technology for storage of information. Information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Flash memory or other memory technology, Compact Disc Read-Only Memory (CD-ROM), Digital Versatile Disc (DVD) or other optical storage, Magnetic tape cartridge, tape magnetic disk storage or other magnetic storage device or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media excludes transitory computer-readable media, such as modulated data signals and carrier waves.

The foregoing describes specific embodiments of the present disclosure. Other implementations are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results.

Other embodiments of the present disclosure will be readily apparent to those skilled in the art from consideration of the specification and practice of the invention claimed herein. The present disclosure is intended to cover any modification, use or adaptation of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or conventional technical means in the technical field for which the present disclosure does not apply . The specification and examples are to be considered exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It should be understood that the present disclosure is not limited to the precise constructions which have been described above and shown in the drawings, and various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

The above descriptions are only preferred embodiments of the present disclosure, and are not intended to limit the present disclosure. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present disclosure shall be included in the present disclosure within the scope of protection.

Claims (15)

1. A special effect rendering method, characterized in that the method comprises:

   Acquiring a target image, where the target image includes a target object;

   obtaining a 3D object model, the 3D object model including the target area;

   Rendering the 3D object model onto the target image to obtain a rendering result image based on the depth information of the 3D object model and the color information of non-target areas other than the target area on the 3D object model, wherein , the target object is displayed in the target area.
2. The method according to claim 1, wherein, based on the depth information of the 3D object model and the color information of non-target areas other than the target area on the 3D object model, the 3D object model is Render onto the target image, including:

   assigning depth information of the target area to pixels corresponding to the target area on the target image, and

   assigning the depth information and color information of the non-target area in the 3D object model to the pixel points corresponding to the non-target area in the 3D object model on the target image.
3. The method according to claim 1, wherein the 3D object model is located in a pre-acquired 3D scene model, and the method further comprises:

   Render the 3D scene model onto the target image to obtain the rendering result image based on the depth information and color information of the non-coverage area in the 3D scene model except the coverage area, wherein the coverage area is The area covered by the target area in the 3D object model from the perspective of the camera is rendered.
4. According to the described method of claim 3, it is characterized in that,

   The 3D object model includes a plurality of sub-object models, at least one of which moves along a corresponding motion path; and/or

   The 3D scene model includes a plurality of sub-scene models, and at least one of the sub-scene models moves along a corresponding motion path.
5. The method according to claim 4, wherein the method further comprises:

   Obtain the display range of the rendering result image,

   Rendering the sub-object model moving beyond the display range at the starting position on the motion path corresponding to the sub-object model.
6. The method according to claim 4, wherein the method further comprises:

   Obtain the display range of the rendering result image,

   Render the sub-scene model whose motion exceeds the display range at the starting position on the motion path corresponding to the sub-scene model.
7. The method according to claim 1 or 3, wherein, before rendering the 3D object model and/or 3D scene model, the method further comprises:

   Obtain the orientation of the target object;

   Adjusting the orientation of the 3D object model and/or the 3D scene model in the rendering result image based on the orientation of the target object.
8. The method according to claim 1 or 3, wherein, before rendering the 3D object model and/or 3D scene model, the method further comprises:

   Obtain the orientation of the target object;

   Adjusting the angle parameters of the rendering camera based on the orientation of the target object; where the orientations of the 3D object model and/or the 3D scene model displayed on the rendering result image are different under different angle parameters.
9. The method according to any one of claims 1 to 8, wherein, in the case where the target image includes a plurality of target objects, the acquiring a 3D object model includes:

   A plurality of 3D object models are acquired, the 3D object models respectively correspond to the target objects, and each 3D object model includes a target area.
10. The method according to claim 9, wherein said acquiring a 3D object model comprises:

    Recognizing the target image to obtain a recognition result;

    A 3D object model corresponding to each of the target objects is respectively determined from the plurality of 3D object models based on the recognition result.
11. The method according to claim 10, wherein the recognition result includes attribute information of the target object, and the 3D object model matches the corresponding attribute information of the target object.
12. The method according to claim 3, wherein the method further comprises:

    Obtain attribute information of the target object;

    determining the attribute information of the 3D scene model according to the attribute information of the target object;

    The rendering of the 3D scene model on the target image based on the depth information and color information of the non-covered area in the 3D scene model includes:

    Rendering the 3D scene model based on the depth information and color information of the non-coverage area in the 3D scene model and the attribute information of the 3D scene model.
13. A special effect rendering device, characterized in that the device comprises:

    A first acquiring module, configured to acquire a target image, where the target image includes a target object;

    A second acquiring module, configured to acquire a 3D object model, the 3D object model including the target area;

    A rendering module, configured to render the 3D object model onto the target image based on the depth information of the 3D object model and the color information of non-target areas on the 3D object model except the target area to obtain A resulting image is rendered, wherein the target object is displayed within the target area.
14. A special effect rendering device, comprising a memory, a processor, and a computer program stored on the memory and operable on the processor, wherein, when the processor executes the computer program, the computer program described in any one of claims 1 to 12 is implemented. described method.
15. A computer-readable storage medium, on which a computer program is stored, characterized in that, when the computer program is executed by a processor, the steps in the method of any one of claims 1 to 12 are implemented.

Images