CN117240831A

CN117240831A - Three-dimensional model transmission method and device, storage medium and program product thereof

Info

Publication number: CN117240831A
Application number: CN202210640757.1A
Authority: CN
Inventors: 王志强
Original assignee: ZTE Corp
Current assignee: ZTE Corp
Priority date: 2022-06-08
Filing date: 2022-06-08
Publication date: 2023-12-15
Also published as: WO2023236815A1

Abstract

The application provides a three-dimensional model transmission method and a device, a storage medium and a program product thereof, wherein the method comprises the following steps: acquiring a plurality of video images, wherein the video images are obtained by a design end through frame extraction processing of a video stream of an augmented reality video call established by the design end and a client; dividing a plurality of video images to obtain a physical image set and an environment image set; modeling processing is carried out on the physical image set and the environment image set respectively to obtain a physical three-dimensional model and an environment three-dimensional model; generating a first three-dimensional scene model according to the physical three-dimensional model and the environment three-dimensional model; and sending the first three-dimensional scene model to the client, namely, obtaining a plurality of video images according to the video stream of the augmented reality video call under the scene of the augmented reality video call established between the design end and the client, obtaining the first three-dimensional scene model according to the video images, and sending the first three-dimensional scene model to the client to realize the transmission of the three-dimensional model.

Description

Three-dimensional model transmission method and device, storage medium and program product thereof

Technical Field

Embodiments of the present application relate to, but are not limited to, the field of communications technologies, and in particular, to a three-dimensional model transmission method, a device, a storage medium, and a program product thereof.

Background

In the related art, in an AR (Augmented Reality ) scene, only files such as video, text, and pictures can be transmitted between a client and a design end, but three-dimensional (3D) models cannot be transmitted. Therefore, how to realize the transmission of the three-dimensional model in the augmented reality scene is a problem to be solved.

Disclosure of Invention

The embodiment of the application provides a three-dimensional model transmission method, a device, a storage medium and a program product thereof, which can realize the transmission of a three-dimensional model in an augmented reality scene.

In a first aspect, an embodiment of the present application provides a three-dimensional model transmission method, including:

acquiring a plurality of video images from a design end, wherein the video images are obtained by the design end through frame extraction processing of a video stream of an augmented reality video call, and the augmented reality video call is established by the design end and a client; dividing a plurality of video images to obtain a real image set and an environment image set; modeling the real object image set to obtain a real object three-dimensional model; modeling the environment image set to obtain an environment three-dimensional model; generating a first three-dimensional scene model according to the physical three-dimensional model and the environment three-dimensional model; and sending the first three-dimensional scene model to the client.

In a second aspect, an embodiment of the present application provides a three-dimensional model transmission method, including:

establishing an augmented reality video call with a design end; receiving a first three-dimensional scene model sent by a server side, wherein the first three-dimensional scene model is obtained by the server side according to a physical three-dimensional model and an environmental three-dimensional model, the physical three-dimensional model is obtained by the server side through modeling processing of a physical image set, the environmental three-dimensional model is obtained by the server side through modeling processing of an environmental image set, the physical image set and the environmental image set are both obtained by the server side through segmentation processing of a plurality of video images, and the video images are obtained by the design side through frame extraction processing of a video stream of an augmented reality video call.

In a third aspect, an embodiment of the present application further provides a three-dimensional model transmission device, including: the system comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor realizes the three-dimensional model transmission method when executing the computer program.

In a fourth aspect, embodiments of the present application also provide a computer-readable storage medium storing computer-executable instructions for performing the three-dimensional model transmission method as described above.

In a fifth aspect, embodiments of the present application further provide a computer program product, including a computer program or computer instructions stored in a computer-readable storage medium, from which a processor of a computer device reads the computer program or the computer instructions, the processor executing the computer program or the computer instructions, causing the computer device to perform the three-dimensional model transmission method as described above.

The embodiment of the application comprises the following steps: the method comprises the steps that a plurality of video images from a design end are acquired by a server end, the video images are obtained by carrying out frame extraction processing on video streams of an augmented reality video call, the augmented reality video call is established by the design end and a client end, then the video images are segmented to obtain a physical image set and an environment image set, then the physical image set is modeled to obtain a physical three-dimensional model, the environment image set is modeled to obtain an environment three-dimensional model, a first three-dimensional scene model is generated according to the physical three-dimensional model and the environment three-dimensional model, finally the first three-dimensional scene model is sent to the client end, namely, the design end carries out frame extraction processing on the video streams of the augmented reality video call, which are established by the design end and the client end, to obtain a plurality of video images, and the server end carries out processing on the video images to finally obtain the first three-dimensional scene model and sends the first three-dimensional scene model to the client end.

Drawings

FIG. 1 is a flow chart of a three-dimensional model transmission method provided by an embodiment of the present application;

FIG. 2 is a flow chart of one specific method of step S150 in FIG. 1;

FIG. 3 is a flow chart of a three-dimensional model transmission method according to another embodiment of the present application;

FIG. 4 is a flow chart of a specific method of step S130 in FIG. 1;

FIG. 5 is a flow chart of a specific method of step S140 in FIG. 1;

FIG. 6 is a flow chart of a three-dimensional model transmission method according to another embodiment of the present application;

FIG. 7 is a flow chart of a three-dimensional model transmission method according to another embodiment of the present application;

FIG. 8 is a flow chart of a three-dimensional model transmission method according to another embodiment of the present application;

FIG. 9 is a flow chart of a three-dimensional model transmission method provided by a specific example of the present application;

FIG. 10 is a schematic structural view of a three-dimensional model transmission device according to an embodiment of the present application;

fig. 11 is a schematic structural diagram of a three-dimensional model transmission device according to another embodiment of the present application.

Detailed Description

The present application will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present application more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.

It should be noted that although a logical order is illustrated in the flowchart, in some cases, the steps illustrated or described may be performed in an order different from that in the flowchart. In the description of the specification, claims and drawings, the meaning of a plurality (or a plurality of) is two or more, and it is to be understood that greater than, less than, exceeding, etc. does not include the present number, and that above, below, within, etc. are to be interpreted as including the present number. If any, the terms "first," "second," etc. are used for distinguishing between technical features only, and should not be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated or implicitly indicating the precedence of the technical features indicated.

The application provides a three-dimensional model transmission method, a device, a storage medium and a program product thereof, wherein a server side firstly acquires a plurality of video images from a design side, the video images are obtained by the design side through frame extraction processing of video streams of an augmented reality video call, the augmented reality video call is established by the design side and a client side, then the plurality of video images are subjected to segmentation processing to obtain a physical image set and an environment image set, then the physical image set is subjected to modeling processing to obtain a physical three-dimensional model, and the environment image set is subjected to modeling processing to obtain an environment three-dimensional model, a first three-dimensional scene model is generated according to the physical three-dimensional model and the environment three-dimensional model, and finally the first three-dimensional scene model is sent to the client side, namely, the design side carries out frame extraction processing of the video streams of the augmented reality video call established by the design side and the client side to obtain a plurality of video images, and finally the first three-dimensional scene model is obtained by processing the plurality of video images, and the first three-dimensional scene model is sent to the client side.

Embodiments of the present application will be further described below with reference to the accompanying drawings.

Referring to fig. 1, fig. 1 is a flowchart of a three-dimensional model transmission method according to an embodiment of the present application, which may include, but is not limited to, step S110, step S120, step S130, step S140, step S150, and step S160.

Step S110: a plurality of video images from a design side are acquired.

In one possible implementation, the video image is obtained by the design side by performing frame extraction processing on a video stream of the augmented reality video call, and the augmented reality video call is established by the design side and the client side.

It can be understood that, because the video is essentially a continuous frame of picture, the design end can intercept a frame of clear image from the video, and can intercept multiple image sets, i.e. the design end obtains multiple video images by performing frame extraction processing on the video stream of the augmented reality video call.

In a feasible implementation mode, the design end sends a video meeting invitation to the client, and after the client confirms the receiving, the design end and the client can successfully establish the augmented reality video call through the authentication confirmation of the server; or the client sends the video meeting invitation to the design end, after the design end confirms and accepts, the design end and the client can successfully establish the augmented reality video call through confirmation of the queuing system of the server end and the authentication system of the server end, and the method is not particularly limited.

In a possible implementation manner, in the process of the augmented reality video call, audio/video switching can be supported, for example, in the process of the augmented reality video call, the client can switch to a playing interface of the application a, and place the augmented reality video call as a background program, so that the augmented reality video call runs in the background, wherein the application a can be audio or video; for another example, in the process of performing the augmented reality video call with the design end, the client may switch the video call to the voice call, which is not limited herein.

In a possible implementation manner, a video stream of the augmented reality video call can adopt YUV coding and h.264 video protocol, wherein YUV is a color coding format, and "Y" represents brightness, namely a gray scale value; "U" and "V" represent chromaticity, and "U" and "V" can describe image color and saturation, and can be used for designating the color of a pixel, wherein the H.264 video protocol is a digital video compression coding standard.

In a possible implementation manner, the video stream of the augmented reality video call may be a video stream of the client side, a video stream of the design side, a video stream of the client side and a video stream of the user side, which may be obtained according to actual situations, and is not limited herein. For example, after the design end and the client end establish the augmented reality video call, the design end can perform frame extraction processing on the video stream at the design end side to obtain a plurality of video images; or after the design end establishes the augmented reality video call with the client, the design end can perform frame extraction processing on the video stream at the client side to obtain a plurality of video images; or after the design end and the client end establish the augmented reality video call, the design end can respectively perform frame extraction processing on the video stream at the design end side and the video stream at the client end side to obtain a plurality of video images.

Step S120: and carrying out segmentation processing on the plurality of video images to obtain a physical image set and an environment image set.

It is understood that the set of physical images comprises a plurality of physical images and the set of environmental images comprises a plurality of environmental images.

One possible implementation manner is to divide a plurality of video images, and many implementations are possible, for example, the method may be to divide a plurality of video images by adopting an edge contour scanning manner to obtain a physical image set and an environmental image set; or, the deep learning algorithm may be used to identify the physical image and the environmental image in the multiple video images, and determine the dividing area corresponding to the identified physical image and the dividing area corresponding to the environmental image in the multiple video images according to the composition rule, and then obtain the physical image set and the environmental image set according to the dividing area corresponding to the physical image and the dividing area corresponding to the environmental image, where the composition rule may include a rule that sets the position and the occupied area of the physical image or the environmental image in the dividing area, and is not limited in particular herein.

Step S130: modeling is carried out on the physical image set, and a physical three-dimensional model is obtained.

Step S140: modeling is carried out on the environment image set, and an environment three-dimensional model is obtained.

Step S150: and generating a first three-dimensional scene model according to the physical three-dimensional model and the environment three-dimensional model.

Step S160: the first three-dimensional scene model is sent to the client.

In a possible implementation manner, the server side may send the first three-dimensional scene model to the design side, which is not limited herein.

In this embodiment, by adopting the three-dimensional model transmission method including the steps S110 to S160, firstly, the server side obtains a plurality of video images from the design side, the video images are obtained by the design side by performing frame extraction processing on the video stream of the augmented reality video call, the augmented reality video call is established by the design side and the client side, then the plurality of video images are subjected to segmentation processing to obtain a physical image set and an environmental image set, then the physical image set is subjected to modeling processing to obtain a physical three-dimensional model, and the environmental image set is subjected to modeling processing to obtain an environmental three-dimensional model, a first three-dimensional scene model is generated according to the physical three-dimensional model and the environmental three-dimensional model, and finally the first three-dimensional scene model is sent to the client side, that is, under the scene of the augmented reality video call established by the design side and the client side, the video stream of the augmented reality video call is subjected to frame extraction processing to obtain a plurality of video images, and the server side finally obtains the first three-dimensional scene model and sends the first three-dimensional scene model to the client side.

In a possible implementation manner, the design side, the server side and the client side may perform three-dimensional rendering processing on the first three-dimensional scene model, which is not limited herein. For example, the client performs three-dimensional rendering processing on the first three-dimensional scene model sent by the server to obtain a rendered first three-dimensional scene model, and then displays the rendered first three-dimensional scene model; or when the server side sends the first three-dimensional scene model to the design side, the design side performs three-dimensional rendering treatment on the first three-dimensional scene model sent by the server side to obtain a rendered first three-dimensional scene model, and then the rendered first three-dimensional scene model is displayed; or the server side performs three-dimensional rendering processing on the first three-dimensional scene model to obtain a rendered first three-dimensional scene model, and then sends the rendered first three-dimensional scene model to the design side or the client side.

In a possible implementation manner, the three-dimensional rendering process may be performed on the first three-dimensional scene model multiple times, and whether the three-dimensional rendering process is required to be performed on the first three-dimensional scene model multiple times may be determined according to the model scale of the first three-dimensional scene model. For example, when the first three-dimensional scene model simulates an amusement park, three-dimensional rendering processing can be performed on the first three-dimensional scene model for multiple times, that is, the server side performs three-dimensional rendering processing on the first three-dimensional scene model to obtain a rendered first three-dimensional scene model, then the rendered first three-dimensional scene model is sent to the design side (or the client side), the design side (or the client side) performs three-dimensional rendering processing on the rendered first three-dimensional scene model sent by the server side again to obtain a re-rendered first three-dimensional scene model, and the re-rendered first three-dimensional scene model is displayed; alternatively, when the first three-dimensional scene model simulates a bedroom, the three-dimensional rendering process may be performed on the first three-dimensional scene model only once, and the three-dimensional rendering process may be performed on the first three-dimensional scene model by the design end, the server end, or the client end, which is not particularly limited herein.

It will be appreciated that three-dimensional rendering is essentially a computer-processed conversion process of a three-dimensional model into a two-dimensional image. Rendering involves techniques such as scanline rendering, ray tracing, photon mapping, etc., which can simulate the interaction of light with substances (e.g., material materials and surface textures), and therefore, three-dimensional plug-ins, three-dimensional software, and hardware support are required for rendering.

In one embodiment, as shown in fig. 2, step S150 is further described, and step S150 may include, but is not limited to, step S210, step S220, and step S230.

Step S210: obtaining a preset three-dimensional model library, wherein the three-dimensional model library comprises an environment model library.

In a possible embodiment, the environment model library may include a plurality of preset environment models, where the plurality of preset environment models may be obtained by modifying existing model data by using a modeling tool (such as a 3D Studio MAX, where the 3D Studio MAX is based on three-dimensional animation rendering and making software of a computer operating system) by using a modeling tool, and may also be an existing model, which is not limited herein.

In a possible implementation manner, the preset three-dimensional model library may include a target three-dimensional scene model, model related data, a model base component, model material materials, and the like, where the model related data includes a model scale, location information of a real scene simulated by the model, and the like, and is not limited herein. For example, in a route planning scene, when a user is at a position under a mountain, the user sends position information to a server through an AR device, the server determines a target three-dimensional scene model corresponding to the mountain from a preset three-dimensional model library according to the position information, and sends the target three-dimensional scene model to a design end, the design end performs route planning on the mountain according to the target three-dimensional scene model and marks marked scenic spots, shops, washrooms and the like on the mountain route, and related information of the route planning and the marks is pushed to an AR client, so that user experience is improved, wherein the AR device is the client.

Step S220: and determining the target environment three-dimensional model from the environment model library according to the physical three-dimensional model and the environment three-dimensional model.

In an embodiment, the real three-dimensional model and the environment three-dimensional model can be matched with a preset environment model in the environment model library, the target environment three-dimensional model is determined, the modeling efficiency is improved, and the modeling time is shortened.

Step S230: and generating a first three-dimensional scene model according to the physical three-dimensional model and the target environment three-dimensional model.

In this embodiment, by adopting the three-dimensional model transmission method including the steps S210 to S230, firstly, the server obtains a preset three-dimensional model library, where the three-dimensional model library includes an environment model library, then determines a target environment three-dimensional model from the environment model library according to the physical three-dimensional model and the environment three-dimensional model, and finally generates a first three-dimensional scene model according to the physical three-dimensional model and the target environment three-dimensional model.

In an embodiment, as shown in fig. 3, after performing step S160, the three-dimensional model transmission method may further include, but is not limited to, step S310, step S320, step S330, step S340, and step S350.

Step S310: and receiving the labeling information sent by the client.

In a possible implementation manner, the labeling information may include coordinate information, text information, a labeled model, and the like, and the labeling information is obtained by performing virtual labeling processing on the first three-dimensional scene model by the client. For example, when the first three-dimensional scene model simulates a living room, and a sofa is placed in the living room at the position of coordinates (8,4,0), if the sofa needs to be replaced by the position of coordinates (4, 0), the coordinate information may be that the three-dimensional model of the real object at the position of coordinates (8,4,0) is modified to the position of coordinates (4, 0).

Step S320: and determining an annotation region in the first three-dimensional scene model according to the annotation information.

It will be understood that the labeling area may be an area before modification, an area after modification, or an area before modification and an area after modification, for example, if the labeling information is that the three-dimensional model of the real object at the coordinate (8,4,0) is modified to the position at the coordinate (4, 0), the labeling area may be an area at the coordinate (8,4,0) or an area at the coordinate (4, 0), which is not limited herein specifically.

Step S330: and nesting marking information on the marking area to obtain a second three-dimensional scene model.

Step S340: and performing superposition processing on the first three-dimensional scene model and the second three-dimensional scene model to obtain a third three-dimensional scene model.

Step S350: and sending the third three-dimensional scene model to the client.

In a possible implementation, the server side may send the third three-dimensional scene model to the design side, which is not limited herein.

In this embodiment, by adopting the three-dimensional model transmission method including steps S310 to S350, firstly, the server receives the labeling information sent by the client, determines the labeling area in the third three-dimensional scene model according to the labeling information, then nests the labeling information on the labeling area to obtain the second three-dimensional scene model, then carries out superposition processing on the third three-dimensional scene model and the second three-dimensional scene model to obtain the third three-dimensional scene model, and finally sends the third three-dimensional scene model to the client.

In a possible implementation manner, the design side, the server side and the client side may perform three-dimensional rendering processing on the third three-dimensional scene model, which is not limited herein. For example, the client performs three-dimensional rendering processing on the third three-dimensional scene model sent by the server to obtain a rendered third three-dimensional scene model, and then displays the rendered third three-dimensional scene model; or when the server side sends the third three-dimensional scene model to the design side, the design side performs three-dimensional rendering treatment on the third three-dimensional scene model sent by the server side to obtain a rendered third three-dimensional scene model, and then the rendered third three-dimensional scene model is displayed; or the server side performs three-dimensional rendering processing on the third three-dimensional scene model to obtain a rendered third three-dimensional scene model, and then sends the rendered third three-dimensional scene model to the design side or the client side.

In a possible implementation manner, the third three-dimensional scene model may be subjected to three-dimensional rendering processing multiple times, and whether the third three-dimensional scene model needs to be subjected to three-dimensional rendering processing multiple times may be determined according to the model scale of the third three-dimensional scene model. For example, when the third three-dimensional scene model simulates an amusement park, three-dimensional rendering processing can be performed on the third three-dimensional scene model for multiple times, that is, the server side performs three-dimensional rendering processing on the third three-dimensional scene model to obtain a rendered third three-dimensional scene model, then the rendered third three-dimensional scene model is sent to the design side (or the client side), the design side (or the client side) performs three-dimensional rendering processing on the rendered third three-dimensional scene model sent by the server side again to obtain a re-rendered third three-dimensional scene model, and the re-rendered third three-dimensional scene model is displayed; alternatively, when the third three-dimensional scene model simulates a bedroom, the third three-dimensional scene model may be subjected to three-dimensional rendering processing only once, and the third three-dimensional scene model may be subjected to three-dimensional rendering processing by the design end, the server end or the client end, which is not particularly limited herein.

In an embodiment, as shown in fig. 4, in the case that the three-dimensional model library includes a physical model library, step S130 is further described, and the step S130 may include, but is not limited to, step S410, step S420, step S430, and step S440.

Step S410: and analyzing the object image set to obtain the graphic information.

In a possible embodiment, the graphic information may include a line profile of the physical image, a position of the physical image, a size of the physical image, a resource address of the physical image, and the like, which are not particularly limited herein.

Step S420: and acquiring the physical image element information from the physical model library according to the graphic information.

In a possible implementation manner, the physical model library may include a plurality of preset physical models, where the plurality of preset physical models may be obtained by modifying existing model data by using a modeling tool (such as a 3D Studio MAX) by a model maker, where the 3D Studio MAX is based on three-dimensional animation rendering and making software of a computer operating system, and the model may also be obtained by performing modeling processing on the modified model data, and may also be an existing model, which is not limited herein.

Step S430: and constructing a physical image base map by using the physical image element information.

In a possible implementation, the physical image base map may be a basic frame of a physical three-dimensional model, which is not particularly limited herein.

Step S440: modeling processing is carried out on the base map of the physical image, and a physical three-dimensional model is obtained.

In an embodiment, when the bottom image of the physical image is a contour of a vase, modeling the vase, that is, performing material veneering, light modulation, texture, recessing, protrusion treatment on the vase, and finally obtaining a physical three-dimensional model of the vase.

In this embodiment, by adopting the three-dimensional model transmission method including the steps S410 to S440, firstly, the server end analyzes the physical image set to obtain graphic information, obtains physical image element information from the physical model library according to the graphic information, then constructs a physical image base map by using the physical image element information, and finally models the physical image base map to obtain a physical three-dimensional model.

In one embodiment, as shown in fig. 5, further describing step S140, step S140 may include, but is not limited to, step S510, step S520, and step S530.

Step S510: and analyzing the environment image set to obtain label information.

In a possible embodiment, the tag information may include material attribute information, color attribute information, light source attribute information, etc. of the environmental appearance, which is not particularly limited herein.

Step S520: and acquiring the environmental image element information from the environmental model library according to the label information.

In a possible embodiment, the environmental image element information may include index information of material attribute information, color attribute information, light source attribute information, etc. of the environmental appearance, which is not particularly limited herein.

Step S530: and modeling the environment image set by using the environment image element information to obtain an environment three-dimensional model.

In a possible implementation manner, the modeling processing is performed on the environmental image set by using the environmental image element information, which may include material veneering, dimming, texture, recessing, protrusion processing, collapsing processing, and the like, so that the model has a stereoscopic impression, which is not particularly limited herein.

In this embodiment, by adopting the three-dimensional model transmission method including the steps S510 to S530, firstly, the server end analyzes the environmental image set to obtain the tag information, then obtains the environmental image element information from the environmental model library according to the tag information, and finally uses the environmental image element information to perform modeling processing on the environmental image set to obtain the environmental three-dimensional model.

In an embodiment, as shown in fig. 6, the three-dimensional model transmission method may further include, but is not limited to, step S610 and step S620.

Step S610: and determining the target environment three-dimensional model from the environment model library according to the physical three-dimensional model.

In an embodiment, the three-dimensional model of the object and the preset environmental model in the environmental model library can be subjected to adaptation processing to determine the three-dimensional model of the target environment, and modeling processing on the environmental image set is not needed, so that the modeling time is shortened.

Step S620: and performing nesting processing on the physical three-dimensional model and the target environment three-dimensional model to generate a first three-dimensional scene model.

In this embodiment, by adopting the three-dimensional model transmission method including the steps S610 to S620, firstly, the server determines the target environment three-dimensional model from the environment model library according to the three-dimensional model of the real object, and finally, the three-dimensional model of the real object and the three-dimensional model of the target environment are subjected to nesting processing, so as to generate the first three-dimensional scene model, that is, only the real object image set can be modeled, so as to obtain the three-dimensional model of the real object, only the three-dimensional model of the target environment is determined from the environment model library according to the three-dimensional model of the real object, the three-dimensional model of the real object and the three-dimensional model of the target environment are subjected to nesting processing, so that the modeling processing of the environment image set is not needed, the modeling time is shortened, and meanwhile, the real object in the scene where the client or the design end is located can be adapted to the preset environment three-dimensional model in the environment model library through the form of the three-dimensional model, so as to achieve the effect of remote design, satisfy the diversified demands of users, and improve the user experience.

In an embodiment, when a living room is subjected to home layout design, a client can send a real object image set including a sofa, a refrigerator, a seat and the like to a server, the server performs modeling processing on the real object image set to obtain a plurality of real object three-dimensional models, then the server determines a target environment three-dimensional model from an environment model library according to the plurality of real object three-dimensional models, and performs nesting processing on the real object three-dimensional model and the target environment three-dimensional model to generate a first three-dimensional scene model.

In one embodiment, further describing step S150, step S150 may include, but is not limited to, the following steps:

and performing nesting processing on the physical three-dimensional model and the environment three-dimensional model to generate a first three-dimensional scene model.

In this embodiment, the server may acquire a plurality of video images from the design end, where the video images are obtained by performing frame extraction processing on a video stream of an augmented reality video call, where the augmented reality video call is established by the design end and the client end, then performing segmentation processing on the plurality of video images to obtain a physical image set and an environmental image set, then performing modeling processing on the physical image set to obtain a physical three-dimensional model, and performing parsing processing on the environmental image set to obtain graphic information, and then performing nesting processing on the physical three-dimensional model and the environmental three-dimensional model to generate a first three-dimensional scene model.

In an embodiment, when a server side can acquire a plurality of video images from a design side, the plurality of video images are subjected to segmentation processing to obtain a physical image set and an environment image set, then the physical image set is subjected to modeling processing to obtain a physical three-dimensional model, the environment image set is subjected to analysis processing to obtain tag information, adaptation is performed according to the tag information and a preset environment model in an environment model library, when the tag information is not adapted to the preset environment model, the environment image set is subjected to modeling processing to obtain an environment three-dimensional model, and the physical three-dimensional model and the environment three-dimensional model are subjected to nesting processing to obtain a first three-dimensional scene model. In addition, the environmental three-dimensional model may be stored in an environmental model library, which is not particularly limited by the implementation of the present application.

In an embodiment, firstly, a server side obtains an environment image set sent by a client side, and carries out modeling processing on the environment image set to obtain an environment three-dimensional model, then, the server side obtains a plurality of video images from a design side, the video images are obtained by the design side through frame extraction processing on video streams of an augmented reality video call, the augmented reality video call is established by the design side and the client side, then, the plurality of video images are subjected to segmentation processing to obtain a physical image set, and then, the physical image set is subjected to modeling processing to obtain a physical three-dimensional model, a first three-dimensional scene model is generated according to the physical three-dimensional model and the environment three-dimensional model, and finally, the first three-dimensional scene model is sent to the client side.

In addition, fig. 7 is a three-dimensional model transmission method according to another embodiment of the present application, which may include, but is not limited to, step S710 and step S720.

Step S710: and establishing an augmented reality video call with the design end.

In a feasible implementation manner, the design end sends a video meeting invitation to the client, and after the client confirms the receiving, the design end can establish an augmented reality video call with the client through the confirmation of the queuing system of the server and the authentication system of the server; or, the client sends the video meeting invitation to the design end, after the design end confirms and accepts, the client can establish the augmented reality video call with the design end through the queuing system of the server end and the authentication system of the server end, and the method is not particularly limited.

Step S720: and receiving the first three-dimensional scene model sent by the server side.

In a possible implementation manner, the first three-dimensional scene model is obtained by the server according to a physical three-dimensional model and an environmental three-dimensional model, the physical three-dimensional model is obtained by the server through modeling processing of a physical image set, the environmental three-dimensional model is obtained by the server through modeling processing of an environmental image set, the physical image set and the environmental image set are both obtained by the server through segmentation processing of a plurality of video images, and the video image is obtained by the design through frame extraction processing of a video stream of an augmented reality video call.

In a feasible implementation manner, a video stream of the augmented reality video call adopts YUV coding and H.264 video protocol, wherein YUV is a color coding format, and 'Y' represents brightness, namely a gray scale value; "U" and "V" represent chromaticity, and "U" and "V" can describe image color and saturation, and can be used for designating the color of a pixel, wherein the H.264 video protocol is a digital video compression coding standard.

In this embodiment, by adopting the three-dimensional model transmission method including the steps S710 to S720, the client and the design end establish an augmented reality video call, then the client receives a first three-dimensional scene model sent by the server end, wherein the first three-dimensional scene model is obtained by the server end according to a physical three-dimensional model and an environmental three-dimensional model, the physical three-dimensional model is obtained by the server end through modeling a physical image set, the environmental three-dimensional model is obtained by the server end through modeling an environmental image set, the physical image set and the environmental image set are both obtained by the server end through segmentation processing of a plurality of video images, the video image is obtained by the design end through frame extraction processing of a video stream of the augmented reality video call, that is, in the scene of the augmented reality video call established by the design end and the client, the design end obtains a plurality of video images through frame extraction processing of the video stream of the augmented reality video call, the server end processes the plurality of video images to finally obtain the first three-dimensional scene model, and finally the client receives the first three-dimensional scene model sent by the server end, so that the three-dimensional model can be implemented in the implementation of the three-dimensional model.

In one embodiment, as shown in fig. 8, the three-dimensional model transmission method may include, but is not limited to, step S810 and step S820.

Step S810: and performing virtual labeling processing on the first three-dimensional scene model to obtain labeling information.

In a possible embodiment, the labeling information may include coordinate information, text information, labeled models, and the like, which is not particularly limited herein.

In a possible implementation manner, the server side may send the first three-dimensional scene model to the design side, and the design side stores the first three-dimensional scene model. When the client performs virtual labeling processing on the first three-dimensional scene model, the client can send the labeling information to the design end in real time, for example, the client can perform view angle switching on the first three-dimensional scene model, when the first three-dimensional scene model is a house comprising a bedroom, a living room and a bathroom, the client can switch the first three-dimensional scene model to the bedroom area and perform virtual labeling processing on the bedroom area, can switch the first three-dimensional scene model to the living room area and perform virtual labeling processing on the living room area, can switch the first three-dimensional scene model to the bathroom area and perform virtual labeling processing on the bathroom area, and finally sends labeling information of all areas to the design end (or the server end), the design end can receive labeling information of all areas in real time, and perform modification design on the first three-dimensional scene model stored in the design end according to the labeling information, so that remote modification on the three-dimensional model is realized, and user experience is improved.

Step S820: and sending the labeling information to a server side.

In a possible implementation manner, the client may also send the standard information to the design end, where the design end modifies and designs the first three-dimensional scene model stored in the design end according to the standard information, and since each module in the first three-dimensional scene model may be responsible for a different designer, that is, multiple designers modify the first three-dimensional scene model concurrently, a concurrency conflict may occur. In order to avoid the problem of concurrency conflict, a plurality of designers in the design end can modify the source data files corresponding to the modules according to the design copies of the designers, and finally the design end submits all the modified source data files to the server end, and the server end collects the source data files and then uniformly delivers the source data files to the users.

In a possible implementation manner, the labeling information is stored in a related file, and is packaged into a data packet to be sent to a server, and after the server receives the data packet, the data packet is analyzed to obtain the labeling information, which is not particularly limited herein.

In this embodiment, by adopting the three-dimensional model transmission method including steps S810 to S820, firstly, the client may perform virtual labeling processing on the first three-dimensional scene model to obtain labeling information, and send the labeling information to the server, so that the server determines a labeling area in the first three-dimensional scene model according to the labeling information, nests the labeling information on the labeling area to obtain a second three-dimensional scene model, and finally, performs superposition processing on the first three-dimensional scene model and the second three-dimensional scene model to obtain a third three-dimensional scene model, that is, the client may feedback the labeling information to the server, modify the first three-dimensional scene model according to the labeling information through the server, thereby reducing communication cost, being beneficial to meeting requirements of users, and improving user experience.

In an embodiment, in a scenario of travel live-action explanation, a user marks a building B in an AR device, sends the building B to a server through the AR device, the server analyzes a physical image set to obtain graphic information, obtains physical image element information from a physical model library according to the graphic information, then constructs a physical image base map by using the physical image element information, models the physical image base map to obtain a physical three-dimensional model, and finally sends the physical three-dimensional model to a client, and the user sends the building B to a design end through the AR device, and the design end obtains historical data related to the building B from a data storage module according to the building B and sends the historical data to the AR device, wherein the design end can amplify or reduce the physical three-dimensional model in the AR device through voice control or external input mode, and switches the view angle of the physical model in the AR device so as to view details of the three-dimensional model, and the design end does not limit details of the AR device to the client.

With respect to the three-dimensional model transmission method provided in the above embodiment, the following is described in detail with specific examples:

in an embodiment, referring to fig. 9, firstly, an design end and a client establish an augmented reality video call, then the design end performs frame extraction processing on a video stream of the augmented reality video call to obtain a plurality of video images, then performs segmentation processing on the plurality of video images to obtain a physical image set and an environment image set, and then performs analysis processing on the physical image set and the environment image set respectively, wherein the analysis processing on the physical image set can perform analysis processing on the physical image set to obtain graphic information, the physical image element information is obtained from a physical model library according to the graphic information, a physical image base map is constructed by using the physical image element information, and the physical image base map is modeled to obtain a physical three-dimensional model; for the analysis processing of the environment image set, the environment image set can be analyzed to obtain label information, environment image element information is obtained from an environment model library according to the label information, the environment image set is modeled by the environment image element information to obtain an environment three-dimensional model, then a first three-dimensional scene model is generated according to the physical three-dimensional model and the environment three-dimensional model, the first three-dimensional scene model is sent to a client, and the client performs three-dimensional rendering processing on the first three-dimensional scene model; or the server performs three-dimensional rendering processing on the first three-dimensional scene model to obtain a rendered first three-dimensional scene model, then sends the rendered first three-dimensional scene model to the client, the client performs three-dimensional rendering processing on the rendered first three-dimensional scene model again to obtain a rendered first three-dimensional scene model, then the client performs virtual labeling processing on the rendered first three-dimensional scene model or the rendered first three-dimensional scene model again to obtain labeling information, the client performs concurrent modification on the rendered first three-dimensional scene model or the rendered first three-dimensional scene model again by the labeling information, namely a plurality of designers in the design end modify source data files corresponding to the modules according to own design copies, and finally the design end submits all the modified source data files to the server end which uniformly sends the source data files to the client so as to realize remote delivery.

In addition, referring to fig. 10, an embodiment of the present application further provides a three-dimensional model transmission apparatus, which includes a design end 100, a client 300, and a server end 200.

The design end 100 includes a first display module 101, a first three-dimensional rendering module 102, a first audio/video processing module 103, a data storage module 104, and an agent integration interface 105. The first display module 101 may be configured to display the rendered first three-dimensional scene model or the third three-dimensional scene model, or display the rendered first three-dimensional scene model or the rendered third three-dimensional scene model, where the display module includes a computer display screen or an AR screen; the first three-dimensional rendering module 102 may be configured to perform three-dimensional rendering processing on the first three-dimensional scene model or the third three-dimensional scene model to obtain a rendered first three-dimensional scene model or a rendered third three-dimensional scene model, or may be configured to perform re-three-dimensional rendering processing on the rendered first three-dimensional scene model or the rendered third three-dimensional scene model to obtain a re-rendered first three-dimensional scene model or a re-rendered third three-dimensional scene model; the first audio/video processing module 103 may be configured to receive a video stream of the augmented reality video call sent by the seat integration interface 105, perform frame extraction processing on the video stream of the augmented reality video call to obtain a plurality of video images, and may also be configured to receive a first three-dimensional scene model or a third three-dimensional scene model from the seat integration interface 105, or receive a rendered first three-dimensional scene model or a rendered third three-dimensional scene model from the seat integration interface 105; the data storage module 104 may be configured to store a video stream, annotation information, model-related data, a first three-dimensional scene model, a third three-dimensional scene model, a rendered first three-dimensional scene model, a rendered third three-dimensional scene model, and the like of the augmented reality video call from the agent integration interface 105; the agent integration interface 105 may be configured to obtain a plurality of video images in the first audio/video processing module 103, send the plurality of video images to the server 200, and may also obtain a first three-dimensional scene model, a third three-dimensional scene model, a rendered first three-dimensional scene model, a rendered third three-dimensional scene model, and so on of the server 200.

In one possible implementation, the first audio/video processing module 103 may support an audio/video encoding format; the data storage module 104 may be invoked; the video meeting invitations may be ordered according to priority, and either the video responses or the audio responses may be selected for the video meeting invitations; simultaneously supporting a media negotiation mode of multiple streams; the first three-dimensional scene model or the third three-dimensional scene model may be transmitted to the first three-dimensional rendering module 102.

In a possible embodiment, the agent integration interface 105 may be configured to receive an access request of the message processing module 203 of the server side 200, parse the access request, and obtain related information in the first audio/video processing module 103, package the related information to obtain a data packet, and send the data packet to the server side 200, which is not limited herein.

Server side 200 includes file store 201, message processing module 203, message caching module 204, data flow module 205, image parsing module 209, model adaptation module 210, three-dimensional model constructor 211, model storage module 208, and second three-dimensional rendering module 206. Wherein the file memory 201 may be used to store a plurality of video images from the agent integration interface 105; the message processing module 203 may be configured to authenticate the video conferencing invitation; the message buffer module 204 may be configured to receive data such as user information and an agent address of the message processing module 203; the data flow module 205 may be configured to transmit a video flow of the augmented reality video call from the agent integration interface 105 of the design side 100, and may be configured to send a video meeting invitation and exit the augmented reality video call; the image analysis module 209 may be configured to perform analysis processing on the physical image set to obtain graphic information, and perform analysis processing on the environmental image set to obtain tag information; the model adapting module 210 may be configured to construct a physical image base map by using the physical image element information, and perform modeling processing on the physical image base map to obtain a physical three-dimensional model; model storage module 208 may be used to obtain environmental image element information from an environmental model library based on the tag information, and may also be used to obtain physical image element information from a physical model library based on the graphic information; the three-dimensional model constructor 211 may be configured to perform modeling processing on the environmental image set by using the environmental image element information to obtain an environmental three-dimensional model; the second three-dimensional rendering module 206 may be configured to perform three-dimensional rendering processing on the first three-dimensional scene model or the third three-dimensional scene model from the three-dimensional model constructor 211, to obtain a rendered first three-dimensional scene model or third three-dimensional scene model.

In a possible implementation, the file storage 201 may further receive a data packet from the agent integration interface 105 through the upload/download module 202, and store the data packet in a specified directory, where the data packet includes a three-dimensional model information file.

In a possible implementation manner, the server 200 further includes an upload/download module 202, which may be used to detect the number of directory layers according to the directory of the file stored in the file storage 201, detect whether the directory of the server 200 exists, detect whether the name or format of the file is correct, and detect the validity of the token. After confirming the error, a URL (Uniform Resource Locator ) address is generated and sent to the design side 100 and the client side 300.

In one possible implementation, the message processing module 203 may forward the account login request; the query operation initiated by the client 300 can be distributed to other modules, and messages can be sent to other modules; and when the number of the processed request messages is excessive, a part of user information, seat addresses and other data can be stored in the message buffer module 204.

In a possible implementation, the image parsing module 209 may also apply an algorithm in terms of image processing and an algorithm in terms of computer vision, and the image parsing module may also be used for image recognition and image segmentation, for example, an object with a red frame label from the design end 100 is processed by the image parsing module 209, and then parses out graphic information of the object, such as sub-model and coordinate information of the label of the object, and then notifies the client 300 to capture the object in the video stream through the message processing module 203 and the message buffering module 204.

In a possible implementation, the image parsing module 209 may obtain a plurality of video images from the file memory 201, and perform segmentation processing on the plurality of video images to obtain a set of physical images and a set of environmental images.

In one possible embodiment, the model storage module 208 includes a preset three-dimensional model library, which includes a physical model library and an environmental model library.

In an embodiment, after the design end 100 and the client 300 establish an augmented reality video call, the first audio/video processing module 103 of the design end 100 may obtain a video stream of the augmented reality video call from the agent integration interface 105 or a video stream of the augmented reality video call from the data storage module 104, and the first audio/video processing module 103 performs frame extraction processing on the video stream of the augmented reality video call to obtain a plurality of video images, and then the first audio/video processing module 103 sends the plurality of video images to the server end 200 through the agent integration interface 105.

In an embodiment, the file storage 201 may acquire a plurality of video images of the agent integration interface 105 of the design end 100 through the upload/download module 202, or may acquire a plurality of video images of the client integration interface 302 of the client 300 through the upload/download module 202, which is not limited herein.

In one embodiment, the file storage 201 stores a plurality of video images from the agent integration interface 105 of the design end 100 or stores a plurality of video images from the client integration interface 302 of the client 300, the image parsing module 209 may acquire a plurality of video images from the file storage 201 and perform segmentation processing on the plurality of video images to obtain a physical image set and an environment image set, and perform parsing processing on the physical image set to obtain graphic information, and perform parsing processing on the environment image set to obtain tag information, and send the graphic information and the tag information to the model adaptation module 210, and then the model adaptation module 210 acquires physical image element information from a physical model library in the model storage module 208 using the graphic information, and constructs a physical image base using the physical image element information, and performs modeling processing on the physical image base, obtaining a physical three-dimensional model, obtaining environmental image element information from an environmental model library in a model storage module 208 by a model adaptation module 210 by using label information, modeling an environmental image set by using the environmental image element information to obtain the environmental three-dimensional model, sending the physical three-dimensional model and the environmental three-dimensional model to a three-dimensional model constructor 211 by the model adaptation module 210, generating a first three-dimensional scene model by using the physical three-dimensional model and the environmental three-dimensional model by the three-dimensional model constructor 211, sending the first three-dimensional scene model to a second three-dimensional rendering module 206, performing three-dimensional rendering processing on the first three-dimensional scene model by the second three-dimensional rendering module 206 to obtain a rendered first three-dimensional scene model, sending the rendered first three-dimensional scene model to a video stream module by the second three-dimensional rendering module 206, the agent integration interface 105 is sent to the design end 100 through the video streaming module; alternatively, the three-dimensional model constructor 211 sends the first three-dimensional scene model to the file memory 201 through the upload/download module 202, and the file memory 201 stores the first three-dimensional scene model, or sends the first three-dimensional scene module to the agent integration interface 105, or sends the first three-dimensional scene module to the agent interface of the client 300.

In an embodiment, the agent integration interface 105 of the design end 100 may obtain the rendered first three-dimensional scene module from the video streaming module and send the first three-dimensional scene module to the data storage module 104, and the first audio/video processing module 103 may obtain the rendered first three-dimensional scene module from the data storage module 104, where the first audio/video processing module 103 directly sends the rendered first three-dimensional scene module to the first display module 101, and the rendered first three-dimensional scene model is displayed by the first display module 101, which is not limited herein.

In another embodiment, the agent integration interface 105 of the design end 100 may obtain the rendered first three-dimensional scene module from the data storage module 104, the first audio/video processing module 103 may obtain the rendered first three-dimensional scene module from the data storage module 104, send the first three-dimensional scene module to the first three-dimensional rendering module 102, the first three-dimensional rendering module 102 performs three-dimensional rendering processing on the rendered first three-dimensional scene model again, and finally the first display module 101 receives the re-rendered first three-dimensional scene model from the first three-dimensional rendering module 102, and displays the re-rendered first three-dimensional scene model without specific limitation.

In an embodiment, the agent integration interface 105 of the design end 100 may acquire a first three-dimensional scene module from the file memory 201 of the server end 200 through the upload/download module 202 of the server end 200, and send the first three-dimensional scene module to the first audio/video processing module 103, where the first audio/video processing module 103 sends the first three-dimensional scene module to the first three-dimensional rendering module 102, performs three-dimensional rendering processing on the first three-dimensional scene model through the first three-dimensional rendering module 102, and finally the first display module 101 receives the rendered first three-dimensional scene model from the first three-dimensional rendering module 102, and displays the rendered first three-dimensional scene model, which is not limited herein.

In another embodiment, the agent integration interface 105 of the design end 100 may acquire the first three-dimensional scene module from the file memory 201 of the server end 200 through the upload/download module 202 of the server end 200 and send the first three-dimensional scene module to the data storage module 104, the first audio/video processing module 103 acquires the first three-dimensional scene module from the data storage module 104, sends the first three-dimensional scene module to the first three-dimensional rendering module 102, performs three-dimensional rendering processing on the first three-dimensional scene model through the first three-dimensional rendering module 102, and finally the first display module 101 receives the rendered first three-dimensional scene model from the first three-dimensional rendering module 102 and displays the rendered first three-dimensional scene model, which is not limited herein.

In a possible implementation manner, the client 300 may click the cloud rendering button through Web software or directly in a 3D program of the local terminal, access resources via the high-speed internet, instruct the client to send out a command from the user terminal, execute a corresponding rendering task according to the command by the server, and transmit a rendering result picture back to the user terminal for display. The rendering capability of the far end is provided for the terminal equipment, so that the short rendering capability plate of the terminal can be compensated.

In a possible embodiment, the server 200 further includes an information base 207, where the information base 207 is a database for storing important information data of the server 200, in order to prevent the system exception from causing the data loss of user information, agent address, and the like in the message buffer module 204, the message buffer module 204 may store some important user information, agent address, and the like in the information base 207, and the information base 207 may be used to resume the interrupted service.

The client 300 includes a client integration interface 302, a camera acquisition module 303, a second display module 306, a third three-dimensional rendering module 305, and a second audio-video processing module 304. The client integration interface 302 may be configured to acquire a plurality of video images in the second audio/video processing module 304, and send the plurality of video images to the server 200; the second display module 306 may be configured to display the rendered first three-dimensional scene model or the third three-dimensional scene model, where the display module includes a computer display screen or an AR screen, and the like. The third three-dimensional rendering module 305 is configured to perform three-dimensional rendering processing on the first three-dimensional scene model or the third three-dimensional scene model from the server 200, to obtain a rendered first three-dimensional scene model or third three-dimensional scene model; the second audio/video processing module 304 may be configured to receive a video stream of the augmented reality video call sent by the client integration interface 302, and perform frame extraction processing on the video stream of the augmented reality video call to obtain a plurality of video images; the camera acquisition module 303 is configured to acquire a video stream of the augmented reality video call.

In a possible embodiment, the client 300 further includes a local storage module 301, where the local storage module 301 is configured to store, when uploading or downloading a file from the file storage 201 of the server 200, the file of the server 200 at a local address or upload the file to the server 200 according to the URL address sent by the uploading and downloading module 202, where a format of the file, such as a picture, may be converted into a binary file.

In a possible embodiment, the client integrated interface 302 may be configured to receive an access request of the message processing module 203 of the server 200, parse the access request, and obtain related information in the first audio/video processing module 103, package the related information to obtain a data packet, and send the data packet to the server 200, which is not limited herein.

In an embodiment, the local storage module 301 of the client 300 may download the first three-dimensional scene module from the file storage 201 of the server side 200 through the upload download module 202 of the server side 200.

In an embodiment, the camera capturing module 303 captures a video stream of the augmented reality video call and sends the video stream to the second audio/video processing module 304, the second audio/video processing module 304 performs frame extraction processing on the video stream of the augmented reality video call to obtain a plurality of video images, the plurality of video images are sent to the client integration interface 302, the client integration interface 302 sends the plurality of video images to the server 200, for example, the client integration interface 302 stores the plurality of video images in the local storage module 301, and the plurality of video images are sent to the file memory 201 of the server 200 through the upload/download module 202 of the server 200 through the local storage module 301.

In an embodiment, the client integration interface 302 of the client 300 may obtain the rendered first three-dimensional scene module from the video streaming module and send the first three-dimensional scene module to the second audio/video processing module 304, where the second audio/video processing module 304 directly sends the rendered first three-dimensional scene module to the second display module 306, and the rendered first three-dimensional scene model is displayed by the second display module 306, which is not limited herein.

In another embodiment, the client integration interface 302 of the client 300 may obtain the rendered first three-dimensional scene module from the video streaming module, send the first three-dimensional scene module to the second audio/video processing module 304, send the first three-dimensional scene module after rendering to the third three-dimensional rendering module 305 by the second audio/video processing module 304, perform three-dimensional rendering processing on the first three-dimensional scene model after rendering again by the third three-dimensional rendering module 305, and finally receive the first three-dimensional scene model after rendering again from the third three-dimensional rendering module 305 by the second display module 306, and display the first three-dimensional scene model after rendering again without specific limitation.

In an embodiment, the client integration interface 302 of the client 300 may obtain the first three-dimensional scene module from the local storage module 301 and send the first three-dimensional scene module to the second audio/video processing module 304, where the second audio/video processing module 304 sends the first three-dimensional scene module to the third three-dimensional rendering module 305, performs three-dimensional rendering processing on the first three-dimensional scene model through the third three-dimensional rendering module 305, and finally the second display module 306 receives the rendered first three-dimensional scene model from the third three-dimensional rendering module 305, and displays the rendered first three-dimensional scene model, which is not limited herein.

It should be noted that, in the embodiments of the present application, when related processing is required according to user information, user behavior data, user history data, user location information, and other data related to user identity or characteristics, permission or consent of the user is obtained first, and the collection, use, processing, and the like of the data comply with related laws and regulations and standards of related countries and regions. In addition, when the embodiment of the application needs to acquire the sensitive personal information of the user, the independent permission or independent consent of the user is acquired through popup or jump to a confirmation page and the like, and after the independent permission or independent consent of the user is definitely acquired, the necessary relevant data of the user for enabling the embodiment of the application to normally operate is acquired.

In addition, referring to fig. 11, an embodiment of the present application provides another three-dimensional model transmission apparatus 400 including a memory 402, a processor 401, and a computer program stored on the memory 402 and executable on the processor 401.

The processor 401 and the memory 402 may be connected by a bus or other means.

Memory 402 acts as a non-transitory computer readable storage medium that may be used to store non-transitory software programs as well as non-transitory computer executable programs. In addition, memory 402 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some implementations, the memory 402 may optionally include memory located remotely from the processor 401, which may be connected to the processor 401 over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The non-transitory software program and instructions required to implement the three-dimensional model transmission method of the above-described embodiment are stored in the memory 402, and when executed by the processor 401, the three-dimensional model transmission method of the above-described embodiment is performed, for example, the method steps S110 to S160 in fig. 1, the method steps S210 to S230 in fig. 2, the method steps S310 to S350 in fig. 3, the method steps S410 to S440 in fig. 4, the method steps S510 to S530 in fig. 5, the method steps S610 to S620 in fig. 6, the method steps S710 to S720 in fig. 7, and the method steps S810 to S820 in fig. 8 described above are performed.

The above described embodiments of the apparatus are only illustrative, wherein the units described as separate components may or may not be physically separate, i.e. may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

Furthermore, an embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions that are executed by a processor or controller, for example, by one of the processors in the above-described device embodiments, which may cause the processor to perform the three-dimensional model transmission method in the above-described embodiment, performing the above-described method steps S110 to S160 in fig. 1, the method steps S210 to S230 in fig. 2, the method steps S310 to S350 in fig. 3, the method steps S410 to S440 in fig. 4, the method steps S510 to S530 in fig. 5, the method steps S610 to S620 in fig. 6, the method steps S710 to S720 in fig. 7, and the method steps S810 to S820 in fig. 8.

Furthermore, an embodiment of the present application provides a computer program product including a computer program or computer instructions stored in a computer-readable storage medium, the computer program or computer instructions being read from the computer-readable storage medium by a processor of a computer device, the processor executing the computer program or computer instructions, causing the computer device to perform the three-dimensional model transmission method in the above embodiment, for example, performing the method steps S110 to S160 in fig. 1, the method steps S210 to S230 in fig. 2, the method steps S310 to S350 in fig. 3, the method steps S410 to S440 in fig. 4, the method steps S510 to S530 in fig. 5, the method steps S610 to S620 in fig. 6, the method steps S710 to S720 in fig. 7, and the method steps S810 to S820 in fig. 8 described above.

Those of ordinary skill in the art will appreciate that all or some of the steps, systems, and methods disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). The term computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as known to those skilled in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer. Furthermore, as is well known to those of ordinary skill in the art, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media.

Claims

1. A three-dimensional model transmission method, comprising:

acquiring a plurality of video images from a design end, wherein the video images are obtained by the design end through frame extraction processing of a video stream of an augmented reality video call, and the augmented reality video call is established by the design end and a client;

dividing a plurality of video images to obtain a real image set and an environment image set;

modeling the real object image set to obtain a real object three-dimensional model;

modeling the environment image set to obtain an environment three-dimensional model;

generating a first three-dimensional scene model according to the physical three-dimensional model and the environment three-dimensional model;

and sending the first three-dimensional scene model to the client.

2. The three-dimensional model transmission method according to claim 1, wherein the generating a first three-dimensional scene model from the physical three-dimensional model and the environmental three-dimensional model comprises:

acquiring a preset three-dimensional model library, wherein the three-dimensional model library comprises an environment model library;

determining a target environment three-dimensional model from the environment model library according to the physical three-dimensional model and the environment three-dimensional model;

And generating a first three-dimensional scene model according to the physical three-dimensional model and the target environment three-dimensional model.

3. The three-dimensional model transmission method according to claim 1, characterized in that after the transmitting the first three-dimensional scene model to the client, the three-dimensional model transmission method further comprises:

receiving the labeling information sent by the client;

determining an annotation region in the first three-dimensional scene model according to the annotation information;

nesting the annotation information on the annotation region to obtain a second three-dimensional scene model;

superposing the first three-dimensional scene model and the second three-dimensional scene model to obtain a third three-dimensional scene model;

and sending the third three-dimensional scene model to the client.

4. The three-dimensional model transmission method according to claim 2, wherein the three-dimensional model library comprises a physical model library, and the modeling processing is performed on the physical image set to obtain a physical three-dimensional model, comprising:

analyzing the real object image set to obtain graphic information;

acquiring the physical image element information from the physical model library according to the graphic information;

Constructing a physical image base map by utilizing the physical image element information;

and modeling the physical image base map to obtain a physical three-dimensional model.

5. The method for transmitting a three-dimensional model according to claim 2, wherein modeling the set of environmental images to obtain the three-dimensional model of the environment comprises:

analyzing the environment image set to obtain label information;

acquiring environmental image element information from the environmental model library according to the label information;

and modeling the environment image set by using the environment image element information to obtain an environment three-dimensional model.

6. The three-dimensional model transmission method according to claim 2, characterized in that the three-dimensional model transmission method further comprises:

determining a target environment three-dimensional model from the environment model library according to the physical three-dimensional model;

and performing nesting processing on the physical three-dimensional model and the target environment three-dimensional model to generate the first three-dimensional scene model.

7. The three-dimensional model transmission method according to claim 1, wherein the generating a first three-dimensional scene model from the physical three-dimensional model and the environmental three-dimensional model comprises:

And performing nesting processing on the physical three-dimensional model and the environment three-dimensional model to generate the first three-dimensional scene model.

8. A three-dimensional model transmission method, comprising:

establishing an augmented reality video call with a design end;

receiving a first three-dimensional scene model sent by a server side, wherein the first three-dimensional scene model is obtained by the server side according to a physical three-dimensional model and an environmental three-dimensional model, the physical three-dimensional model is obtained by the server side through modeling processing of a physical image set, the environmental three-dimensional model is obtained by the server side through modeling processing of an environmental image set, the physical image set and the environmental image set are both obtained by the server side through segmentation processing of a plurality of video images, and the video images are obtained by the design side through frame extraction processing of a video stream of the augmented reality video call.

9. The three-dimensional model transmission method according to claim 8, characterized in that the three-dimensional model transmission method further comprises:

performing virtual labeling processing on the first three-dimensional scene model to obtain labeling information;

and sending the labeling information to the server side.

10. A three-dimensional model transmission device, comprising: memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the three-dimensional model transmission method according to any one of claims 1 to 9 when executing the computer program.

11. A computer-readable storage medium storing computer-executable instructions for performing the three-dimensional model transmission method according to any one of claims 1 to 9.

12. A computer program product comprising a computer program or computer instructions, characterized in that the computer program or the computer instructions are stored in a computer readable storage medium, from which the computer program or the computer instructions are read by a processor of a computer device, which processor executes the computer program or the computer instructions, so that the computer device performs the three-dimensional model transmission method according to any one of claims 1 to 9.