TW202324064A - Method and system for rapid conversion of style for a three-dimensional space - Google Patents

Abstract

A method and a system for rapid conversion of style for a three-dimensional space are provided. The system includes a storage medium used for storing multiple style files respective to multiple styles. Every style file includes a style image file, a three-dimensional space file, and a three-dimensional object files corresponding to a specific scene. The system also provides a three-dimensional space browser operated in a user device. A style menu having multiple style options is shown on the three-dimensional spatial browser. The system firstly provides a first style file of a first style to be loaded to a memory of the user device so as to display a first three-dimensional space image with respect to the scene when applying the first style. After that, a second style file is rapidly loaded to the memory of the user device after a second style is selected through the style menu. The scene is then switched to a second three-dimensional space image when applying the second style.

Description

Three-dimensional space style fast conversion method and system

The specification discloses a technology for three-dimensional space conversion, especially a method and system for quickly converting three-dimensional space styles after a user selects a style.

With the gradual maturity of stereo image formation and stereo model modeling technology, many ways of using three-dimensional stereo images to present objects have been developed. For example, a camera can be used to capture an image of a scene, and the space can be scanned using stereo modeling software. A three-dimensional image can be formed.

Furthermore, after the 3D space model of a scene is established, the image that has been prepared for this scene can be used for texture mapping. For example, after deconstructing the 3D space model, the area, angle and vertex of each azimuth angle can be obtained, A corresponding image is generated according to each block, and each block image also has three-dimensional spatial coordinate information, and is stored as a file. When generating a 3D space image later, select a 3D space model, perform a pixel-by-pixel mapping of the corresponding block image in a three-dimensional space, and map the texture to generate a 3D space image.

In order to provide convenient browsing of 3D space images of various styles, the specification discloses a method and system for quickly converting 3D space styles. The system includes a storage medium in which multiple style files corresponding to multiple styles are stored. Style image files and 3D space files, the system provides a 3D space browser running on the user's device, and a style menu is displayed in the 3D space browser, which lists multiple style options for quick conversion of the 3D space in the scene style.

In this 3D space style fast conversion system, the 3D space style fast conversion method is executed. The method includes providing a corresponding first style file for the first style among multiple styles, loading the corresponding first style image file and the first 3D space The file is stored in the memory of the user's device to display the first three-dimensional space image of the scene with the first style applied. Afterwards, the style menu can be displayed in the three-dimensional space browser, and the corresponding second style file can be provided according to the user's selection of the second style option in the style menu, and the corresponding second style image file and the second three-dimensional space file can be loaded into In the memory of the user device, the second three-dimensional space image of the scene applying the second style is switched and displayed.

Wherein, each style image file in the plurality of style image files records a style image file applied to the scene after rendering, light and space settings, and each three-dimensional space file records a three-dimensional space model obtained by scanning the scene with a space scanner, The 3D spatial model is used to describe the spatial information of the scene.

Further, each style file for each style also includes a 3D object file, and the 3D object file records the 3D model and spatial coordinates of one or more objects in the scene.

In one embodiment, the plurality of style image files, the plurality of 3D space files and the plurality of 3D object files are loaded into the memory of the user's device in advance, when the user selects any style from the style menu, immediately Apply to the scene and display the corresponding 3D space image.

In another embodiment, the system may provide a server, and the storage medium is implemented as a database of the server, wherein a plurality of style image files, a plurality of 3D space files and a plurality of 3D object files are stored.

In this way, the server can load any style selected by the user through the style menu, that is, load it into the memory of the user's device, apply it to the scene, and display the corresponding three-dimensional space image.

In order to further understand the features and technical content of the present invention, please refer to the following detailed description and drawings related to the present invention. However, the provided drawings are only for reference and description, and are not intended to limit the present invention.

The implementation of the present invention is described below through specific specific examples, and those skilled in the art can understand the advantages and effects of the present invention from the content disclosed in this specification. The present invention can be implemented or applied through other different specific embodiments, and various modifications and changes can be made to the details in this specification based on different viewpoints and applications without departing from the concept of the present invention. In addition, the drawings of the present invention are only for simple illustration, and are not drawn according to the actual size, which is stated in advance. The following embodiments will further describe the relevant technical content of the present invention in detail, but the disclosed content is not intended to limit the protection scope of the present invention.

It should be understood that although terms such as "first", "second", and "third" may be used herein to describe various elements or signals, these elements or signals should not be limited by these terms. These terms are mainly used to distinguish one component from another component, or one signal from another signal. In addition, the term "or" used herein may include any one or a combination of more of the associated listed items depending on the actual situation.

The disclosure book proposes a method and system for quickly converting three-dimensional space styles. The system is a computer system that provides end users with browsing and editing three-dimensional spaces. For fast-transition services, the system concept can refer to the system architecture embodiment diagram shown in Figure 1.

According to the structure embodiment diagram of the three-dimensional space style rapid conversion system shown in FIG. 1, the system is realized by a computer system and a network environment. The computer system can be implemented not only as a computer host, but also as a server 107 in the figure. There are circuit elements such as processor, memory and storage device, and related software programs for operating the server 107 . The system provides a storage medium. In addition to the storage device in the host computer, the database 109 can be implemented for the server 107. In addition to the necessary data, the storage medium mainly stores multiple style files corresponding to multiple styles in specific scenes. , each style file may include a style image file 110 and a three-dimensional space file 111 .

Among the plurality of style image files 110, each style image file records a style image file applied to a specific scene after rendering, light and space settings, and each 3D space file records a 3D space model obtained by scanning the scene with a space scanner, The 3D space model is used to describe the spatial information of the scene, such as a space formed by walls, ceilings, and floors of a space, and each part of the scene has depth information scanned by a space scanner. The system can present the three-dimensional space image of the scene through the style image file and the three-dimensional space file.

In an embodiment, the style file may further include a 3D object file 112 for each style. The 3D object file is mainly an object file stored in a three-dimensional format, which records the 3D model and spatial coordinates of one or more objects in the scene.

The above several files can be stored in the database 109 after being compressed and encrypted. The server 107 provides the service of fast conversion of the three-dimensional space style to various terminal devices through the network 10, such as the user devices 101, 103 and 105 in the figure.

According to an embodiment, the system provides a three-dimensional space browser executed in each user device. When running the three-dimensional space style fast switching method, the user can operate the three-dimensional space browser to select the style to be switched. When one of the styles is selected, Loading the corresponding style file means obtaining at least a three-dimensional space file and a style image file.

In the user device, the default browsing environment can be loaded first, that is, the three-dimensional space file 111 and the style image file 110 selected or preset by the user can be loaded from the server 107 (or computer host), and the same can also include 3D object file 112 . The file can be compressed and encrypted and sent to the user device 101, 103 or 105. After being decompressed by the browser, the 3D space image is presented in a specific format according to the 3D space model and style image. An image of a preset style can be displayed first, and then the user can select a style option provided by the server 107 through a menu.

In particular, in the storage medium provided by the system, such as the database 109 on the server 107 side, or the storage device in the host computer, there is stored a style image file 110, which is a three-dimensional computer rendering technology used to paste maps on the three-dimensional space model. Various styles of image data are formed, and after rendering, lighting design and space setting, a style image file that is applied to the 3D space model in a specific scene is generated. When applied to a 3D space model of a specific scene, the 3D computer graphics technology used in it uses a computer system to realize the processes of modeling, texture mapping, lighting, and rendering.

FIG. 2 then shows a schematic diagram of an embodiment of a functional unit for forming a 3D space image in the 3D space style fast conversion system.

The system shown in the figure is mainly equipped with a computer system 200, which includes various functional elements implemented by software or matching hardware (such as processors, memories, storage devices, and circuits, etc.), and at least has an operation for executing intelligent algorithms Unit 201, an editing unit 203 that provides an editing interface, a three-dimensional modeling unit 208 that generates a three-dimensional space model, and an integration unit 207 that integrates a panorama and a three-dimensional space model. The style production unit 205 of the style file, and the floor editing unit 209 that can provide users with editing three-dimensional space images of different floors, and the relevant software functions provide a browsing unit 211 for viewing images in real time when the user performs editing, that is, the three-dimensional images provided by the above-mentioned system. Spatial browser.

One of the methods for forming a three-dimensional space image in the three-dimensional space style rapid conversion system is a method of converting a two-dimensional image into a three-dimensional space image. In the figure, the computer system 200 is connected to a photography device 20 for one or more images in one scene. Take panoramas on a site. According to the embodiment, the photographing device 20 in the system is a camera, and the camera shoots the scene and generates a corresponding panorama at each site through a software splicing method, or drives the camera to shoot the scene with a rotation instrument to generate a corresponding panorama at each site The panorama, or a panorama camera, can be realized by software or hardware to shoot scenes and generate corresponding panoramas at each site. The obtained images can be sent to the computer system 200 for processing.

The system also has a space scanner 28 connected to the computer system 200. The system scans the scene through the space scanner 28, and can generate one or more three-dimensional space models describing the depth information of the scene at one or more stations. According to the embodiment, the spatial scanner 28 can obtain the depth information in the space through various technologies, such as time of flight (TOF) technology, laser ranging technology, or various optical ranging methods. After the space scanner 28 obtains the depth information in the scene, it is sent to the computer system 200. The mathematical method can perform modeling according to the depth information in the scene. For example, the point cloud (point cloud) technology can be used to establish the scene space. 3D space model.

The computer system 200 performs the task of converting a two-dimensional image into a three-dimensional space image. According to the embodiment, the computer system 200 can be a host computer, or a server system running on the cloud, wherein the computing power can be processed in real time from the scene captured by the camera device 20. One or more panoramas, and one or more three-dimensional space models obtained by the space scanner 28 scanning the scene, the number of panoramas and three-dimensional space models depends on the size and complexity of the scene.

The computer system 200 can distinguish multiple functional units according to functions, as shown in the schematic diagram of FIG. 2 , but the actual implementation is not limited to the illustrated example. The main components include a computing unit 201. The computing unit 201 represents the computing power established by the computing circuit, memory and storage in the computer system 200, and executes the intelligent algorithm required to convert the two-dimensional image into a three-dimensional space image. According to the embodiment, A deep learning method of DuLa-Net (Dual-Projection Network) and a deep learning method of ResNet (Deep Residual Network) can be used.

The computer system 200 is provided with a three-dimensional modeling unit 208, which runs a three-dimensional modeling algorithm. According to the method proposed in the disclosure, when the system obtains one or more three-dimensional space models corresponding to the panorama through the space scanner 28, it can use the three-dimensional The modeling unit 208 obtains one or more 3D spatial images.

The integration unit 207 is a software means in the computer system 200. When the system uses the photography equipment 20 to capture the panorama at one or more sites on the shooting moving line designed in the scene, and obtain the corresponding one or more three-dimensional spaces When modeling, the space and site information can be integrated through the integration unit 207, and the 3D space scale and object recognition and positioning algorithm can be applied to make the panorama and 3D space model obtain one or more 3D space images through intelligent algorithms.

The editing unit 203 is a software means, which provides an editing interface through an application program, especially implemented by a graphical user interface. When the system obtains the panorama and the 3D space model, and uses an intelligent algorithm to obtain one or more 3D space images, then the editing unit 203 can be used to start the editing interface to display multiple 3D space images in the scene through a graphical user interface and numbering, providing users with editing and splicing of overlapping parts of adjacent 3D space images to form a total 3D space image of the scene.

For the concept of the method of converting 2D images into 3D images applied to fast conversion of 3D space style, please refer to FIG. 3 and FIG. 4 , which show a schematic diagram of an embodiment showing a stereoscopic image of a field formed by splicing multiple plane panoramas.

Figure 3 shows the planar image before being converted into a three-dimensional space image. This example shows the panoramas of different spaces in an indoor scene. The illustration shows the first panorama 301 and the second panorama shot by cameras in different spaces Figure 302, the third panorama 303, the fourth panorama 304, the fifth panorama 305, and the sixth panorama 306, which show planar images spliced according to the connections between different spaces.

According to the embodiment, one of the ways to obtain these panoramas is to use a camera to shoot images in a space in batches, which can be assisted by software programs to shoot images covering all positions (total 720 degrees) in each space, and then through the stitching method Generate panoramas of each space; or a camera can be driven by a rotation instrument to capture images of each space in the scene to generate corresponding panoramas; or further, a panoramic camera can be used to capture images of each space to generate corresponding panoramas.

Afterwards, Figure 4 shows a schematic diagram of converting multiple panoramas (301, 302, 303, 304, 305, 306) in the above scene into multiple three-dimensional space images through intelligent algorithms. Realize space recognition and object recognition, and then calculate the correlation between adjacent spaces, for example, establish the correlation between spaces based on shared information such as doors, windows, floors, ceilings, colors, and sizes, or add user-edited information. According to the connection relationship of adjacent spaces, different three-dimensional space images are stitched together to form the first perspective view 401, the second perspective view 402, the third perspective view 403, the fourth perspective view 404, the fifth perspective view 405 and the sixth perspective view 406 to form a corresponding combination The total 3D spatial image of the scene.

A three-dimensional space file corresponding to a specific scene can be established through the above method of diagramming and forming a three-dimensional space image. In addition, it can generate 3D object files for specific objects, such as interior furnishings, furniture, doors and windows, etc., and then create style images applying 3D space models according to various styles, and finally form style files. For related embodiments, please refer to the flow chart of an embodiment of a method for creating image files (style image files 110 , 3D space files 111 , and 3D object files 112 ) for each scene shown in FIG. 5 .

Referring to the process of the above-mentioned embodiment, a panorama covering each space in a specific scene is captured by a photographic device (step S501), and then the computing unit in the computer system executes an intelligent algorithm to identify the space covered in the panorama and create a three-dimensional image. The module unit establishes a three-dimensional space model according to the above process (step S503), and then provides space editing tools (step S505), allowing the user to operate the panorama on the editing interface or the three-dimensional space image on the graphical user interface, according to the scene The correlation between multiple spaces integrates the structure of all panoramas in a single scene, and uses intelligent algorithms to identify objects (such as various objects, structures, colors, etc.) in the scene to generate 3D object files in the scene (step S507) .

Then, the above three-dimensional space file and three-dimensional object file are stored in the storage medium provided by the system, so that the user can operate the three-dimensional space browser in the user device to perform editing, previewing, and downloading. Afterwards, according to different styles, you can use the style production unit to design and apply the style image in the scene, and execute the preview, and store it in the storage device provided by the system after completion, such as the local storage device or the server-side database (step S509 ), and then through rendering, lighting and space settings, an image file is generated, and a plurality of style image files are generated (step S511 ). The style image can be different styles of decoration and color system designed for a specific scene, and includes a variety of styles of style image files for users to choose one of them, which can achieve fast (such as one-click change) conversion style effect.

For multi-floor scenes, the provided three-dimensional modeling method splices the three-dimensional space objects of each floor in a modular manner, which greatly reduces the complexity of multi-floor and the modeling time of the whole building. The software means of the floor editing unit shown in Figure 2 209. For the process, refer to the flow chart of the embodiment shown in FIG. 6 for creating a floor image file.

According to the process of the above embodiment, the three-dimensional modeling of a plane scene is completed, as well as the object identification and modeling in it, and the three-dimensional space files and three-dimensional object files of each floor scene are generated. In the process of editing multi-storey floors, first import the three-dimensional space files of each floor. With the 3D object file (step S601), the user can manually or through the intelligent algorithm recognition technology, use the editing interface to edit the floor order (step S603), and then use software to combine the spatial images and objects of each floor (step S605), and finally Store and form a floor space file (step S607 ), which becomes a part of the three-dimensional space file.

According to the embodiment of the three-dimensional space style fast conversion method, similarly, the style can be produced for each floor, and the software means can use the style production unit 205 shown in Figure 2. The rendering technology pastes maps on the 3D space models of each floor to form various style image data. After rendering, light design and space setting, the same style image files are generated to be applied to the 3D space models of each floor.

For the flow of the method for quickly converting the three-dimensional space style, please refer to the flow chart of the embodiment shown in FIG. 7 . The method is run on a computer system, such as a server as shown in FIG. 1 , or a host computer.

Initially, for a specific scene, the system can provide multiple style files corresponding to multiple styles, and each style file includes at least a style image file and a 3D space file, and may also include a 3D object file (step S701 ).

In one embodiment, the user device executes a three-dimensional image browser, which displays a menu of styles created by various designers, and the system can be loaded into the memory of the user device according to the user's selection, or can be previewed Pre-loaded into the memory of each user device, so that the user can decide the style of browsing according to his preferences, whether it is loaded from the server to the user device according to demand, or pre-loaded into the memory of the user device body, can achieve the purpose of fast switching, especially the effect of one-key changing.

At the beginning, the system can provide the corresponding first style file for the first style among the multiple styles prepared by the system according to the user's pre-selection, or the system provides the default style, and load the corresponding first style image The file and the first 3D space file are stored in the memory of the user device to display the first 3D space image of the scene with the first style applied. It can also include 3D object files for importing various 3D object images in the scene. What is mentioned here is that the 3D object files mainly record the 3D models of one or more objects in the scene. and space coordinates, so after selection, the object can be applied to the corresponding position on the 3D space model.

The user can use the 3D space browser running in the user device to initially browse the first style applied to the scene provided by the system, and the system also provides a style menu through the 3D space browser (step S703), wherein the style menu can list the system The style options that have been completed, such as the style options that have been stored in the server database, can provide a preview mechanism, which is convenient for the user to confirm before executing the application. The user operates the user interface of the three-dimensional space browser to select one of the style options (step S705), so that the system will provide the corresponding second style file according to the second style option in the style menu (step S707), and then The corresponding second style image file and the second 3D space file are loaded into the memory of the user device, so as to switch and display the second 3D space image of the scene applying the second style.

The effect of the fast conversion of the three-dimensional space style can refer to the schematic diagrams of the embodiment shown in FIG. 8A and FIG. 8B .

Both Fig. 8A and Fig. 8B show that the style menu 800 is provided by the three-dimensional space browser 80, wherein Fig. 8A shows that the three-dimensional space image with the style one 81 is being browsed by default or selected by the user, and the user can continue to use the style menu 800 to select Style 2 82, once selected, the corresponding three-dimensional space file, three-dimensional object file and style image file can be loaded from the storage medium into the memory of the user device, wherein the standardized decompression and decryption can be performed first The program unpacks the style file to obtain and display a three-dimensional space image with style 2 82 as shown in FIG. 8B .

According to the application method of the three-dimensional space style rapid conversion method, for example, it can be applied to the websites of decoration, renting and selling houses. The manufacturer provides multiple sets of style files through the three-dimensional space style rapid conversion system, so that users can browse house objects. , you can quickly switch between various styles provided by the system, which is convenient for users to preview and select.

To sum up, according to the above-mentioned implementation of the 3D space style quick conversion method, by creating multiple 3D space files, 3D object files and style image files, the user can select the 3D space file that he wants to apply a specific style to in a specific scene. For 3D space images, since the style files of different styles have been established in the same scene in advance, the style can be quickly switched in the 3D space browser.

The content disclosed above is only a preferred feasible embodiment of the present invention, and does not therefore limit the scope of the patent application of the present invention. Therefore, all equivalent technical changes made by using the description and drawings of the present invention are included in the application of the present invention. within the scope of the patent.

10: Internet 107: Server 109: Database 110: Style Image File 111: 3D Space Archives 112:3D object file 101, 103, 105: user device 200: Computer system 201: Operation unit 203: Edit unit 205: Style making unit 207: Integration unit 208: Stereo Modeling Unit 209: Floor editing unit 211: Browsing unit 20: Photographic equipment 28: Spatial Scanner 301: The first panorama 302: Second panorama 303: The third panorama 304: The fourth panorama 305: The fifth panorama 306: The sixth panorama 401: The first stereogram 402: The second stereogram 403: The third stereogram 404: The fourth stereogram 405: The fifth stereogram 406: The sixth stereogram 80: 3D space browser 800: Style menu 81: style one 82: style two Steps S501-S511: the process of creating an image file for each scene Steps S601-S607: the process of creating floor image files Steps S701-S707: three-dimensional space style rapid conversion process

Fig. 1 shows the framework embodiment figure of three-dimensional space style rapid conversion system;

Fig. 2 shows a schematic diagram of a system embodiment of a method for converting a two-dimensional image into a three-dimensional image;

Figure 3 shows a schematic plan view of stitching multiple panoramas to form a field;

FIG. 4 shows a schematic diagram of a stereoscopic image of a field formed after the panorama is converted;

Fig. 5 shows the flow chart of the method embodiment of establishing the image file of each scene;

Fig. 6 shows the flow chart of the method embodiment of establishing floor image file;

Fig. 7 shows the flow chart of an embodiment of the three-dimensional space style fast conversion method; and

FIG. 8A and FIG. 8B show a schematic diagram of an embodiment of fast conversion of a three-dimensional space style.

10: Internet

107: Server

109: Database

110: Style Archives

111: 3D Space Archives

112:3D object file

101, 103, 105: user devices

Claims (10)

A three-dimensional space style fast conversion method, comprising: For a scene, provide multiple style files corresponding to multiple styles, each style file includes a style image file and a 3D space file; Provide a corresponding first style file for a first style among the plurality of styles, load a corresponding first style image file and a first three-dimensional space file into the memory of a user device to display the application a first 3D image of the scene in the first style; and Provide a style menu through the user device, provide a corresponding second style file according to selecting a second style option in the style menu, load a corresponding second style image file and a second three-dimensional space file into Switching and displaying a second three-dimensional image of the scene applying the second style in the memory of the user device; Wherein, each style image file in the plurality of style files records a style image file applied to the scene after rendering, light and space settings, and each 3D space file records a 3D space model obtained by scanning the scene with a space scanner, The 3D spatial model is used to describe the spatial information of the scene. The method for quickly converting 3D space styles as described in Claim 1, wherein each style file for each style further includes a 3D object file, and the 3D object file records the 3D models and spatial coordinates of one or more objects in the scene. The three-dimensional space style rapid conversion method as described in claim 2, wherein the multiple style image files, the multiple three-dimensional space files and the multiple three-dimensional object files are loaded into the memory of the user device in advance, when a The user selects any one of the styles through the style menu, and it is immediately applied to the scene, and the corresponding three-dimensional space image is displayed. The three-dimensional space style rapid conversion method as described in claim 2, wherein the plurality of style image files, the plurality of three-dimensional space files and the plurality of three-dimensional object files are stored in a server, and the server is passed by a user Any one of the styles selected from the style menu is loaded into the memory of the user device to be applied to the scene, and the corresponding three-dimensional space image is displayed. The three-dimensional space style rapid conversion method as described in any one of claims 1 to 4, wherein a three-dimensional space browser is executed on the user device, and the three-dimensional space browser displays the style menu, in which a plurality of The style option is used to quickly convert the style of the 3D space in the scene. A three-dimensional space style fast conversion system, including: a storage medium, the storage medium stores a plurality of style files corresponding to a plurality of styles, and each style file includes a style image file and a three-dimensional space file corresponding to a scene; and A 3D space browser running on a user device, displaying a style menu on the 3D space browser, in which a plurality of style options are listed for quickly changing the style of the 3D space in the scene; Wherein the three-dimensional space style fast conversion system operates a three-dimensional space style fast conversion method, including: Provide a corresponding first style file for a first style among the plurality of styles, and load a corresponding first style image file and a first three-dimensional space file into the memory of the user device to display and apply a first 3D image of the scene in the first style; and Through the style menu, a corresponding second style file is provided according to selecting a second style option in the style menu, and a corresponding second style image file and a second three-dimensional space file are loaded into the user device In the memory, a second three-dimensional space image of the scene applying the second style is switched to be displayed; Wherein, each style image file in the plurality of style files records a style image file applied to the scene after rendering, light and space settings, and each 3D space file records a 3D space model obtained by scanning the scene with a space scanner, The 3D spatial model is used to describe the spatial information of the scene. The 3D space style rapid conversion system as described in Claim 6, wherein each style file for each style further includes a 3D object file, and the 3D object file records the 3D model and spatial coordinates of one or more objects in the scene. The 3D space style rapid conversion system as described in claim item 7, wherein the multiple style image files, the multiple 3D space files and the multiple 3D object files are loaded into the memory of the user device in advance, when a The user selects any one of the styles through the style menu, and it is immediately applied to the scene, and the corresponding three-dimensional space image is displayed. The three-dimensional space style rapid conversion system as described in claim item 7, wherein a server is provided, and the storage medium is implemented as a database of the server, wherein the plurality of style image files, the plurality of three-dimensional space files and the Multiple 3D object files. The three-dimensional space style rapid conversion system as described in claim 9, wherein the server loads any style selected by a user through the style menu into the memory of the user's device for applying to the scene , and display the corresponding three-dimensional space image.

Images