CN116958408A

CN116958408A - Three-dimensional scene construction method, device, equipment, medium and program product

Info

Publication number: CN116958408A
Application number: CN202310341982.XA
Authority: CN
Inventors: 陈学霖; 李威宇; 陈宝权
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2023-03-24
Filing date: 2023-03-24
Publication date: 2023-10-27

Abstract

The application discloses a three-dimensional scene construction method, a device, equipment, a medium and a program product, and relates to the technical field of computers. The method comprises the following steps: acquiring a sample scene and generating a mapping field, wherein the mapping size of the generated mapping field is the same as the scene size of the sample scene; disturbing a plurality of mapping blocks in the generated mapping field to obtain a disturbed generated mapping field; mapping the sample scene to the disturbed generated mapping field to obtain a generated mapping scene; and carrying out coordinate assignment on the disturbed generated mapping field based on the voxel block association relation between the disturbed three-dimensional voxel block in the generated mapping scene and the three-dimensional voxel block in the sample scene, and obtaining the target scene. Coordinate data corresponding to each three-dimensional voxel block in the generated mapping field can be determined more robustly, and then the generated mapping field after random disturbance is updated, so that a target scene with high sense of reality is generated. The method can be applied to various scenes such as cloud technology, artificial intelligence, intelligent traffic and the like.

Description

Three-dimensional scene construction method, device, equipment, medium and program product

Technical Field

The embodiment of the application relates to the technical field of computers, in particular to a three-dimensional scene construction method, a device, equipment, a medium and a program product.

Background

With the rapid development of computer technology, the need to construct three-dimensional scenes is applied to more and more fields, such as: in the field of games, virtual reality, building design and the like, in the construction of a three-dimensional scene, various scene elements are beneficial to bringing rich visual effects to users, and detailed features of the three-dimensional scene are vividly displayed.

In the related art, a learning-based method is generally selected to learn a large number of scene element samples of the same type, and determine a scene element generation rule corresponding to the scene element of the type, so that the scene element of the type can be generated by using the scene element generation rule.

However, in the learning-based method, a large number of scene element samples of the same type are strongly relied on, so that the computing resource consumption of the learning process is large, only single type scene elements can be generated in the application process, and the current scene construction method is difficult to generate three-dimensional scenes with high quality geometric and realistic appearances in a diversified manner. So that a great deal of manual construction of three-dimensional scenes is still required at present.

Disclosure of Invention

The embodiment of the application provides a three-dimensional scene construction method, a device, equipment, a medium and a program product, which can more robustly determine coordinate data corresponding to each three-dimensional voxel block in a generated mapping field, further update the generated mapping field after random disturbance, smooth the local appearance and geometric characteristics of each three-dimensional voxel block and generate a target scene with high sense of reality. The technical scheme is as follows.

In one aspect, a three-dimensional scene construction method is provided, the method comprising:

acquiring a sample scene, wherein the sample scene is a three-dimensional scene consisting of a plurality of three-dimensional voxel blocks;

acquiring a generated mapping field, wherein the generated mapping field consists of a plurality of mapping blocks, the plurality of mapping blocks respectively store coordinate data, and the mapping size of the generated mapping field is the same as the scene size of the sample scene;

disturbing the plurality of mapping blocks in the generated mapping field to obtain a disturbed generated mapping field, wherein the plurality of mapping blocks in the disturbed generated mapping field store disturbed coordinate data;

mapping the sample scene to the perturbed generated mapping field to obtain a generated mapping scene, wherein voxel attribute data of a perturbed three-dimensional voxel block is stored in the generated mapping scene, and mapped to perturbed coordinate data;

and carrying out coordinate assignment on a plurality of mapping blocks in the disturbed generated mapping field based on the voxel block association relation between the disturbed three-dimensional voxel blocks in the generated mapping scene and the three-dimensional voxel blocks in the sample scene, and obtaining a target scene, wherein the target scene is a three-dimensional scene obtained by transformation on the basis of the sample scene.

In another aspect, there is provided a three-dimensional scene construction apparatus, the apparatus including:

the system comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring a sample scene, and the sample scene is a three-dimensional scene;

the second acquisition module is used for acquiring a generated mapping field, the generated mapping field is composed of a plurality of mapping blocks, the plurality of mapping blocks respectively store coordinate data, and the mapping size of the generated mapping field is the same as the scene size of the sample scene;

the disturbance module is used for carrying out disturbance on the plurality of mapping blocks in the generated mapping field to obtain a disturbed generated mapping field, and the plurality of mapping blocks in the disturbed generated mapping field store disturbed coordinate data;

the mapping module is used for mapping the transformed sample scene to the perturbed generated mapping field to obtain a generated mapping scene, voxel attribute data of the perturbed three-dimensional voxel block is stored in the generated mapping scene, and the voxel attribute data of the perturbed three-dimensional voxel block is mapped to the perturbed coordinate data;

the generation module is used for carrying out coordinate assignment on a plurality of mapping blocks in the disturbed generated mapping field based on the voxel block association relation between the disturbed three-dimensional voxel blocks in the generated mapping scene and the three-dimensional voxel blocks in the sample scene, and obtaining a target scene, wherein the target scene is a three-dimensional scene obtained by transformation on the basis of the sample scene.

In another aspect, a computer device is provided, where the computer device includes a processor and a memory, where the memory stores at least one instruction, at least one program, a set of codes, or a set of instructions, where the at least one instruction, the at least one program, the set of codes, or the set of instructions are loaded and executed by the processor to implement the three-dimensional scene building method according to any one of the embodiments of the present application.

In another aspect, a computer readable storage medium is provided, where at least one instruction, at least one program, a set of codes, or a set of instructions is stored, where the at least one instruction, the at least one program, the set of codes, or the set of instructions are loaded and executed by a processor to implement a three-dimensional scene construction method according to any of the embodiments of the application described above.

In another aspect, a computer program product or computer program is provided, the computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the three-dimensional scene construction method according to any one of the above embodiments.

The technical scheme provided by the embodiment of the application has the beneficial effects that at least:

after a sample scene composed of a plurality of three-dimensional voxel blocks is obtained, a generated mapping field with the same mapping size as the scene size of the sample scene is obtained, and a plurality of mapping blocks in the generated mapping field are disturbed, so that a disturbed generated mapping field storing disturbed coordinate data is obtained; mapping the sample scene to a disturbed generated mapping field, combining and arranging three-dimensional voxel blocks in the sample scene according to the disturbed generated mapping field to obtain a generated mapping scene, and further carrying out coordinate assignment on a plurality of mapping blocks in the disturbed generated mapping field based on a voxel block association relationship between the disturbed three-dimensional voxel blocks in the generated mapping scene and the three-dimensional voxel blocks in the sample scene, so as to obtain a target scene after the sample scene is transformed. By the method, the coordinate data corresponding to each three-dimensional voxel block in the generated mapping field can be determined more robustly by means of the coordinate updating process, the generated mapping field after random disturbance is updated, the sample scene is subjected to scene transformation to obtain a target scene after the scene transformation, the local appearance and geometric characteristics of each three-dimensional voxel block are smoothed, the target scene with high reality can be generated, and the scene transformation process for generating various target scenes through a single sample scene can be realized.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a schematic illustration of an implementation environment provided by an exemplary embodiment of the present application;

FIG. 2 is a flow chart of a three-dimensional scene construction method provided by an exemplary embodiment of the present application;

FIG. 3 is a schematic representation of a two-dimensional cut plane for generating a mapping field provided by an exemplary embodiment of the present application;

FIG. 4 is a schematic diagram of an acquisition generation map scenario provided by an exemplary embodiment of the present application;

FIG. 5 is a flow chart of a three-dimensional scene construction method provided by another exemplary embodiment of the present application;

FIG. 6 is a flow chart of a three-dimensional scene construction method provided by a further exemplary embodiment of the present application;

FIG. 7 is a schematic diagram of a scene of an acquired sample provided by an exemplary embodiment of the present application;

FIG. 8 is a schematic diagram of a scene block extraction process provided by an exemplary embodiment of the application;

FIG. 9 is a schematic diagram of updated voxel property data obtained by weighted fusion, provided by an exemplary embodiment of the present application;

FIG. 10 is a schematic diagram of coordinate assignment provided by an exemplary embodiment of the present application;

FIG. 11 is a flow chart of a three-dimensional scene construction method provided by yet another exemplary embodiment of the present application;

FIG. 12 is a pyramid diagram of a sample scene constructing different scene resolutions provided by an exemplary embodiment of the present application;

FIG. 13 is a schematic diagram of a target scene obtained by applying a three-dimensional scene construction method according to an exemplary embodiment of the present application;

FIG. 14 is a schematic diagram of an updated generated mapping field at an intermediate stage of acquisition using a three-dimensional scene construction method according to an exemplary embodiment of the present application;

FIG. 15 is a schematic diagram of a three-dimensional scene construction method according to an exemplary embodiment of the present application;

FIG. 16 is a flowchart of a feature preprocessing process provided by an exemplary embodiment of the present application;

FIG. 17 is a schematic diagram of resolution of a next size scene provided by an exemplary embodiment of the application;

FIG. 18 is a schematic diagram of generating a target scene from a sample scene provided by an exemplary embodiment of the present application;

FIG. 19 is a schematic view of a three-dimensional scene construction method according to an exemplary embodiment of the application;

FIG. 20 is a schematic diagram of an application three-dimensional scene construction method provided by an exemplary embodiment of the present application;

FIG. 21 is a block diagram of a three-dimensional scene building apparatus provided in an exemplary embodiment of the application;

FIG. 22 is a block diagram of a three-dimensional scene building apparatus provided in accordance with yet another exemplary embodiment of the application;

FIG. 23 is a block diagram of a three-dimensional scene building apparatus according to yet another exemplary embodiment of the present application;

fig. 24 is a block diagram of a server according to an exemplary embodiment of the present application.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the present application more apparent, the embodiments of the present application will be described in further detail with reference to the accompanying drawings.

First, a brief description will be given of terms involved in the embodiments of the present application.

Artificial intelligence (Artificial Intelligence, AI): the system is a theory, a method, a technology and an application system which simulate, extend and extend human intelligence by using a digital computer or a machine controlled by the digital computer, sense environment, acquire knowledge and acquire an optimal result by using the knowledge. In other words, artificial intelligence is an integrated technology of computer science that attempts to understand the essence of intelligence and to produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence, i.e. research on design principles and implementation methods of various intelligent machines, enables the machines to have functions of sensing, reasoning and decision.

The artificial intelligence technology is a comprehensive subject, and relates to the technology with wide fields, namely the technology with a hardware level and the technology with a software level. Artificial intelligence infrastructure technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and other directions.

Machine Learning (ML): is a multi-domain interdisciplinary, and relates to a plurality of disciplines such as probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and the like. It is specially studied how a computer simulates or implements learning behavior of a human to acquire new knowledge or skills, and reorganizes existing knowledge structures to continuously improve own performance. Machine learning is the core of artificial intelligence, a fundamental approach to letting computers have intelligence, which is applied throughout various areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, induction learning, teaching learning, and the like.

In the related art, a learning-based method is generally selected to learn a large number of scene element samples of the same type, and obtain a scene element generation rule corresponding to the scene element of the type, so that the scene element of the type can be generated by using the scene element generation rule. For example: a large number of automobile samples (such as automobile sample a, automobile sample b, automobile sample c and the like) are learned to determine automobile element generation rules, so that automobile elements (such as automobile element A) can be generated based on the automobile element generation rules. However, in the learning-based method, a large number of samples of the same type are strongly relied on, so that not only is the consumption of computing resources in the learning process large, but also only a single type of scene element can be generated in the application process, namely: the method for constructing the scene cannot generate a three-dimensional scene with high quality and sense of reality, so that the construction effect of the three-dimensional scene is poor.

In the embodiment of the application, the coordinate data corresponding to each three-dimensional voxel block in the generated mapping field can be determined more robustly, the generated mapping field after random disturbance is updated, the appearance and geometric characteristics of the local part of each three-dimensional voxel block are smoothed, and the target scene with high sense of reality is generated. The three-dimensional scene construction method provided by the embodiment of the application can be applied to a plurality of fields and various scenes in the game field (such as a virtual terrain generation scene, a virtual city generation scene and the like), the film special effect production field (such as a virtual space generation scene, a virtual object synthesis scene and the like), the virtual reality field, the augmented reality field and the like, so that various three-dimensional scenes can be quickly and automatically generated, the development time of technical arts is greatly reduced, the requirement on the quantity of materials is reduced, the development iteration process is accelerated, and personalized visual experience can be brought to the use of objects. It is noted that the above scenarios and fields are merely illustrative examples, and embodiments of the present application are not limited thereto.

It should be noted that, the information (including but not limited to user equipment information, user personal information, etc.), data (including but not limited to data for analysis, stored data, presented data, etc.), and signals related to the present application are all authorized by the user or are fully authorized by the parties, and the collection, use, and processing of the related data is required to comply with the relevant laws and regulations and standards of the relevant countries and regions. For example, the contents of a sample scene, a three-dimensional voxel grid and the like related to the application are all acquired under the condition of full authorization.

Next, an implementation environment according to an embodiment of the present application will be described. The three-dimensional scene construction method provided by the embodiment of the application can be implemented by the independent execution of the terminal, the execution of the terminal and the server, or the data interaction between the terminal and the server, and the embodiment of the application is not limited to the above. Alternatively, a three-dimensional scene construction method is described by taking a terminal and a server as an example.

Next, an implementation environment according to an embodiment of the present application will be described, schematically, with reference to fig. 1, where a terminal 110 and a server 120 are involved, and the terminal 110 and the server 120 are connected through a communication network 130.

In some embodiments, the terminal 110 has installed therein an application program that obtains sample scene functionality. Illustratively, the terminal 110 acquires a plurality of two-dimensional images and restores a three-dimensional sample scene. Optionally, the terminal 110 is configured to send the sample scene to the server 120. Wherein the sample scene is a three-dimensional scene consisting of a plurality of three-dimensional voxel blocks.

In some embodiments, after receiving the sample scene, the server 120 obtains a generated mapping field based on a scene size of the sample scene, the generated mapping field is composed of a plurality of mapping blocks, the plurality of mapping blocks respectively store coordinate data, wherein a mapping size of the generated mapping field is the same as a scene size of the sample scene, and a mapping block size of the mapping blocks in the generated mapping field is greater than or equal to a voxel block size of the three-dimensional voxel blocks in the sample scene.

After obtaining the generated mapping field, the server 120 perturbs a plurality of mapping blocks in the generated mapping field to obtain a perturbed generated mapping field, where the perturbed coordinate data is stored in the plurality of mapping blocks in the perturbed generated mapping field, that is: perturbation to multiple map blocks will change the coordinate locations of the map blocks therein.

In addition, the server 120 maps the obtained sample scene onto the perturbed generated mapping field to obtain a generated mapping scene, the generated mapping scene stores voxel attribute data of the perturbed three-dimensional voxel block, and the voxel attribute data of the perturbed three-dimensional voxel block is mapped onto the perturbed coordinate data.

After obtaining the generated mapping scene, the server 120 performs coordinate assignment on a plurality of mapping blocks in the generated mapping field after disturbance and obtains the target scene based on the voxel block association relationship between the three-dimensional voxel blocks after disturbance in the generated mapping scene and the three-dimensional voxel blocks in the sample scene. The target scene is a three-dimensional scene obtained by transformation on the basis of the sample scene.

Optionally, after obtaining the target scene, the server 120 sends the target scene to the terminal 110, and the terminal 110 performs scene rendering on the target scene to display the target scene after performing scene reconstruction on the sample scene, so as to enrich the display form of the scene.

It should be noted that the above-mentioned terminals include, but are not limited to, mobile terminals such as mobile phones, tablet computers, portable laptop computers, intelligent voice interaction devices, intelligent home appliances, vehicle-mounted terminals, and the like, and may also be implemented as desktop computers and the like; the server may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), basic cloud computing services such as big data and artificial intelligence platforms.

Cloud technology (Cloud technology) refers to a hosting technology that unifies serial resources such as hardware, application programs, networks and the like in a wide area network or a local area network to realize calculation, storage, processing and sharing of data. The cloud technology is based on the general names of network technology, information technology, integration technology, management platform technology, application technology and the like applied by the cloud computing business mode, can form a resource pool, and is flexible and convenient as required.

In some embodiments, the servers described above may also be implemented as nodes in a blockchain system.

The three-dimensional scene construction method provided by the application is described by combining the noun introduction and the application scene, and the method is applied to a server for example, as shown in fig. 2, and the method comprises the following steps 210 to 250.

Step 210, a sample scene is acquired.

Wherein the sample scene is a three-dimensional scene consisting of a plurality of three-dimensional voxel blocks.

Optionally, the sample scene is a three-dimensional scene generated based on a plurality of two-dimensional images, the plurality of two-dimensional images being images acquired from different acquisition angles for a given scene.

Illustratively, the specified scene may be implemented as a real-world scene. For example: acquiring a plurality of two-dimensional images from different acquisition angles for the building A, acquiring a sample scene based on the plurality of two-dimensional images, and the like.

Illustratively, the specified scene may also be implemented as a digitally synthesized virtual scene. For example: the scene synthesis software stores a plurality of virtual elements, and combines partial virtual elements or all virtual elements based on a random generation mode to obtain a sample scene; or, combining part or all of the virtual elements in the scene synthesis software based on the software using the objects to obtain a sample scene and the like.

The three-dimensional voxel block is used to indicate voxels that constitute a sample scene, and the voxels are also called volume pixels or stereo pixels, which are units for measuring the resolution of the scene in three-dimensional space, similar to units for measuring the resolution of an image in two-dimensional space, pixels.

In an alternative embodiment, the scene resolution of the sample scene is measured by a three-dimensional voxel grid, i.e. the voxel grid resolution of the three-dimensional voxel grid is used to represent the scene resolution of the sample scene.

Illustratively, a three-dimensional voxel grid is a data structure that uses a voxel cube of a fixed size as a minimum unit to represent a three-dimensional scene, and the voxel grid resolution of the three-dimensional voxel grid is used to represent the division granularity (or fineness) of the three-dimensional voxel grid, which can be understood as the scene resolution of a sample scene. For example: if the division granularity of the three-dimensional voxel grid is smaller, the higher the voxel grid resolution of the three-dimensional voxel grid is, the higher the scene resolution of the corresponding sample scene is; conversely, if the division granularity of the three-dimensional voxel grid is larger, the lower the voxel grid resolution of the three-dimensional voxel grid is, the lower the scene resolution of the sample scene is correspondingly.

Under the same sample scene, the higher the resolution of the voxel grid of the three-dimensional voxel grid is, the higher the scene resolution of the sample scene is, the smaller the voxel block sizes of the three-dimensional voxel blocks forming the sample scene are, and the more the number of the voxel blocks of the three-dimensional voxel blocks is; conversely, the lower the voxel grid resolution of the three-dimensional voxel grid, the lower the scene resolution of the sample scene, the larger the voxel block sizes of the three-dimensional voxel blocks constituting the sample scene, and the smaller the number of voxel blocks of the three-dimensional voxel blocks.

Step 220, obtain and generate the mapping field.

Wherein the mapping size of the generated mapping field is the same as the scene size of the sample scene. Namely: the generation of the mapping field is implemented as a three-dimensional mapping field.

Illustratively, a scene size of the sample scene is determined after the sample scene is obtained, the scene size being indicative of a region size sufficient to encompass a three-dimensional region of the sample scene. For example: the sample scene is a cube scene of 3 x 3, and 3 x 3 is taken as the scene size corresponding to the sample scene; or the sample scene is located in a 2 x 2 square scene, 2 x 2 is taken as a sample scene corresponding scene size, etc.

The generated mapping field is composed of a plurality of mapping blocks, and the plurality of mapping blocks respectively store coordinate data.

Illustratively, the mapping block sizes of the plurality of mapping blocks in the generated mapping field are equal to the voxel block sizes of the three-dimensional voxel blocks in the sample scene; alternatively, the map block size of the plurality of map blocks in the map field is generated to be greater than the voxel block size of the three-dimensional voxel block in the sample scene.

And 230, perturbing the plurality of mapping blocks in the generated mapping field to obtain a perturbed generated mapping field.

Illustratively, after the generated mapping field is obtained, the generated mapping field is disturbed, and a disturbance process of disturbing a plurality of mapping blocks in the generated mapping field is realized through the disturbance process.

Alternatively, the disturbance process is implemented by a process of adding gaussian noise. For example: after the generated mapping field is obtained, the generated mapping field is randomly disturbed by using Gaussian noise, so that a disturbance process of a plurality of mapping blocks in the generated mapping field is realized.

The plurality of mapping blocks in the perturbed generated mapping field store perturbed coordinate data.

Schematically, as shown in fig. 3, a schematic tangential diagram for generating a mapping field is shown, namely: a schematic two-dimensional representation of the three-dimensional generated mapping field is presented. Before the generation of the mapping field is disturbed, the two-dimensional surface of the generated mapping field is presented as an arrangement mode 310, wherein the arrangement mode comprises coordinate data (data such as 00, 01 and the like are data identifications of the coordinate data and do not represent the coordinate data); after the generated mapping field is perturbed, the two-dimensional surface of the generated mapping field is presented as an arrangement 320, which includes the perturbed coordinate data. Namely: since at least two pieces of coordinate data of the perturbed generated map field vary from that of the generated map field, a plurality of map blocks in the perturbed generated map field store the perturbed coordinate data.

And step 240, mapping the sample scene to the perturbed generated mapping field to obtain a generated mapping scene.

Illustratively, the sample scene includes a plurality of three-dimensional voxel blocks, and when the sample scene is mapped onto the perturbed generated mapping field, the plurality of three-dimensional voxel blocks in the sample scene are mapped onto the perturbed generated mapping field.

Optionally, dividing the plurality of three-dimensional voxel blocks in the sample scene according to the mapping block sizes of the mapping blocks in the generated mapping field, and mapping the divided three-dimensional voxel blocks onto the perturbed generated mapping field.

In some embodiments, the plurality of three-dimensional voxel blocks in the sample scene each have corresponding raw coordinate data indicating positional information of each three-dimensional voxel block relative to the sample scene. Wherein the scene size of the sample scene is the same as the mapping size of the generated mapping field, and the original coordinate data of the three-dimensional voxel block in the sample scene corresponds to the coordinate data of the mapping block in the generated mapping field (before disturbance).

Schematically, as shown in fig. 4, which includes a schematic two-dimensional section view of the sample scene 410, the two-dimensional section view of the three-dimensional voxel blocks 411 in the sample scene 410 is shown in dotted line form, and each three-dimensional voxel block 411 has corresponding original coordinate data for distinguishing the position information of the different three-dimensional voxel block 411 with respect to the sample scene.

Optionally, the sample scene is mapped onto the perturbed generated mapping field based on the divided three-dimensional voxel blocks and the original coordinate data corresponding to the three-dimensional voxel blocks.

Illustratively, the arrangement mode 310 shown in fig. 3 is used as a generated mapping field (before disturbance), and data identifiers corresponding to coordinate data of mapping blocks in the generated mapping field are respectively denoted as "00", "01", "10", "11"; if the arrangement 320 shown in fig. 3 is used as the perturbed generated mapping field, as shown in fig. 4, in the process of mapping the sample scene 410 with the perturbed generated mapping field 420, the plurality of three-dimensional voxel blocks in the sample scene 410 are divided according to the mapping block sizes of the mapping blocks in the perturbed generated mapping field 420, for example, every four three-dimensional voxel blocks 411 are divided into one three-dimensional voxel block group, so as to obtain the three-dimensional voxel block group 412, the three-dimensional voxel block group 413, the three-dimensional voxel block group 414 and the three-dimensional voxel block group 415 shown in fig. 4.

Optionally, in the sample scene 410, the coordinate information identification indicating the three-dimensional voxel block group 412 is determined to be 00, the coordinate information identification indicating the three-dimensional voxel block group 413 is determined to be 01, the coordinate information identification indicating the three-dimensional voxel block group 414 is determined to be 10, and the coordinate information identification indicating the three-dimensional voxel block group 415 is determined to be 11 based on the original coordinate data of the three-dimensional voxel block and the data identification corresponding to the coordinate data of the mapping block in the generated mapping field.

In some embodiments, the plurality of three-dimensional voxel groups are mapped onto the perturbed mapping blocks at corresponding positions according to the coordinate information identifiers corresponding to the plurality of three-dimensional voxel groups in the sample scene 410, respectively, to obtain the generated mapping scene.

As shown in fig. 4, when mapping the sample scene 410 onto the perturbed generated mapping field 420, according to the coordinate information identifiers corresponding to the three-dimensional voxel groups in the sample scene 410, the three-dimensional voxel groups are mapped onto perturbed mapping blocks at corresponding positions, respectively, to obtain the generated mapping scene 430.

For example: determining a perturbed mapping block 421 with the data identification of the coordinate data of "00" from the perturbed generated mapping field 420 based on the coordinate information identification of the three-dimensional voxel block 412, and mapping the three-dimensional voxel block 412 onto the perturbed mapping block 421; similarly, based on the coordinate information of the three-dimensional voxel block 413 being identified as 01, determining a perturbed mapping block 422 with the data identification of the coordinate data being "01" from the perturbed generated mapping field 420, and mapping the three-dimensional voxel block 413 onto the perturbed mapping block 422; determining a perturbed mapping block 423 of which the data identification of the coordinate data is 10 from the perturbed generated mapping field 420 based on the coordinate information identification of the three-dimensional voxel block 414, and mapping the three-dimensional voxel block 414 onto the perturbed mapping block 423; based on the coordinate information of the three-dimensional voxel block group 415 being identified as 11, a perturbed map block 424 having the data identification of the coordinate data of "11" is determined from the perturbed generated map field 420, and the three-dimensional voxel block group 415 is mapped onto the perturbed map block 424, thereby obtaining the generated map scene 430.

The method comprises the steps of generating voxel attribute data of a three-dimensional voxel block after disturbance in a mapping scene, and mapping the voxel attribute data of the three-dimensional voxel block after disturbance to coordinate data after disturbance.

Illustratively, each three-dimensional voxel block includes voxel attribute data corresponding to the three-dimensional voxel block, where the voxel attribute data is used to indicate a voxel characteristic specific to the three-dimensional voxel block.

Illustratively, the voxel attribute data includes a voxel density value, where the voxel density value is used to represent density information corresponding to the three-dimensional voxel block, and may be regarded as a geometric feature indicated by the three-dimensional voxel block. For example: the voxel density value is 1, which indicates that an object exists at the three-dimensional voxel block; alternatively, a voxel density value of 0 indicates that no object is present at the three-dimensional voxel block.

Illustratively, the voxel attribute data further comprises spherical harmonic parameters, and the spherical harmonic parameters are used for representing change information of the display effect of the three-dimensional voxel block along with the change of the observation angle, and can be regarded as appearance characteristics indicated by the three-dimensional voxel block.

Based on the process of mapping the sample scene to the perturbed generated mapping field, the obtained generated mapping scene stores voxel attribute data of the perturbed three-dimensional voxel block, and the voxel attribute data in the perturbed three-dimensional voxel block is unchanged, but the coordinate data of the perturbed three-dimensional voxel block is changed, as shown by the generated mapping scene 430 in fig. 4, that is: voxel attribute data of the perturbed three-dimensional voxel block is mapped onto perturbed coordinate data.

In an alternative embodiment, after performing feature transformation on the sample scene, obtaining a transformed sample scene; mapping the transformed sample scene to the perturbed generated mapping field to obtain a generated mapping scene.

Optionally, after obtaining the sample scene, the voxel attribute data of the three-dimensional voxel block in the sample scene may be subjected to feature transformation by a feature transformation processing manner, and a mapping scene is generated by means of the transformed sample scene.

Illustratively, a plurality of three-dimensional voxel blocks in the sample scene each have corresponding voxel attribute data including voxel density values and spherical harmonic parameters. In analyzing a sample scene, the following description will be given by taking analysis of a plurality of three-dimensional voxel blocks as an example.

In an alternative embodiment, feature processing is performed on voxel density values corresponding to a plurality of three-dimensional voxel blocks in the sample scene, so as to obtain geometric features corresponding to the sample scene.

Schematically, the voxel density value is used to represent the voxel density condition of the sample scene in the corresponding three-dimensional voxel block, and can be regarded as the geometric feature indicated by the three-dimensional voxel block, and the geometric feature corresponding to the sample scene is obtained based on the feature processing of the geometric feature respectively indicated by the three-dimensional voxel blocks.

Optionally, the geometric features of the sample scene obtained after feature processing can achieve the purpose of finer representation of the sample scene.

In an alternative embodiment, feature compression is performed on spherical harmonic parameters corresponding to a plurality of three-dimensional voxel blocks in a sample scene, so as to obtain appearance features corresponding to the sample scene.

Schematically, the spherical harmonic function parameter is used for representing the change information of the presentation effect of the three-dimensional voxel block along with the change of the observation angle, and can be regarded as the appearance characteristic indicated by the three-dimensional voxel block, and the appearance characteristic corresponding to the sample scene is obtained based on the characteristic compression of the appearance characteristic respectively indicated by the three-dimensional voxel blocks.

Optionally, the appearance characteristics of the sample scene obtained after the characteristic compression can achieve the purpose of reducing the calculation consumption when the sample scene is analyzed.

In an alternative embodiment, the transformed sample scene is constructed based on geometric features and appearance features.

And 250, carrying out coordinate assignment on a plurality of mapping blocks in the disturbed generated mapping field based on the voxel block association relation between the disturbed three-dimensional voxel blocks in the generated mapping scene and the three-dimensional voxel blocks in the sample scene, and obtaining a target scene.

Illustratively, the voxel block association relationship is used for representing the association condition between the three-dimensional voxel block and the three-dimensional voxel block after disturbance, and the voxel block association relationship comprises at least one of a voxel block proximity relationship and a voxel block similarity relationship.

Optionally, determining the nearest three-dimensional voxel block from the three-dimensional voxel blocks in the sample scene based on the three-dimensional voxel blocks after disturbance in the generated mapping scene; or, determining the three-dimensional voxel block with the highest similarity from the three-dimensional voxel blocks in the sample scene by taking the three-dimensional voxel block after disturbance in the generated mapping scene as a reference; or, determining a three-dimensional voxel block with the highest similarity and the nearest neighbor from the three-dimensional voxel blocks in the sample scene by taking the three-dimensional voxel blocks after disturbance in the generated mapping scene as a reference.

The proximity relation is used to indicate a positional proximity relation of the three-dimensional voxel block in a distributed form, for example: and analyzing the distribution form of the three-dimensional voxel blocks, wherein 8 voxel blocks with adjacent relation with the three-dimensional voxel block A exist, and the 8 adjacent voxel blocks are used as the three-dimensional voxel blocks adjacent to the three-dimensional voxel block A.

The similarity relationship is used to indicate the similarity condition of the three-dimensional voxel block on voxel attribute data, for example: the similarity between the voxel attribute data of the three-dimensional voxel block A and the voxel attribute data of the three-dimensional voxel block B is 86%; the similarity between the voxel attribute data of the three-dimensional voxel block a and the voxel attribute data of the three-dimensional voxel block C is 42%.

In an alternative embodiment, after determining the association relation of the voxel blocks, coordinate assignment is performed on the mapping blocks in the perturbed generated mapping field by using the sample scene, and a target scene is obtained.

Optionally, after determining the proximity relation and the similarity relation between the disturbed three-dimensional voxel block in the generated mapping scene and the three-dimensional voxel block in the sample scene, searching the three-dimensional voxel block which is closest and most similar to the disturbed three-dimensional voxel block in the generated mapping scene from the sample scene based on the disturbed three-dimensional voxel block in the generated mapping scene, and performing coordinate assignment on the mapping block in the disturbed generated mapping field corresponding to the updated generated mapping scene based on the coordinate data of the three-dimensional voxel block in the sample scene.

In some embodiments, after coordinate assignment is performed on the mapping blocks in the perturbed generated mapping field, an updated generated mapping field is obtained, and the sample scene is mapped to the updated generated mapping field, so as to obtain the target scene.

Illustratively, the coordinate data in the disturbed generated mapping field is updated through the coordinate assignment process, so that an updated generated mapping field after the coordinate data is updated is obtained, and when the sample scene is transformed, the sample scene is mapped to the updated generated mapping field, so that the process of transforming to obtain the target scene can be realized.

It should be noted that the above is only an illustrative example, and the embodiments of the present application are not limited thereto.

In summary, through the method, the coordinate data corresponding to each three-dimensional voxel block in the generated mapping field can be determined more robustly by means of the coordinate updating process, the generated mapping field after random disturbance is updated, the scene is transformed to the sample scene, the target scene after scene transformation is obtained, the appearance and geometric characteristics of the local part of each three-dimensional voxel block are smoothed, the target scene with high sense of reality can be generated, and the scene transformation process for generating various target scenes through a single sample scene can be realized.

In an alternative embodiment, when coordinate assignment is performed on a plurality of mapping blocks in the perturbed generated mapping field based on the voxel block association relationship, the update process of the perturbed generated mapping field is realized by comparing the proximity relationship and the similarity relationship between the perturbed three-dimensional voxel block in the generated mapping field and the three-dimensional voxel block in the sample scene. Illustratively, as shown in fig. 5, the step 250 shown in fig. 2 described above may also be implemented as steps 510 through 520 described below.

Step 510, based on the proximity relation and the similarity relation between the disturbed three-dimensional voxel block in the generated mapping scene and the three-dimensional voxel block in the sample scene, performing attribute update on the three-dimensional voxel block in the generated mapping scene to obtain an updated generated mapping scene.

Illustratively, the three-dimensional voxel block on the perturbed generated mapping field is referred to as a perturbed three-dimensional voxel block. After the generated mapping scene and the sample scene are obtained, the attribute of the three-dimensional voxel block in the generated mapping scene is updated based on the proximity relation and the similarity relation between the three-dimensional voxel block in the generated mapping scene and the three-dimensional voxel block in the sample scene.

The proximity relation is used for indicating the position proximity relation of the three-dimensional voxel block in a distributed form; the similarity relationship is used to indicate the similarity condition of the three-dimensional voxel block on the voxel attribute data. In an alternative embodiment, after determining the proximity relation and the similarity relation, the attribute update is performed on the three-dimensional voxel block in the generated mapping scene with the sample scene.

Optionally, after determining the proximity relation and the similarity relation between the three-dimensional voxel block in the sample scene and the perturbed three-dimensional voxel block in the generated mapping scene, searching the three-dimensional voxel block which is closest to and is most similar to the three-dimensional voxel block in the generated mapping scene from the sample scene by taking the three-dimensional voxel block in the generated mapping scene as a reference, and updating the attribute of the corresponding three-dimensional voxel block in the generated mapping scene by using the three-dimensional voxel block in the sample scene.

Illustratively, taking the three-dimensional voxel block A in the generated mapping scene as an example, searching a three-dimensional voxel block which is closest to and is most similar to the three-dimensional voxel block A from the sample scene to obtain a three-dimensional voxel block B in the sample scene, and carrying out attribute updating on the three-dimensional voxel block A by using the three-dimensional voxel block B.

Optionally, after determining the proximity relation and the similarity relation between the three-dimensional voxel blocks in the sample scene and the disturbed three-dimensional voxel blocks in the generated mapping scene, searching m three-dimensional voxel blocks which are similar and similar to the three-dimensional voxel blocks in the generated mapping scene from the sample scene by taking the three-dimensional voxel blocks in the generated mapping scene as a reference, and updating the attribute of the corresponding three-dimensional voxel blocks in the generated mapping scene based on the m three-dimensional voxel blocks in the sample scene.

Illustratively, taking the three-dimensional voxel block A in the generated mapping scene as an example for analysis, searching p three-dimensional voxel blocks closest to the three-dimensional voxel block from the sample scene, determining the similarity respectively corresponding to the p three-dimensional voxel blocks and the three-dimensional voxel block A, selecting the first m three-dimensional voxel blocks with the highest similarity, and carrying out attribute updating on the three-dimensional voxel block A by the first m three-dimensional voxel blocks.

In an alternative embodiment, the updated generated mapping scene is obtained in response to the number of updates for attribute updating of the three-dimensional voxel block in the generated mapping scene reaching a preset number of thresholds.

Schematically, the preset frequency threshold is a preset frequency value, for example: the preset frequency threshold value is 9, and after the update frequency of the attribute update of the three-dimensional voxel block in the generated mapping scene reaches 9 times, the updated generated mapping scene is obtained. Namely: the three-dimensional voxel blocks in the generated mapped scene may be subjected to a multiple attribute update process by the three-dimensional voxel blocks in the sample scene.

In some embodiments, attribute updating is performed on a plurality of three-dimensional voxel blocks in the generated mapping scene based on the attribute updating method, so as to obtain an updated generated mapping scene.

In an alternative embodiment, after performing feature transformation on the sample scene, obtaining a transformed sample scene; mapping the transformed sample scene to a disturbed generated mapping field to obtain a generated mapping scene; based on the adjacent relation and the similarity relation between the disturbed three-dimensional voxel blocks in the generated mapping scene and the three-dimensional voxel blocks in the transformed sample scene, carrying out attribute updating on the three-dimensional voxel blocks in the generated mapping scene to obtain an updated generated mapping scene.

Illustratively, a feature preprocessing process may be performed on a sample scene through a feature transformation process to obtain a transformed sample scene, and after a mapping scene is generated based on the transformed sample scene, an update process for the generated mapping scene is implemented based on the generated mapping scene and the transformed sample scene.

In an alternative embodiment, the scene update process may also be performed by performing scene block extraction on the sample scene and the generated mapping scene based on a preset scene block size, and comparing the similarity relationship and the proximity relationship between the scene blocks.

Optionally, extracting the scene blocks from the three-dimensional voxel blocks in the sample scene with a preset scene block size to obtain a plurality of first scene blocks corresponding to the sample scene. Wherein the preset scene block size is greater than the size of the three-dimensional voxel block.

Optionally, extracting the scene block from the three-dimensional voxel blocks in the generated mapping scene with a preset scene block size to obtain a plurality of second scene blocks corresponding to the generated mapping scene.

Optionally, based on the proximity relations and the similarity relations between the plurality of first scene blocks and the plurality of second scene blocks, attribute updating is performed on the three-dimensional voxel blocks in the generated mapping scene, and the updated generated mapping scene is obtained.

For an illustration, reference may be made to the embodiment shown in fig. 5 with respect to the updating of the properties of the generated mapped scene by the first scene block and the second scene block.

And step 520, performing coordinate assignment on the mapping blocks in the disturbed generated mapping field by using the sample scene based on the proximity relation and the similarity relation between the three-dimensional voxel blocks in the updated generated mapping scene and the three-dimensional voxel blocks in the sample scene, and obtaining the target scene.

Illustratively, after obtaining the updated generated mapping scene, determining coordinate data of the three-dimensional voxel block in the updated generated mapping scene based on a proximity relation and a similarity relation between the three-dimensional voxel block in the updated generated mapping scene and the three-dimensional voxel block in the sample scene, and performing coordinate assignment on the mapping block in the perturbed generated mapping field based on the coordinate data, namely: and carrying out a coordinate updating process on the coordinate data in the disturbed generated mapping field.

The proximity relation is used for indicating the position proximity relation of the three-dimensional voxel block in a distributed form; the similarity relationship is used to indicate the similarity condition of the three-dimensional voxel block on the voxel attribute data.

In an alternative embodiment, after determining the proximity relation and the similarity relation between the three-dimensional voxel block in the updated generated mapping scene and the three-dimensional voxel block in the sample scene, searching the three-dimensional voxel block closest to and similar to the three-dimensional voxel block in the updated generated mapping scene from the sample scene based on the three-dimensional voxel block in the updated generated mapping scene, and performing coordinate assignment on the mapping block in the perturbed generated mapping field corresponding to the updated generated mapping scene by using the three-dimensional voxel block in the sample scene.

Illustratively, taking the updated three-dimensional voxel block C in the generated mapping scene as an example, searching a three-dimensional voxel block which is closest to and is most similar to the three-dimensional voxel block C from a sample scene to obtain a three-dimensional voxel block D in the sample scene, and determining coordinate data (representing the position information of the three-dimensional voxel block D in the sample scene) of the three-dimensional voxel block D in the sample scene; in addition, the coordinate data of the mapping block C corresponding to the three-dimensional voxel block C in the disturbed generated mapping field is determined, and the coordinate assignment is performed on the mapping block C corresponding to the three-dimensional voxel block C in the disturbed generated mapping field by using the coordinate data of the three-dimensional voxel block D. And the coordinate updating process of replacing the coordinate data of the mapping block c in the disturbed generated mapping field with the coordinate data of the three-dimensional voxel block D is realized.

Optionally, after determining the proximity relation and the similarity relation between the three-dimensional voxel blocks in the updated generated mapping scene and the three-dimensional voxel blocks in the sample scene, searching m three-dimensional voxel blocks which are similar and similar to the three-dimensional voxel blocks in the updated generated mapping from the sample scene based on the three-dimensional voxel blocks in the sample scene, and performing coordinate assignment on the mapping blocks in the perturbed generated mapping field corresponding to the updated generated mapping scene based on the m three-dimensional voxel blocks in the sample scene.

Illustratively, taking the three-dimensional voxel block A in the updated generated mapping scene as an example, searching p three-dimensional voxel blocks closest to the three-dimensional voxel block A from the sample scene, determining the similarity respectively corresponding to the p three-dimensional voxel blocks and the three-dimensional voxel block A, selecting the first m three-dimensional voxel blocks with the highest similarity from the p three-dimensional voxel blocks, determining coordinate data respectively corresponding to the first m three-dimensional voxel blocks, carrying out coordinate assignment on the mapping block a corresponding to the three-dimensional voxel block A in the generated mapping field after disturbance by using the coordinate data of the first m three-dimensional voxel blocks as an example. Such as: carrying out coordinate assignment on a mapping block a corresponding to a three-dimensional voxel block A in a disturbed generated mapping field by using the average value of coordinate data of the previous m three-dimensional voxel blocks; or, coordinate position intersections of the coordinate data of the previous m three-dimensional voxel blocks are used for assigning coordinates to the mapping block a corresponding to the three-dimensional voxel block a in the perturbed generated mapping field.

In some embodiments, based on the above coordinate assignment method, coordinate assignment is performed on mapping blocks corresponding to the plurality of three-dimensional voxel blocks in the perturbed generated mapping field, so as to implement a process of updating coordinates of the perturbed generated mapping field, and obtain an updated generated mapping field.

In an alternative embodiment, after obtaining the updated generated mapping field, the sample scene is mapped to the updated generated mapping field to obtain the target scene.

Optionally, performing coordinate assignment on a plurality of mapping blocks in the perturbed generated mapping field by using the sample scene based on the proximity relation and the similarity relation between the three-dimensional voxel blocks in the updated generated mapping scene and the three-dimensional voxel blocks in the sample scene, so as to obtain the updated generated mapping field; and mapping the sample scene to the updated generated mapping field to obtain a target scene.

Schematically, compared with the generated mapping field after disturbance, the updated generated mapping field is a mapping field obtained by adjusting based on the sample scene, after the updated generated mapping field is obtained, the sample scene is mapped to the updated generated mapping field, and then the position of the three-dimensional voxel block in the sample scene can be adjusted, so that a target scene with high authenticity can be generated under the condition that more diversified scenes can be generated.

In an alternative embodiment, after obtaining the updated generated mapping field, determining an update condition of the updated generated mapping field; and obtaining the target scene based on the updated condition.

Illustratively, the update condition is used to measure whether the updated generated mapping field reaches the update condition. Illustratively, the update condition is a preset condition, such as: the update condition is implemented as an update times condition that generates a mapping field.

When the updated generated mapping field reaches the updating condition, determining the updated generated mapping field as a target generated mapping field, and mapping the sample scene onto the target generated mapping field, thereby obtaining the target scene and the like.

In an alternative embodiment, after performing feature transformation on the sample scene, obtaining a transformed sample scene; mapping the transformed sample scene to a disturbed generated mapping field to obtain a generated mapping scene; based on the adjacent relation and the similarity relation between the disturbed three-dimensional voxel blocks in the generated mapping scene and the three-dimensional voxel blocks in the transformed sample scene, carrying out attribute updating on the three-dimensional voxel blocks in the generated mapping scene to obtain an updated generated mapping scene; based on the proximity relation and the similarity relation between the three-dimensional voxel blocks in the generated mapping scene after updating and the three-dimensional voxel blocks in the sample scene after transformation, carrying out coordinate assignment on a plurality of mapping blocks in the generated mapping field after sample scene disturbance, and obtaining the target scene.

Optionally, based on the proximity relation and the similarity relation between the plurality of first scene blocks and the plurality of second scene blocks, performing attribute update on the three-dimensional voxel blocks in the generated mapping scene to obtain an updated generated mapping scene; based on the proximity relation and the similarity relation between the three-dimensional voxel blocks in the updated generated mapping scene and the three-dimensional voxel blocks in the sample scene, carrying out coordinate assignment on a plurality of mapping blocks in the disturbed generated mapping field by the sample scene, and obtaining a target scene.

For an illustration, reference may be made to the embodiment shown in fig. 6 for the assignment of coordinates to the perturbed generated mapping field by the first scene block and the second scene block.

In the embodiment of the application, the proximity relation and the similarity relation are integrated to be used as the voxel block association relation, firstly, the updated generated mapping scene after attribute updating is carried out on the three-dimensional voxel blocks is obtained based on the proximity relation and the similarity relation between the disturbed three-dimensional voxel blocks in the generated mapping scene and the three-dimensional voxel blocks in the sample scene, the updated generated mapping scene can smooth voxel attribute data relation among a plurality of three-dimensional voxel blocks, and further coordinate assignment is carried out on the mapping blocks in the disturbed generated mapping field based on the updated generated mapping scene, and the accuracy of coordinate assignment is improved by integrating the proximity relation and the similarity relation, so that the obtained target scene is not generated into inappropriate scene content under the condition that the obtained target scene is different from the sample scene, and the target scene has stronger authenticity.

In an optional embodiment, when updating the generated mapping scene and the perturbed generated mapping field based on the proximity relation and the similarity relation, the sample scene may be further subjected to feature transformation to obtain a transformed sample scene; in addition, scene block extraction is performed on the transformed mapping scene and the generated mapping scene with a preset scene block size, and an updating process is performed based on the extracted first scene block and second scene block. Illustratively, as shown in FIG. 6, the embodiment shown in FIG. 2 described above may also be implemented as steps 610 through 674 below.

In step 610, a sample scene is acquired.

In an alternative embodiment, a plurality of two-dimensional images acquired from different acquisition angles for a given scene are acquired.

Illustratively, the specified scene is a preselected three-dimensional scene. For example: the specified scene may be implemented as a real world scene, such as "cactus", "stone", etc.; alternatively, the specified scene may be implemented as a digitally synthesized virtual scene, or the like.

Taking a specified scene as an example of a building U in the real world, a plurality of two-dimensional images acquired by a camera from different acquisition angles for the building U are adopted, and the plurality of two-dimensional images show the observation effect of the building U at different observation angles, such as: the two-dimensional image 1 shows the front face of the building U, the two-dimensional image 2 shows the oblique side face of the building U, the two-dimensional image 3 shows the side face of the building U, and the like.

Illustratively, the plurality of two-dimensional images are implemented as multi-view images, namely: images acquired from multiple angles for the same designated scene. Illustratively, for facilitating the subsequent operations, it is assumed that the entire scene E corresponding to the plurality of two-dimensional images is located at the origin and inside a unit cube having one center at the origin and a side length of 2.

In some embodiments, the sample scene is obtained based on a plurality of two-dimensional image acquisitions.

Optionally, after obtaining a plurality of two-dimensional images, a full volume element (Plenoptic volume elements, plenoxels) is used to reconstruct the scene from the plurality of two-dimensional images to obtain a sample scene, wherein Plenoxels are implemented as a sparse three-dimensional voxel grid.

The sparse degree indicated by the sparsity is used for reflecting whether data exists in three-dimensional voxel blocks corresponding to the sample scene, and the sparse three-dimensional voxel grid is used for indicating that any significance does not exist in some three-dimensional voxel blocks, namely: no actual data is saved.

Alternatively, the reconstructed scene may be visualized by Volume Rendering (Volume Rendering) to render a high quality result as a picture. In the process of mapping a scene by using the Plenoxels and obtaining a sample scene, as shown in fig. 7, a schematic diagram of obtaining the sample scene is obtained.

The plurality of two-dimensional images 710 are mapped onto the three-dimensional voxel grid 720 (Sparse Voxel Grid) as training images, and a plurality of voxel grids 721 (small cubic blocks) are included in the three-dimensional voxel grid 720, and the plurality of voxel grids 721 have the same size, so that a scene is constructed on the plurality of two-dimensional images 710 by the three-dimensional voxel grid 720.

Illustratively, in the process of performing scene mapping on the plurality of two-dimensional images 710 through the three-dimensional voxel grid 720, a voxel center corresponding to each voxel grid 721 in the three-dimensional voxel grid 720 is determined, and based on the mapping process of the plurality of two-dimensional images 710, a density value ρ and a spherical harmonic parameter h corresponding to each voxel center are determined, where the density value ρ and the spherical harmonic parameter h are used for displaying scene characteristics of the sample scene.

Illustratively, a process of constructing a scene on a plurality of two-dimensional images based on three-dimensional voxel grids, in a sample scene, determining a three-dimensional voxel block corresponding to each voxel grid in the three-dimensional voxel grids, and taking a density value ρ and a spherical harmonic parameter h determined by a voxel center corresponding to each voxel grid as voxel attribute data of the three-dimensional voxel block.

The density value ρ is used for showing the density characteristic condition of the sample scene corresponding to the voxel center, and is also the density characteristic condition in the three-dimensional voxel block corresponding to the voxel center. For example: ρ=1 indicates that the sample scene is in a solid state at the voxel center, i.e., there is an object in the three-dimensional voxel block corresponding to the voxel center; ρ=0 indicates that the sample scene is in a hollow state at the voxel center, i.e.: no object exists in the three-dimensional voxel block corresponding to the voxel center.

The spherical harmonic function parameter h is used for showing the change information of the presentation effect corresponding to the sample scene at the voxel center along with the change of the observation angle, and is also the change information of the presentation effect of the three-dimensional voxel block corresponding to the voxel center along with the change of the observation angle.

Illustratively, the process of processing multiple two-dimensional images through Plenoxels and obtaining a sample scene can be represented using the following formula.

E:x→(ρ,h)

Wherein E is used to represent a sample scene; x is used to represent the coordinates of the voxel center point.

As shown in fig. 7, x may alternatively be implemented as a point other than the voxel center point, namely: points that lie within voxel grid 721 but are not at the voxel center. When determining the density value ρ and the spherical harmonic parameter h corresponding to points other than the voxel center, the density value ρ and the spherical harmonic parameter h corresponding to the point are obtained by the tri-linear interpolation method 730.

Schematically, for any point x ₁ Determining a point x ₁ A plurality of voxel center points closest to each other in the three-dimensional voxel grid, and determining a density value rho and a spherical harmonic parameter h corresponding to each of the plurality of voxel center points; performing tri-linear interpolation through density values rho and spherical harmonic parameters h respectively corresponding to a plurality of voxel center points to obtain a point x ₁ A corresponding density value ρ and a spherical harmonic parameter h.

For example: determining point x ₁ 8 voxel center points closest to the three-dimensional voxel grid, and a density value rho and a spherical harmonic parameter h corresponding to the 8 voxel center points respectively; by passing throughPerforming tri-linear interpolation on the density value rho and the spherical harmonic parameter h corresponding to the 8 voxel central points respectively to obtain a point x ₁ A corresponding density value ρ and a spherical harmonic parameter h.

In some embodiments, for the spherical harmonics, the order of the spherical harmonics used and the parameter dimension of the spherical harmonics may be preset, where the order of the spherical harmonics is used to represent the degree of freedom that the spherical harmonics can represent, and the higher the order is, the higher the appearance detail that the spherical harmonics can express higher frequencies; the parameter dimension of the spherical harmonic is used to represent the coefficient form from which the spherical harmonic is derived.

Alternatively, spherical harmonics of order 2 (or 2 degrees) are used by default, and the dimension of the spherical harmonic parameter h is set to 27 by default.

Schematically, the overall flow of FIG. 7 is briefly described as follows.

(a) Using the plurality of two-dimensional images 710 as training images, performing a construction process of the sample scene through the three-dimensional voxel grid 720 (Sparse Voxel Grid); a density value ρ and a spherical harmonic parameter h corresponding to each voxel center are determined based on the plurality of two-dimensional images 710.

(b) To render a ray, the density value ρ and spherical harmonic parameter h of the spatial point of the sample scene are calculated by the tri-linear interpolation method 730, so that the color and opacity of the corresponding spatial point can be determined.

(c) The density values ρ and spherical harmonic parameters h of the plurality of spatial points are integrated by using the differentiable volume rendering method, so that the opacity σ of the spatial points to light at different distances can be determined along the ray distance in the tri-linear interpolation method 730, thereby predicting the color.

(d) A standard mean square error (Mean Squared Error, MSE) reconstruction penalty with respect to the plurality of two-dimensional images 710 and a total variation regularizer are used to optimize voxel coefficients.

Step 620, obtain the generated mapping field.

Step 620 is already described in step 220 above and will not be described again here.

And 630, perturbing the plurality of mapping blocks in the generated mapping field to obtain a perturbed generated mapping field.

Illustratively, after the generated mapping field is obtained, the generated mapping field is disturbed, and a disturbance process of disturbing a plurality of mapping blocks in the generated mapping field is realized through the disturbance process. Alternatively, the disturbance process is implemented by a process of adding gaussian noise.

Step 630 is already described in step 230 above, and will not be described again here.

And step 640, performing feature transformation on the sample scene to obtain a transformed sample scene.

Illustratively, a plurality of three-dimensional voxel blocks in the sample scene each have corresponding voxel attribute data including voxel density values and spherical harmonic parameters.

In some embodiments, feature processing is performed on voxel density values respectively corresponding to a plurality of three-dimensional voxel blocks in the sample scene to obtain geometric features corresponding to the sample scene.

Illustratively, the voxel density values are used to represent the voxel density of the sample scene within the corresponding three-dimensional voxel block to distinguish the appearance of the sample scene within the corresponding three-dimensional voxel block.

Optionally, determining voxel density values respectively corresponding to the plurality of three-dimensional voxel blocks in the sample scene, and performing feature processing on the voxel density values respectively corresponding to the plurality of three-dimensional voxel blocks in the sample scene to obtain geometric features corresponding to the sample scene.

The geometric features of the sample scene obtained after the feature processing can be used for showing the sample scene more finely.

Optionally, for a target three-dimensional voxel block of the plurality of three-dimensional voxel blocks, determining a target density value for a three-dimensional voxel center point in the target three-dimensional voxel block.

Wherein the three-dimensional voxel center point is used to represent the center point of the three-dimensional voxel block.

Schematically, the target three-dimensional voxel block is any one three-dimensional voxel block in a plurality of three-dimensional voxel blocks corresponding to the sample scene, a target density value corresponding to a three-dimensional voxel center point in the target three-dimensional voxel block is determined, and the target density value is the voxel density value stored in the three-dimensional voxel center point in the target three-dimensional voxel block.

Optionally, determining three-dimensional voxel center points corresponding to the three-dimensional voxel blocks in the sample scene respectively; and determining a target density value corresponding to a target three-dimensional voxel center point in the plurality of three-dimensional voxel center points.

Optionally, density filling is performed on the target three-dimensional voxel block with the target density value to obtain a filled three-dimensional voxel block.

Illustratively, after determining a target density value corresponding to a three-dimensional voxel center point in a target three-dimensional voxel block, performing density filling on the target three-dimensional voxel block by using the target density value, and calling the target three-dimensional voxel block after density filling as a filling three-dimensional voxel block.

In some embodiments, the target three-dimensional voxel block is density filled using a JumpFlood algorithm to implement a process of filling the interior of the sample scene such that the interior of the sample scene becomes solid.

Optionally, feature processing is performed on the filled three-dimensional voxel block, and a geometric feature corresponding to the sample scene is obtained.

In an alternative embodiment, a filled three-dimensional voxel block corresponding to each of a plurality of three-dimensional voxel blocks is acquired; constructing a filling three-dimensional scene with filling three-dimensional voxel blocks; extracting an isosurface from the filling three-dimensional scene to obtain a target isosurface; and obtaining geometrical characteristics corresponding to the sample scene based on the position relation between the target isosurface and the plurality of three-dimensional voxel center points.

Illustratively, after the filled three-dimensional scene is obtained by the density filling operation, the internal noise impact of the filled three-dimensional scene is reduced. A mesh filling the three-dimensional scene is extracted using an iso-surface extraction method (Marching Cubes), and then signed distance field (Signed Distance Field, SDF) values for voxel center point locations are calculated from the extracted mesh.

Illustratively, each scene point in the filled three-dimensional scene has an attribute value. The isosurface extraction method is used for extracting a surface with a 0 attribute value in the filled three-dimensional scene, the 0 attribute value is used for drawing the scene edge of the three-dimensional scene, so that an isosurface uniquely corresponding to the three-dimensional scene is obtained, optionally, the isosurface formed by the 0 attribute value is called a 0 isosurface, and the 0 isosurface is used as the target isosurface.

In some embodiments, after the 0 isosurface is extracted, determining a plurality of voxel center points, and determining distances between the plurality of voxel center points and the 0 isosurface, thereby obtaining SDF values respectively corresponding to the plurality of voxel center points and the 0 isosurface.

For example: the plurality of voxel center points comprise a voxel center point 1, a voxel center point 2 and the like, the extracted isosurface is a 0 isosurface, the SDF value 1 between the voxel center point 1 and the 0 isosurface is determined, the SDF value 2 between the voxel center point 2 and the 0 isosurface is determined, and therefore a plurality of SDF values are obtained.

Illustratively, if the SDF value is greater than 0, it indicates that the voxel center point location is inside the value plane, i.e.: the voxel center point is located within the filled three-dimensional scene; if the SDF value is less than 0, it indicates that the voxel is outside the plane of the value, i.e.: the closer the SDF is to 0, the closer it is to the iso-surface, etc., the voxel center is outside the filled three-dimensional scene.

In some embodiments, the following formula is used simultaneously to truncate it to a truncated signed distance function (Truncated Signed Distance Function, TSDF) and to serve as a geometric feature of the sample scene (i.e., determined by the density-filled three-dimensional scene).

Wherein G (x) represents a geometric feature; max () represents the maximum value; min () represents a minimum value; SDF (x) represents the SDF value of point x; t is a preselected cutoff range. Illustratively, 3 times the current voxel side length is used by default as the range of TSDF truncations.

Illustratively, the spherical harmonic function parameter is used for representing the change information of the presentation effect of the three-dimensional voxel block along with the change of the observation angle, and can be regarded as the appearance characteristic indicated by the three-dimensional voxel block.

In an optional embodiment, performing dimension reduction operation on spherical harmonic parameters respectively corresponding to a plurality of three-dimensional voxel blocks in the sample scene to obtain dimension reduction information respectively corresponding to the plurality of three-dimensional voxel blocks; and splicing the plurality of dimension reduction information to obtain appearance characteristics corresponding to the sample scene.

Alternatively, taking the default use of the spherical harmonic parameter h with dimension 27 as an example, a significant amount of computational consumption may be introduced if the dimension 27 is directly employed. The principal component analysis (Principal Component Analysis, PCA) can therefore be employed for dimension reduction, defaulting to 27 dimensions down to 3 dimensions, thereby significantly reducing the computational resource requirements.

P(h)＝PCA(h)

Wherein, P (h) is used for representing the appearance characteristics of the spherical harmonic parameter h subjected to dimension reduction by adopting PCA.

Schematically, after the geometric features and the appearance features are obtained, a transformed sample scene is constructed, and the geometric features in the transformed sample scene can be displayed more finely; the appearance characteristics of the transformed sample scene can reduce the calculation consumption when analyzing the sample scene.

Step 650, mapping the transformed sample scene onto the perturbed generated mapping field to obtain a generated mapping scene.

Illustratively, the transformed sample scene includes a plurality of three-dimensional voxel blocks, and when the transformed sample scene is mapped onto the perturbed generated mapping field, the plurality of three-dimensional voxel blocks in the transformed sample scene are mapped onto the perturbed generated mapping field.

Optionally, referring to the process of mapping the sample scene onto the perturbed generated mapping field in step 240 as described above, the transformed sample scene is mapped onto the perturbed generated mapping field to obtain the generated mapping scene. The transformed sample scene is a preferred embodiment over the sample scene.

In step 661, the three-dimensional voxel blocks in the transformed sample scene are extracted with the preset scene block size, so as to obtain a plurality of first scene blocks corresponding to the transformed sample scene.

Illustratively, the preset scene block size is used for representing a preset scene block size, and is used for dividing the three-dimensional voxel block in the transformed sample scene, and the preset scene block size is larger than the size of the three-dimensional voxel block. Namely: when the three-dimensional voxel blocks in the transformed sample scene are extracted through the preset scene block size, the size of one extracted scene block is larger than that of one three-dimensional voxel block, and the extracted scene block comprises a plurality of three-dimensional voxel blocks.

In some embodiments, after the transformed sample scene is divided by the preset scene block size, a plurality of first scene blocks are obtained, where the plurality of first scene blocks belong to scene blocks having the same scene block size.

In an alternative embodiment, a scene block size may be preset to perform scene block extraction on three-dimensional voxel blocks in a sample scene (a sample scene before transformation), so as to obtain a plurality of first scene blocks corresponding to the sample scene.

Step 662, extracting the scene blocks from the three-dimensional voxel blocks in the generated mapping scene with the preset scene block size to obtain a plurality of second scene blocks corresponding to the generated mapping scene.

Schematically, the size of the preset scene block for extracting the scene block of the generated mapping scene is the same as the size of the preset scene block for extracting the scene block of the transformed sample scene, that is: and extracting the scene blocks of the generated mapping scene and the transformed sample scene with the same scene block size.

Optionally, extracting the scene blocks from the three-dimensional voxel blocks in the generated mapping scene by presetting the scene block size to obtain a plurality of second scene blocks corresponding to the generated mapping scene.

Step 663, based on the proximity relations and the similarity relations between the plurality of first scene blocks and the plurality of second scene blocks, performing attribute update on the three-dimensional voxel blocks in the generated mapping scene to obtain an updated generated mapping scene.

Wherein the proximity relation is used for indicating the position proximity relation of the first scene block and the second scene block (the scene block comprises the three-dimensional voxel block) in a distributed form; the similarity relationship is used to indicate the similarity condition of the three-dimensional voxel block on the voxel attribute data.

In an alternative embodiment, the scene sizes of the sample scene after transformation and the generated mapping scene are the same, and are determined based on the sample scene, and after the scene block extraction is performed on the sample scene after transformation and the generated mapping scene with the same preset scene block size, a scene block corresponding relation exists between a plurality of first scene blocks and a plurality of second scene blocks.

Illustratively, after obtaining the plurality of first scene blocks and the plurality of second scene blocks, determining a proximity relationship between the first scene blocks and the second scene blocks, and determining a similarity relationship between the first scene blocks and the second scene blocks under the proximity relationship.

In some embodiments, for a target second scene block of the plurality of second scene blocks, n first scene blocks nearest to the target second scene block are determined, and scene block similarities for the target second scene block and the n first scene blocks, respectively, in voxel properties are determined.

Wherein n is a positive integer.

Illustratively, the target second scene block is any one of the plurality of second scene blocks, and when the target second scene block is analyzed, the similarity of the target second scene block and the scene blocks corresponding to the n first scene blocks respectively in voxel attribute is determined.

Optionally, a first scene block corresponding to the maximum scene similarity is determined as the first updated scene block. Namely: a first updated scene block having a closest proximity relationship to the target second scene block is obtained.

The first updating scene block is used for updating the attribute.

In an alternative embodiment, the voxel attribute data of the perturbed three-dimensional voxel block in the target second scene block is updated with the voxel attribute data of the perturbed three-dimensional voxel block in the first updated scene block.

Illustratively, after determining a first updated scene block corresponding to the target second scene block, determining voxel attribute data of the perturbed three-dimensional voxel block in the first updated scene block, and performing attribute update on the voxel attribute data of the perturbed three-dimensional voxel block in the target second scene block through the voxel attribute data of the perturbed three-dimensional voxel block in the first updated scene block.

Optionally, replacing voxel attribute data of the perturbed three-dimensional voxel block in the target second scene block with voxel attribute data of the perturbed three-dimensional voxel block in the first updated scene block, and obtaining an updated generated mapping scene.

Optionally, first voxel attribute data of the perturbed three-dimensional voxel block in the first updated scene block is acquired.

Wherein the voxel attribute data comprises a voxel density value and a spherical harmonic parameter. The first voxel attribute data corresponds to a first updated scene block, and a plurality of first scene blocks respectively have corresponding first voxel attribute data.

Schematically, the voxel density value is used for representing density information corresponding to the three-dimensional voxel block, and can be regarded as a geometric feature indicated by the three-dimensional voxel block; the spherical harmonic function parameter is used for representing the change information of the display effect of the three-dimensional voxel block along with the change of the observation angle, and can be regarded as the appearance characteristic indicated by the three-dimensional voxel block.

Optionally, second voxel attribute data of the perturbed three-dimensional voxel block in the target second scene block is acquired.

The second voxel attribute data corresponds to a target second scene block, and the plurality of second scene blocks have corresponding second voxel attribute data, respectively.

Optionally, the second voxel attribute data is replaced with the first voxel attribute data, resulting in updated voxel attribute data corresponding to the target second scene block.

Illustratively, after the first voxel attribute data corresponding to the first updated scene block and the second voxel attribute data corresponding to the target second scene block are obtained, when the generated mapping scene is updated through the transformed sample scene, the second voxel attribute data is replaced by the first voxel attribute data, so that the target second scene block is updated, and the updated voxel attribute data corresponding to the target second scene block is obtained.

In an alternative embodiment, a plurality of target first scene blocks corresponding to the target second scene blocks are determined, namely: the plurality of target first scene blocks and the target second scene blocks conform to the nearest most similar relationship, such as: the nearest r select the first n similar target first scene blocks.

Optionally, determining first voxel attribute data corresponding to the plurality of target first scene blocks respectively, and performing weighted fusion on the plurality of first voxel attribute data to obtain updated voxel attribute data corresponding to the target second scene blocks.

In step 671, for a target second scene block of the plurality of second scene blocks, determining m first scene blocks that are nearest to the target second scene block, and determining scene block similarities of the target second scene block and the m first scene blocks that respectively correspond in voxel attribute.

Wherein m is a positive integer.

The target second scene block belongs to any one of the plurality of second scene blocks, and the similarity of the target second scene block and the scene blocks corresponding to the plurality of first scene blocks in voxel attribute is determined.

And when analyzing the target second scene block, determining the similarity of the target second scene block and the scene blocks respectively corresponding to the plurality of first scene blocks in voxel attribute.

Step 672, determining the first scene block corresponding to the maximum scene similarity as the second updated scene block.

The second updated scene block is used for coordinate assignment.

Schematically, after obtaining the scene similarity of the target first scene block and m second scene blocks with similarity relations on voxel attribute data, determining the second scene block corresponding to the maximum scene similarity, and taking the second scene block as a second updated scene block. Namely: and obtaining a target second scene block with the nearest and most similar relation with the target first scene block.

In step 673, a target map block corresponding to the target second scene block is determined from the perturbed generated map field.

In an alternative embodiment, the generated mapping scene is a scene obtained by mapping the transformed sample scene onto the perturbed generated mapping field, so that the mapping size of the perturbed generated mapping field is also the same as the scene size of the generated mapping scene, and is implemented as the scene size determined based on the sample scene.

After scene block extraction is performed on the transformed sample scene and the generated mapping scene with the same preset scene block size, a scene block corresponding relation exists between the obtained plurality of first scene blocks and the obtained plurality of second scene blocks, and according to the target second scene blocks, a target mapping block corresponding to the target second scene blocks can be determined from the disturbed generated mapping field, so that the position information of the target mapping block on the disturbed generated mapping field can be determined.

And step 674, taking the coordinate data of the disturbed three-dimensional voxel block in the second updated scene block as the coordinate data of the target mapping block, and obtaining the target scene.

Schematically, after obtaining coordinate data of the disturbed three-dimensional voxel block in the second updated scene block, taking the coordinate data as coordinate data of the target mapping block based on a corresponding relation between the second updated scene block and the target second scene block and a corresponding relation between the target second scene block and the target mapping block, thereby realizing a coordinate assignment process, namely: and adjusting the coordinate position of the target mapping block in the disturbed generated mapping field.

In an optional embodiment, based on the above coordinate assignment method, coordinate assignment is performed on mapping blocks corresponding to the plurality of three-dimensional voxel blocks in the perturbed generated mapping field, so as to implement a process of performing coordinate update on the perturbed generated mapping field, and obtain an updated generated mapping field.

Optionally, mapping the sample scene to the updated generated mapping field to obtain a target scene; or mapping the transformed sample scene to the updated generated mapping field to obtain the target scene.

In some embodiments, determining an updated update condition of the updated generated mapping field; based on the update condition, a generated mapping scene corresponding to the updated generated mapping field is determined, and the generated mapping scene is taken as a target scene.

In an alternative embodiment, as shown in fig. 8, the scene block extraction process described above is illustrated by the following example. Wherein the input is a transformed sample sceneGenerating a mapping scene->The process of generating the target scene is performed by the nearest neighbor search method 810.

Schematically, the transformed sample scene is also referred to as "input scene transformed with the plenum feature"; the generation of the Mapping scene is also referred to as "generation scene (Mapping field)". Optionally, manually specified generation limits (e.g., reference samples) may also be entered, etc.

In some embodiments, scene blocks (patches) are extracted by presetting scene block sizes. For a scene represented by a three-dimensional voxel grid, a three-dimensional deconvolution approach may be used, cubes of p x p dimensions are extracted from many of the scenes, wherein, the liquid crystal display device comprises a liquid crystal display device, the p×p×p size is a preset scene block size.

Illustratively, p=5 is chosen by default, approximately occupying 1/3 of the lowest scene resolution, and the step size of the extraction block is 1.

Optional pair, for transformed sample sceneGenerating a mapping scene->The transformed sample scene +_ with the preset scene block size>And generate mapping scene->Extracting scene blocks (patches) respectively to obtain scene blocksA set Q and a set K, wherein a plurality of first scene blocks after scene block extraction of the transformed sample scene are stored in the set Q; the set K stores a plurality of second scene blocks obtained by extracting scene blocks of the generated mapped scene.

In some embodiments, for generating a mapped sceneThe extracted scene block set Q finds its nearest neighbor (nearest neighbor and most similar) in the set K. And calculates a metric matrix of similarity between scene blocks as shown in the following formula. />

Wherein Q is ^a Spherical harmonic parameters representing a first scene block, K ^a Spherical harmonic parameters representing a second scene block; q (Q) ^g Voxel density values representing a first scene block; k (K) ^g Voxel density values representing a second scene block; II represents the L2 distance (Euclidean norm); w (w) _a Representing pre-selected parameters such as: to balance the importance of the appearance features (spherical harmonic parameters) and the geometric features (voxel density values), w is chosen by default _a ＝0.5。

And then, the similarity measurement matrix is changed to obtain a measurement matrix C shown in the following formula.

Wherein, the number of all scene blocks in the scene block set Q; a controls the weight of the integrity and diversity of the whole scene, a smaller a indicates a better integrity of the generated scene, illustratively, a=0.01 is set by default.

In an alternative embodiment, as shown in fig. 8, the process of updating the generated mapping scene and the perturbed generated mapping field is as follows.

(1) Based on the value generation module 820, the transformed generated mapping scene is updatedCharacteristic values (voxel attribute data) of (a) a sample scene after transformation without updating +.>Is included in the coordinate data of the image data. For the nearest neighbors with the highest similarity obtained by calculation, each overlapped characteristic value is subjected to weighted fusion (the step length is 1, the size p is 5, and therefore, the overlapping exists) to obtain an updated characteristic value (updated voxel attribute data);

As shown in fig. 9, for any one second scene block 910, the most similar nearest neighbor in the first scene blocks corresponding to the second scene block 910 is determined, and the overlapping feature values among the plurality of first scene blocks of the most similar nearest neighbor are weighted and fused, so as to obtain updated feature values (updated voxel attribute data).

(2) Based on the coordinate generation module 830, the perturbed generated mapping field is updatedFor the nearest neighbor with the highest similarity obtained by calculation, the coordinates of the center point of the block are obtained and used as the generated mapping field after disturbance>The coordinate data of the coordinate data is obtained.

As shown in fig. 10, for the second scene block 1010, the most similar nearest neighbor in the first scene block corresponding to the second scene block 1010 is determined, and the coordinate data of the first scene block of the most similar nearest neighbor is determined, and is used as the coordinate data for updating the generated mapping field after disturbance.

In an alternative embodiment, a larger matrix needs to be calculated in the above-described most similar nearest neighbor determination process, which results in a large amount of computational resource consumption.

Optionally, in the process of calculating the similarity matrix, an accurate nearest neighbor query may be employed in the case where the sample scene has a lower scene resolution; in the case where the sample scene has a higher scene resolution, an approximate nearest neighbor query is employed.

In the embodiment of the application, the process of updating the generated mapping scene and the disturbed generated mapping field based on the adjacent relation and the similarity relation between the disturbed three-dimensional voxel block in the generated mapping scene and the three-dimensional voxel block in the sample scene is introduced, the scene block extraction is carried out on the generated mapping scene and the sample scene according to the preset scene block size, the attribute update is carried out on the three-dimensional voxel block in the generated mapping scene based on the adjacent relation and the similarity relation between the extracted first scene block and the extracted second scene block, the coordinate assignment is carried out on the mapping block in the disturbed generated mapping field based on the attribute update condition, the target scene is generated according to the updated generated mapping field, and the target scene is generated more comprehensively and robustly from two aspects of voxel attribute data and coordinate data by a generating algorithm based on the scene block, so that the target scene with high fidelity is obtained.

In an alternative embodiment, the sample scenes are represented by using different scene resolutions, and the target scene is obtained based on the sample scenes respectively corresponding to the different scene resolutions and the updated generated mapping field. Illustratively, as shown in fig. 11, the embodiment shown in fig. 2 described above may also be implemented as follows steps 1110 to 1180.

At step 1110, a specified scene is determined.

Wherein the specified scene is used to represent the scene from which the sample scene was generated. Namely: the specified scene is used to represent the three-dimensional scene for which the sample scene is intended. For example: the method comprises the steps that a designated scene is a building U, and a sample scene to be analyzed is obtained aiming at the building U.

And 1120, performing scene expression on the specified scene with a plurality of scene resolutions to obtain sample scenes respectively corresponding to the plurality of scene resolutions.

In an alternative embodiment, a plurality of two-dimensional images acquired from different acquisition angles for a specified scene are acquired; and obtaining a sample scene through the three-dimensional voxel grid based on the plurality of two-dimensional images.

Illustratively, in the process of acquiring sample scenes corresponding to a plurality of scene resolutions, the scene resolution adjustment process is performed by voxel grid resolutions of different three-dimensional voxel grids.

Optionally, in adjusting the resolution of the scene by adjusting the resolution of the voxel grid, the process of adjusting the resolution of the voxel grid is achieved by adjusting the division granularity of the three-dimensional voxel grid.

Illustratively, the higher the granularity of division of the three-dimensional voxel grid, the higher the scene resolution; the lower the granularity of division into three-dimensional voxel grids, the lower the scene resolution.

Optionally, a plurality of different division granularities are adopted to express the designated scene, so that sample scenes respectively corresponding to different scene resolutions are obtained.

For example: adopting a first division granularity pair to designate a scene to obtain a sample scene with first scene resolution corresponding to the first division granularity; and adopting the second division granularity to the designated scene to obtain a sample scene with the second scene resolution corresponding to the second division granularity.

Illustratively, the setting is a process performed in a coarse-grained to fine-grained manner in the process of reconstructing a sample scene to a target scene, thereby constructing a scene pyramid of different scene resolutions.

Optionally, the first scene resolution corresponding to the three-dimensional voxel grid at the first division granularity is a coarsest scene resolution. For example: first scene resolution Res (E) ₀ ) Default to 16 indicates that in the three-dimensional voxel grid at the first division granularity, there is 16 voxel grids for one row.

Optionally, the second scene resolution corresponding to the three-dimensional voxel grid at the second division granularity is a finer scene resolution. In accordance with the procedure from coarse scene resolution to fine scene resolution, the rule of change in the scene resolution is determined to be: res (E) _n+1 )＝r*Res(E _n ) For example: r defaults to 4/3.

In an alternative embodiment, as shown in fig. 12, 8 sample scenes with different scene resolutions are preset, namely: an 8-layer scene pyramid with the layer number of N=8 is constructed, and different pyramids represent different scene resolutions.

For example: when n=1, the sample scene 1210 is implemented as E ₀ The method comprises the steps of carrying out a first treatment on the surface of the When n=n, sample scene 1220 is implemented as E _n The method comprises the steps of carrying out a first treatment on the surface of the When n=n, sample scene 1230 is implemented as E _N . The number of layers N and the proportion r of the scene pyramid can be adjusted according to the application and the requirement.

Illustratively, the scene resolution of sample scene 1210 is the lowest and the scene resolution of sample scene 1230 is the highest.

Step 1130, obtaining a first sample scene corresponding to the first scene resolution from the plurality of sample scenes.

Illustratively, the first scene resolution is the lowest scene resolution among the plurality of scene resolutions, and the sample scene corresponding to the first scene resolution is referred to as the first sample scene, that is: and acquiring a sample scene corresponding to the lowest scene resolution from the plurality of sample scenes.

In step 1140, a first updated generated mapping field is obtained based on the first sample scene acquisition.

Illustratively, the first updated generated mapping field is obtained in the manner described above in steps 210 through 250.

Namely: the method comprises the steps of obtaining a generated mapping field, wherein the generated mapping field consists of a plurality of mapping blocks, the plurality of mapping blocks respectively store coordinate data, and the mapping size of the generated mapping field is the same as the scene size of a sample scene; disturbing a plurality of mapping blocks in the generated mapping field to obtain a disturbed generated mapping field, wherein the plurality of mapping blocks in the disturbed generated mapping field store disturbed coordinate data; mapping the first sample scene to the perturbed generated mapping field to obtain a first generated mapping scene, storing voxel attribute data of the perturbed three-dimensional voxel block in the first generated mapping scene, and mapping the voxel attribute data of the perturbed three-dimensional voxel block to perturbed coordinate data; based on the adjacent relation and the similarity relation between the disturbed three-dimensional voxel block in the first generated mapping scene and the three-dimensional voxel block in the sample scene, carrying out attribute updating on the three-dimensional voxel block in the first generated mapping scene to obtain a first updated generated mapping scene; and carrying out coordinate assignment on the mapping blocks in the disturbed generated mapping field by using the first updated generated mapping scene based on the proximity relation and the similarity relation between the three-dimensional voxel blocks in the first updated generated mapping scene and the three-dimensional voxel blocks in the first sample scene, so as to obtain the first updated generated mapping field.

In an alternative embodiment, the target scene is derived from the sample scene based on a comparison between the first updated generated mapping field and the update conditions. Wherein the update condition is a preset condition. Illustratively, when the first updated generated mapping field does not reach the update condition, the following step 1150 is performed.

In step 1150, in the case that the first updated generated mapping field does not reach the update condition, an upsampling operation is performed on the first updated generated mapping field to obtain a first candidate mapping field.

The updating condition is a preset condition and is used for measuring whether the generated mapping field after current updating is suitable to be used as the mapping field to carry out the mapping process on the sample scene. When the first updated generated mapping field does not reach the update condition, that is, the first updated generated mapping field is not suitable for being used as the mapping field to perform the mapping process on the sample scene, the first updated generated mapping field needs to be processed.

Illustratively, an upsampling operation is performed on the first updated generated mapped field to obtain a first candidate mapped field.

Wherein the first candidate mapped field corresponds to a second scene resolution, the second scene resolution being a higher scene resolution than the first scene resolution.

Illustratively, as shown in fig. 13, the sample scene portion 1310 includes a plurality of sample scenes with respective scene resolutions.

Schematically, for generating a mapping field S ₀ After disturbance (random disturbance), a generated mapping field after disturbance is obtainedFor example: post-perturbation generated mapping field->Is to generate a mapping field S ₀ And random scrambling by using Gaussian noise, wherein the random scrambling is obtained by using Gaussian noise, and the random scrambling is obtained by using the following formula.

Wherein z is ₀ ＝Gauss(0,σ ² ) For example: default σ=0.5, and can be adjusted as needed.

Generating mapping field based on disturbanceAnd a first sample scene E ₀ Obtaining a first updated generationMapping field 1330.

Upsampling the first updated generated mapped field 1330 to obtain a first candidate mapped fieldBy setting up the upsampling operation, the first candidate mapped field +.>And corresponding to the second scene resolution, wherein the second scene resolution is the scene resolution corresponding to the second sample scene.

As shown in FIG. 13, where the region 1340 is described in detail, the region 1340 is implemented as shown in FIG. 14.

(1) In the case of sample scene E ₀ 1410, sample scene E may be processed ₀ 1410 to obtain a transformed sample scene

(2) Based on sample scene E ₀ 1410, resulting in a generated mapped field S having the same mapped size as the scene size ₀ 1430, and to generate a mapping field S ₀ 1430 perturbation to obtain a generated mapping field after perturbation

(3) Scene of the transformed sampleMapping to post-disturbance generated mapping field +.>On top of that, a generated mapping scenario ∈>

(4) For the transformed sample sceneAnd extracting the scene blocks to obtain a scene block set K.

(5) For generating mapping sceneAnd extracting the scene blocks to obtain a scene block set Q.

(6) For each scene block in scene block set Q, find the most similar and nearest neighbor scene block in scene block set K, generate a mapping scene pairThe voxel attribute data in the model is subjected to attribute updating, and a mapping field generated after disturbance is also subjected to +.>Performing coordinate assignment to obtain updated generated mapping field S ₁ And get the updated generated mapping field S ₁ And performing up-sampling operation to obtain a first candidate mapping field.

Step 1160, using the first candidate mapping field as the perturbed generated mapping field, and obtaining a second updated generated mapping field based on the first candidate mapping field and the second sample scene.

The scene resolution corresponding to the second sample scene is the second scene resolution; the scene resolution corresponding to the first candidate mapped field is also the second scene resolution.

After the first candidate mapping field is used as the perturbed generated mapping field, a second updated generated mapping field is obtained through steps 240 to 250.

As shown in fig. 13, after the first candidate mapped field is obtainedBased on the first candidate mapping field->And a second sample scene E ₁ Obtaining a second updated generated mapping field S ₁ 。

Wherein, a second updated generated mapping field S is obtained ₁ Can refer to the generation of mapping fields according to disturbanceAnd a first sample scene E ₀ The treatment process between them is obtained. />

Illustratively, if a plurality of sample scenes corresponding to the scene resolutions are obtained, a plurality of candidate mapping fields can be obtained based on the content shown in fig. 13And a corresponding plurality of updated generated mapping fields.

In an alternative embodiment, the target scene is derived from the sample scene based on a comparison between the second updated generated mapping field and the update conditions. Wherein the update condition is a preset condition.

In step 1170, in response to the ith updated generated mapping field reaching the update condition, the ith updated generated mapping field is targeted to the generated mapping field.

Wherein i is a positive integer greater than 2.

The ith updated generated mapping field corresponds to the ith scene resolution, the ith scene resolution is the scene resolution corresponding to the ith sample scene, and the ith scene resolution is higher than the ith-1 scene resolution.

Optionally, the updating condition is used for indicating that the ith scene resolution indicated by the ith updated generated mapping field is the maximum scene resolution, and determining the ith updated generated mapping field as the target generated mapping field.

In an alternative embodiment, in response to the ith updated generated mapping field not reaching the update condition, an upsampling operation is performed on the ith updated generated mapping field to obtain an ith candidate mapping field.

In an alternative embodiment, the i-1 th candidate mapping field is used as a perturbed generated mapping field, and the i-1 th updated generated mapping field is obtained based on the i-1 th candidate mapping field and the i-th sample scene; and carrying out iterative updating on the generated mapping field after the i-th updating until a target generated mapping field is obtained. Wherein i is a positive integer.

And step 1180, mapping the sample scene onto a target generation mapping field to obtain a target scene.

Schematically, after determining the target generation mapping field, mapping the sample scene onto the target generation mapping field to obtain a target scene after transforming the sample scene.

Optionally, the ith updated generated mapping field corresponds to the (i+1) th scene resolution. The (i+1) th scene resolution is higher than the (i) th scene resolution.

When mapping a sample scene to a target generation mapping field to obtain a target scene, mapping the sample scene corresponding to the (i+1) th scene resolution to the target generation mapping field to obtain the target scene. Namely: the target scene comprises a scene obtained by mapping a sample scene corresponding to the (i+1) th scene resolution to a target generated mapping field.

In addition, as shown in the upper part of fig. 13, in order to process the sample scene, a local region 1350 is displayed after processing the scene height of the sample scene; there is a change in the height of the scene element in the local region 1350 as compared to the corresponding region in the sample scene.

In the embodiment of the application, a method for constructing and obtaining a target scene by means of sample scenes corresponding to different scene resolutions is introduced, the processing condition of the updated generated mapping field is determined according to whether the updated generated mapping field reaches the updating condition, when the i updated generated mapping field reaches the updating condition, the i updated generated mapping field is used as the target generated mapping field, and the sample scene is mapped to the target generated mapping field to obtain the target scene. The object generation mapping field which is more suitable for mapping the sample scene can be obtained through the process of updating voxel attribute data to coordinate assignment, and the distribution condition of mapping blocks in the object generation mapping field is more balanced, so that a target scene with more realism is obtained; in addition, by means of sample scenes with lower scene resolution to sample scenes with higher field Jing Fenbian rate, the problem that analysis data are less in sample scene analysis is solved, and a target scene with high sense of reality is obtained by means of an iterative training process.

In an alternative embodiment, the above three-dimensional scene construction method is referred to as a sample block-based three-dimensional scene generation and editing method. Firstly, starting from a scene expression, in order to ensure the high sense of reality of a generated scene, using nerve radiation field variety Plenoxels as a basis, and converting implicit geometric expression of the nerve radiation field variety Plenoxels into an explicit symbol distance field SDF; in order to solve the problem of too little sample data, a progressive pyramid three-dimensional generation model from coarse granularity to fine granularity is provided; in order to perform robust generation, a generation algorithm based on values and based on coordinate serial is provided; meanwhile, the performance optimization of a nearest neighbor query algorithm from accurate to rough is proposed due to the large resource consumption of three-dimensional calculation. Fig. 15 is a schematic diagram of a frame according to an embodiment of the present application.

One) input multiview picture 1510

Schematically, a two-dimensional image acquired from a plurality of angles for a specified scene is acquired, so as to obtain a multi-view picture, namely: a plurality of images taken from different perspectives are obtained.

(II) reconstruction of three-dimensional scene Condition pyramid 1520

Illustratively, for an input multi-view picture, it is scene reconstructed using the Plenoxels expression. For convenience of the subsequent operations, it is assumed that the entire sample scene E is located at the origin and inside a unit cube having a side length of 2 at one center at the origin.

Referring to fig. 4, the Plenoxels expression is a sparse voxel grid, and a density value ρ and a spherical harmonic parameter h are stored in the center of each voxel, so that the whole three-dimensional scene can be expressed by the following formula.

E:x→(ρ,h)

Where x represents the coordinates of the voxel center point. For each point in space, the density value and the spherical harmonic parameter of the point can be obtained by performing three linear interpolations on the density value ρ and the spherical harmonic parameter h stored in the nearest voxel center. For example: by default, a spherical harmonic of 2 degrees is used, with a corresponding spherical harmonic parameter h of dimension 27. The reconstructed scene can be visualized by Volume Rendering (Volume Rendering), and a result with high quality like a picture can be rendered.

The entire scene reconstruction process is performed in coarse to fine granularity, thereby constructing scene pyramids of different resolutions, as shown in fig. 12. Coarsest scene resolution Res (E ₀ ) Default to 16, resolution of the next finer scene is Res (E _n+1 )＝r*Res(E _n ) Where r defaults to 4/3. In general, for the balance of efficiency and mass, a total of n=8 layer pyramids are constructed. The number of layers and the proportion r of the scene pyramid can be adjusted according to the application and the change of the requirement.

(III) feature pretreatment Change 1530

Schematically, as shown in fig. 16, a flow chart of a feature preprocessing process is shown. First, a sample scene 1610 (for example, a three-dimensional scene expressed by using Plenoxels) is input, and a feature preprocessing change process is performed from two aspects of geometric features (voxel density values ρ) and appearance features (spherical harmonic parameters h).

Namely: before generation can take place, feature preprocessing is required for each layer of scene of the input scene pyramid. The whole scene expression is divided into two parts, namely a voxel density value rho (a voxel center point density value rho) and a spherical harmonic parameter h, which respectively correspond to the geometric feature and the appearance feature of the sample scene. For the above two features, the following feature pretreatment processes were performed, respectively.

(1) Geometric feature preprocessing

Illustratively, the JumpFlood algorithm is used to fill the interior of the scene first, so that the interior of the scene becomes solid, and the influence of internal noise is reduced for better subsequent mesh (mesh) extraction results. Then the grid (mesh) of the scene is extracted using the Marching Cubes algorithm. And then calculating the SDF value of the position of the voxel center point according to the extracted mesh (mesh), and simultaneously cutting off the SDF value by using the following formula, and converting the SDF value into a final TSDF as the geometric feature of the scene.

Where t is the artificially specified cutoff range. For example: by default, 3 times the current voxel side length is used as the range of TSDF cutoff.

(2) Appearance feature pretreatment

Illustratively, if a spherical harmonic parameter h of dimension 27 is used by default, a significant amount of computational effort is introduced. Thus, using PCA for dimension reduction for appearance features, defaults to 27 dimensions down to 3 dimensions, thereby significantly reducing the need for computing resources.

P(h)＝PCA(h)

And (5) integrating the feature preprocessing and the appearance feature preprocessing processes to obtain a three-dimensional scene (a sample scene after transformation) after feature processing.

In some embodiments, in addition to processing the two-dimensional image to obtain the sample scene using the neural radiation field variety Plenoxels as the basis, the two-dimensional image may be processed by replacing the three-dimensional scene representation based on the neural radiation distance field with other similar three-dimensional scene representations, such as: original neural radiation field method (Neural Radiance Fields, neRF), tensor radiation field (Tensorial Radiance Fields, tensorRF), etc. Meanwhile, when other methods are adopted to process the two-dimensional image, the processing of the scene features can be replaced by other modes, such as: the depth features are used or the appearance features of the scene are processed and used more finely. Namely: the above is merely illustrative examples, and embodiments of the present application are not limited thereto, when other methods are used to process two-dimensional images.

(IV) Multi-scale Generation Module 1540

(1) Generating scene representations

The generated scene and the input scene are different expressions. The input scene is the transformed plenoxels (sample scene), and the generated scene in the embodiment of the application is the coordinate-based mapping field (generated mapping field). Namely: the data structure of the generated scene (generated mapped field) is similar to plenoxels, but internally stores the spatial coordinates of the input scene. Thus features in the input scene (sample scene) can be mapped to generate a new generated scene. The default generated scene is consistent with the resolution of the input scene, with the stored spatial coordinates initialized to their original positions.

S:x _s →x _e

For any point in space, the mapped position of the point can also be obtained by calculating the offset of the stored coordinates of the nearest voxel.

S(x)＝s(N(x))+δ

Where N (·) represents the voxel center point of the grid nearest to x, δ=x-N (x) represents the coordinate distance of the input point from the voxel center point of the nearest grid in the generation space.

(2) Generating scene initialization

At the coarsest resolution, the input scene is randomly disturbed by using Gaussian noise, and an initialized noise scene is obtained.

(3) Base groupSerial generation of values and coordinates

Alternatively, to generate a scene more robust, a serial implementation of value-based and coordinate-based generation is employed. The input of the generating module is the input scene and the generating scene after the feature transformation, and the artificially specified limit can be added, and the generating scene is updated after the generating model. In the generating module, the generating based on the value and the generating based on the coordinates are respectively performed for T-1 times and 1 time, so as to balance efficiency and generating quality, t=10 can be defaulted, wherein specific parameters can be adjusted according to practical applications, such as: adjusted to t=15, t=5, etc.

(4) Generating module

Whether a value-based or coordinate-based generation module, the method can be generally divided into three steps, extracting scene blocks (patches), searching the most similar nearest neighbors of each scene block (patch), and updating the generated scene.

1. Scene blocks (patches) are extracted. For a scene expressed in a voxel grid, cubes of p x p dimensions can be extracted from many of the scene in a three-dimensional deconvolution. Where by default p=5, approximately 1/3 of the coarsest scene resolution is occupied, and the step size of the extraction block is 1. For an input sceneBy generating scene-transformed input scenes And respectively extracting the scene blocks to obtain a scene block set K and a scene block set Q.

2. For input scenes after generating scene changesThe extracted scene block set Q finds the nearest neighbor most similar to the scene block set Q in the scene block set K and calculates a measurement matrix D of the similarity between the scene blocks _i,j The method comprises the steps of carrying out a first treatment on the surface of the Then, the similarity measurement matrix is changed to obtain the final degreeAnd a quantity matrix C.

3. Updating the generated scene. The value-based and coordinate-based generation modules differ in how the generated scene is updated.

4. To the nearest neighbor query. If a larger matrix is to be calculated, this results in a significant consumption of computing resources. In the process of calculating the similarity matrix, the accurate nearest neighbor query can be adopted at the lower resolution, and the approximate nearest neighbor query can be adopted at the higher resolution.

Illustratively, front N of scene pyramid _e Layer, use accurate nearest neighbor queries, such as: default N _e =5; at the same time T in the generation module _e =10, so that the generated result is relatively stable. At N _e After +1 layer, using PatchMatch algorithm to find approximate nearest neighbor, using jump flood algorithm to make parallel acceleration, T in generation module _a =2, the purpose is to add detail to the scene.

(5) Generating scene upsamples

In an alternative embodiment, after the single scale generation is completed, an up-sampling operation is required for the generated scene (updated generated mapping field) to prepare for the next scale scene generation.

Illustratively, as shown in fig. 17, new voxel center point coordinates are sampled according to the next layer scene resolution, and the mapped coordinates are obtained by generating a scene mapping process 1710. And then the scene resolution of the generated scene is the resolution of the next layer of scene, and the coordinates stored in the generated scene are updated.

In an alternative embodiment, additional manually specified restrictions may be added, such as: by manually specifying the restrictions, the same generation module is adopted to support applications such as editing a scene, such as: scene stretching compression, editing, analogy, sample replacement, etc. Alternatively, the gaussian noise of the initial generation scene is set to σ=0 for various applications.

Illustratively, when applied in scene stretching, compression, stretching and compression of the scene is achieved while maintaining the local shape of the scene, for example: by using different sizes of generated scenes S to realize, the length, width and height of the S are artificially set to be the specified proportion, so that new scenes with different proportions can be generated.

Illustratively, under the scene editing application, the coordinates saved in S are specified manually or specified manuallyThe stored characteristics can edit the scene. For example: the object in the scene can be deleted by pointing the coordinates of the specified position to the air, while the hole left by deletion is complemented, by artificial editing +.>The features saved in the scene, the objects in the scene are repeated and enlarged.

Illustratively, when applied in the presence Jing Leibi, by artificially designating a sceneThe saved features are replaced by the features after the new scene is changed, and a new scene with similar geometric structure with the designated scene can be generated.

Illustratively, when the same mapping field S is generated under the condition of replacing the sample scene application, the sample scene E can be a similar scene, so that more various new target scenes can be generated.

Optionally, specific parameters may be weighted according to actual needs in an actual application scenario and restrictions on computing and memory resources, and the following reference ranges and general settings of other parameters are schematically given.

(1) The resolution of the generated scene is approximately between 100 and 1000; (2) The proportion r=4/3 of the scene pyramid, the lowest scene resolution is 12-16; (3) the main component dimension of PCA dimension reduction is 3-27; (4) The cut-off range of TSDF is 1-5 times of the current voxel side length; (5) The weight of the appearance characteristic is w _a =0.1 to 0.9; (6) The radius of the jump flood algorithm is 8, and random search is performedThe radius of the cable is consistent with the current scene resolution.

In an alternative embodiment, the three-dimensional scene construction method provided above can overcome the problem of requiring a large amount of training data, and can quickly generate various high-quality new scenes only through a single three-dimensional scene, so that various types of scenes such as terrains, sculpture, plants, buildings and the like can be generated.

Illustratively, as shown in fig. 18, within the left box 1810 is an input sample scene, and the right region 1820 is a generated target scene.

Illustratively, various features of the original scene can be maintained by adopting the three-dimensional scene construction method, as shown in fig. 19, the reflection and shadow of the water surface 1910 along with the change of the visual angle can show different effects, and the like, so that the high quality of the appearance of the generated scene is ensured. For example: the microside view 1911 shows less reflection from the water surface 1910; the water surface 1910 shown in the oblique side view 1912 and the oblique side view 1913 is highly reflective; the side view 1914 shows a greater reflection of light from the water surface 1910, etc.

Optionally, in the application scenario of applying the three-dimensional scene construction method to the scenario editing, the scenario analogy, etc., as shown in the region 2010 in fig. 20, the stretching and shrinking operations may be performed on the premise of maintaining the local geometry for the object in the input scenario; as shown in region 2020 in fig. 20, objects in the scene may be added, deleted, and edited; as shown in region 2030 in fig. 20, a new scene with a similar structure to that can be generated by artificially designating the scene; as shown in the area 2040 in fig. 20, a plurality of objects having different scenes may be combined, and the input scene may be replaced to be mapped to a new scene, or the like.

Fig. 21 is a block diagram of a three-dimensional scene construction apparatus according to an exemplary embodiment of the present application, and as shown in fig. 21, the apparatus includes:

a first obtaining module 2110, configured to obtain a sample scene, where the sample scene is a three-dimensional scene;

a second obtaining module 2120, configured to obtain a generated mapping field, where the generated mapping field is composed of a plurality of mapping blocks, where the plurality of mapping blocks respectively store coordinate data, and a mapping size of the generated mapping field is the same as a scene size of the sample scene;

A perturbation module 2130, configured to perturb the plurality of map blocks in the generated mapping field to obtain a perturbed generated mapping field, where the plurality of map blocks in the perturbed generated mapping field store perturbed coordinate data;

the mapping module 2140 is configured to map the transformed sample scene onto the perturbed generated mapping field, so as to obtain a generated mapping scene, where voxel attribute data of the perturbed three-dimensional voxel block is stored in the generated mapping scene, and the voxel attribute data of the perturbed three-dimensional voxel block is mapped onto the perturbed coordinate data;

the generating module 2150 is configured to perform coordinate assignment on a plurality of mapping blocks in the perturbed generated mapping field based on a voxel block association relationship between the perturbed three-dimensional voxel block in the generated mapping scene and the three-dimensional voxel block in the sample scene, and obtain a target scene, where the target scene is a three-dimensional scene obtained by transforming on the basis of the sample scene.

In an alternative embodiment, as shown in fig. 22, the generating module 2150 further includes:

an updating unit 2151, configured to update attributes of three-dimensional voxel blocks in the generated mapping scene based on a proximity relation and a similarity relation between the disturbed three-dimensional voxel blocks in the generated mapping scene and the three-dimensional voxel blocks in the sample scene, so as to obtain an updated generated mapping scene;

And an assigning unit 2152, configured to assign coordinates to a plurality of mapping blocks in the perturbed generated mapping field by using the sample scene based on the proximity relation and the similarity relation between the three-dimensional voxel blocks in the updated generated mapping scene and the three-dimensional voxel blocks in the sample scene, and obtain the target scene.

In an optional embodiment, the updating unit 2151 is further configured to perform scene block extraction on a three-dimensional voxel block in the sample scene with a preset scene block size, to obtain a plurality of first scene blocks corresponding to the sample scene, where the preset scene block size is greater than the size of the three-dimensional voxel block; extracting the scene blocks from the three-dimensional voxel blocks in the generated mapping scene according to the preset scene block size to obtain a plurality of second scene blocks corresponding to the generated mapping scene; and based on the proximity relations and the similarity relations between the plurality of first scene blocks and the plurality of second scene blocks, performing attribute updating on the three-dimensional voxel blocks in the generated mapping scene to obtain the updated generated mapping scene.

In an optional embodiment, the updating unit 2151 is further configured to determine, for a target second scene block of the plurality of second scene blocks, n first scene blocks that are nearest to the target second scene block, and determine scene similarities of the target second scene block and the n first scene blocks that respectively correspond in voxel attribute, where n is a positive integer; determining a first scene block corresponding to the maximum scene similarity as a first updated scene block; and replacing voxel attribute data of the disturbed three-dimensional voxel block in the target second scene block with voxel attribute data of the disturbed three-dimensional voxel block in the first updated scene block, and obtaining the updated generated mapping scene.

In an alternative embodiment, the updating unit 2151 is further configured to obtain first voxel attribute data of a perturbed three-dimensional voxel block stored in the first updated scene block, where the voxel attribute data includes a voxel density value and a spherical harmonic parameter; acquiring second voxel attribute data of a three-dimensional voxel block stored in the target second scene block; and replacing the second voxel attribute data with the first voxel attribute data to obtain updated voxel attribute data corresponding to the target second scene block, and obtaining the updated generated mapping scene.

In an optional embodiment, the updating unit 2151 is further configured to determine, for a target second scene block of the plurality of second scene blocks, m first scene blocks that are nearest to the target second scene block, and determine scene similarities of the target second scene block and the m first scene blocks that respectively correspond in voxel attribute, where m is a positive integer; determining a first scene block corresponding to the maximum scene similarity as a second updated scene block; determining a target mapping block corresponding to the target second scene block from the perturbed generated mapping field; and taking the coordinate data of the disturbed three-dimensional voxel block in the second updated scene block as the coordinate data of the target mapping block, and obtaining the target scene.

In an optional embodiment, the assignment unit 2152 is further configured to perform coordinate assignment on the plurality of mapping blocks in the perturbed generated mapping field by using the sample scene based on a proximity relationship and a similarity relationship between the three-dimensional voxel block in the updated generated mapping scene and the three-dimensional voxel block in the sample scene, so as to obtain an updated generated mapping field; and mapping the sample scene to the updated generated mapping field to obtain the target scene.

In an optional embodiment, the first obtaining module 2110 is further configured to determine a specified scene, where the specified scene is used to represent a scene that generates the sample scene; performing scene expression on the specified scene with a plurality of scene resolutions to obtain sample scenes respectively corresponding to the plurality of scene resolutions;

as shown in fig. 23, the apparatus further includes:

a comparing module 2160, configured to obtain a first sample scene corresponding to a first scene resolution from a plurality of sample scenes, where the first scene resolution is a lowest scene resolution of the plurality of scene resolutions; acquiring a first updated generated mapping field based on the first sample scene; and obtaining the target scene through the sample scene based on the comparison relation between the first updated generated mapping field and the updating condition, wherein the updating condition is a preset condition.

In an alternative embodiment, the comparing module 2160 is further configured to, if the first updated generated mapping field does not reach the update condition, perform an upsampling operation on the first updated generated mapping field to obtain a first candidate mapping field, where the first candidate mapping field corresponds to a second scene resolution, and the second scene resolution is higher than the first scene resolution; taking the first candidate mapping field as the perturbed generated mapping field, and obtaining a second updated generated mapping field based on the first candidate mapping field and a second sample scene, wherein the second sample scene corresponds to the second scene resolution; and obtaining the target scene through the sample scene based on the comparison relation between the second updated generated mapping field and the updating condition.

In an alternative embodiment, the comparing module 2160 is further configured to, in response to the ith updated generated mapping field reaching the update condition, target the ith updated generated mapping field to generate a mapping field, where i is a positive integer greater than 1, and the ith updated generated mapping field corresponds to the i+1th scene resolution; and mapping the sample scene to the target to generate a mapping field to obtain the target scene, wherein the target scene comprises a scene obtained by mapping the sample scene corresponding to the (i+1) th scene resolution to the target to generate the mapping field.

In an alternative embodiment, the comparing module 2160 is further configured to use the i-1 th candidate mapping field as the perturbed generated mapping field, and obtain the i-1 th updated generated mapping field based on the i-1 th candidate mapping field and the i-th sample scene; and carrying out iterative updating on the generated mapping field after the ith updating until the target generated mapping field is obtained.

In an alternative embodiment, the comparing module 2160 is further configured to, in response to the ith updated generated mapping field not reaching the update condition, perform an upsampling operation on the ith updated generated mapping field to obtain an ith candidate mapping field.

In an optional embodiment, the mapping module 2140 is further configured to perform feature transformation on the sample scene to obtain a transformed sample scene; mapping the transformed sample scene to the perturbed generated mapping field to obtain the generated mapping scene.

In an optional embodiment, the mapping module 2140 is further configured to perform feature processing on voxel density values corresponding to a plurality of three-dimensional voxel blocks in the sample scene, so as to obtain geometric features corresponding to the sample scene; performing feature compression on spherical harmonic parameters respectively corresponding to a plurality of three-dimensional voxel blocks in the sample scene to obtain appearance features corresponding to the sample scene; and constructing and obtaining the transformed sample scene based on the geometric features and the appearance features.

In an optional embodiment, the mapping module 2140 is further configured to determine three-dimensional voxel center points corresponding to a plurality of three-dimensional voxel blocks in the sample scene, where the three-dimensional voxel center points are used to represent center points of the three-dimensional voxel blocks; determining a target density value corresponding to a target three-dimensional voxel center point in a plurality of three-dimensional voxel center points; determining a three-dimensional voxel block corresponding to the target three-dimensional voxel center point; performing density filling on the three-dimensional voxel block by using the target density value to obtain a filled three-dimensional voxel block corresponding to a target three-dimensional voxel center point; and carrying out feature processing on the filled three-dimensional voxel block, and obtaining the geometric feature corresponding to the sample scene.

In an optional embodiment, the mapping module 2140 is further configured to obtain filled three-dimensional voxel blocks corresponding to the plurality of three-dimensional voxel blocks, respectively; constructing a target three-dimensional scene by the filled three-dimensional voxel blocks; extracting an isosurface from the target three-dimensional scene to obtain a target isosurface; and acquiring the geometric feature corresponding to the sample scene based on the position relation between the target isosurface and a plurality of three-dimensional voxel center points.

In an optional embodiment, the mapping module 2140 is further configured to perform a dimension reduction process on spherical harmonic parameters corresponding to a plurality of three-dimensional voxel blocks in the sample scene, so as to obtain dimension reduction information corresponding to the plurality of three-dimensional voxel blocks; and splicing the plurality of dimension reduction information to obtain the appearance characteristics corresponding to the sample scene.

In an optional embodiment, the generating module 2150 is further configured to update attributes of the three-dimensional voxel block in the generated mapping scene based on a proximity relationship and a similarity relationship between the perturbed three-dimensional voxel block in the generated mapping scene and the three-dimensional voxel block in the transformed sample scene, to obtain the updated generated mapping scene.

In an optional embodiment, the generating module 2150 is further configured to perform coordinate assignment on a plurality of mapping blocks in the perturbed generated mapping field in the sample scene based on a proximity relationship and a similarity relationship between the three-dimensional voxel blocks in the updated generated mapping scene and the three-dimensional voxel blocks in the transformed sample scene, and obtain the target scene.

In an alternative embodiment, the first obtaining module 2110 is further configured to obtain a plurality of two-dimensional images acquired from different acquisition angles for a specified scene; and acquiring the sample scene through a three-dimensional voxel grid based on the two-dimensional images.

It should be noted that: the three-dimensional scene construction device provided in the above embodiment is only exemplified by the division of the above functional modules, and in practical application, the above functional allocation may be performed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules, so as to complete all or part of the functions described above. In addition, the three-dimensional scene construction device and the three-dimensional scene construction method provided in the above embodiments belong to the same concept, and specific implementation processes thereof are detailed in the method embodiments, and are not repeated here.

Fig. 24 is a schematic diagram showing a structure of a server according to an exemplary embodiment of the present application. The server 2400 includes a central processing unit (Central Processing Unit, CPU) 2401, a system Memory 2404 including a random access Memory (Random Access Memory, RAM) 2402 and a Read Only Memory (ROM) 2403, and a system bus 2405 connecting the system Memory 2404 and the central processing unit 2401. The server 2400 also includes a mass storage device 2406 for storing an operating system 2413, application programs 2414, and other program modules 2415.

The mass storage device 2406 is connected to the central processing unit 2401 through a mass storage controller (not shown) connected to the system bus 2405. The mass storage device 2406 and its associated computer-readable media provide non-volatile storage for the server 2400. That is, mass storage device 2406 may include a computer readable medium (not shown) such as a hard disk or compact disc read only memory (Compact Disc Read Only Memory, CD-ROM) drive.

Computer readable media may include computer storage media and communication media, without generality. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. The system memory 2404 and mass storage 2406 described above may be collectively referred to as memory.

According to various embodiments of the application, server 2400 may also operate with a remote computer connected to the network through a network such as the Internet. That is, the server 2400 may be connected to the network 2412 through a network interface unit 2411 connected to the system bus 2405, or alternatively, the network interface unit 2411 may be used to connect to other types of networks or remote computer systems (not shown).

The memory also includes one or more programs, one or more programs stored in the memory and configured to be executed by the CPU.

Embodiments of the present application also provide a computer device, where the computer device includes a processor and a memory, where the memory stores at least one instruction, at least one program, a code set, or an instruction set, and where the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the three-dimensional scene construction method provided by the foregoing method embodiments.

Embodiments of the present application also provide a computer readable storage medium having at least one instruction, at least one program, a code set, or an instruction set stored thereon, where the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the three-dimensional scene construction method provided by the above method embodiments.

Embodiments of the present application also provide a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the three-dimensional scene construction method according to any one of the above embodiments.

The foregoing description of the preferred embodiments of the present application is not intended to limit the application, but rather, the application is to be construed as limited to the appended claims.

Claims

1. A method of three-dimensional scene construction, the method comprising:

2. The method according to claim 1, wherein the performing coordinate assignment on the plurality of mapping blocks in the perturbed generated mapping field based on the voxel block association relationship between the perturbed three-dimensional voxel block in the generated mapping scene and the three-dimensional voxel block in the sample scene, and obtaining the target scene includes:

based on the adjacent relation and the similarity relation between the disturbed three-dimensional voxel blocks in the generated mapping scene and the three-dimensional voxel blocks in the sample scene, carrying out attribute updating on the three-dimensional voxel blocks in the generated mapping scene to obtain an updated generated mapping scene;

and carrying out coordinate assignment on a plurality of mapping blocks in the perturbed generated mapping field by using the sample scene based on the proximity relation and the similarity relation between the three-dimensional voxel blocks in the updated generated mapping scene and the three-dimensional voxel blocks in the sample scene, and obtaining the target scene.

3. The method according to claim 2, wherein the performing attribute update on the three-dimensional voxel block in the generated mapping scene based on the proximity relation and the similarity relation between the perturbed three-dimensional voxel block in the generated mapping scene and the three-dimensional voxel block in the sample scene to obtain an updated generated mapping scene includes:

extracting a scene block from a three-dimensional voxel block in the sample scene according to a preset scene block size to obtain a plurality of first scene blocks corresponding to the sample scene, wherein the preset scene block size is larger than the three-dimensional voxel block;

extracting the scene blocks from the three-dimensional voxel blocks in the generated mapping scene according to the preset scene block size to obtain a plurality of second scene blocks corresponding to the generated mapping scene;

and based on the proximity relations and the similarity relations between the plurality of first scene blocks and the plurality of second scene blocks, performing attribute updating on the three-dimensional voxel blocks in the generated mapping scene to obtain the updated generated mapping scene.

4. A method according to claim 3, wherein said updating the attributes of the three-dimensional voxel blocks in the generated mapping scene based on the proximity relations and the similarity relations between the plurality of first scene blocks and the plurality of second scene blocks to obtain the updated generated mapping scene comprises:

For a target second scene block in the plurality of second scene blocks, determining n first scene blocks nearest to the target second scene block, and determining scene similarity of the target second scene block and the n first scene blocks respectively corresponding to each other on voxel attributes, wherein n is a positive integer;

determining a first scene block corresponding to the maximum scene similarity as a first updated scene block;

and replacing voxel attribute data of the disturbed three-dimensional voxel block in the target second scene block with voxel attribute data of the disturbed three-dimensional voxel block in the first updated scene block, and obtaining the updated generated mapping scene.

5. The method of claim 3, wherein the assigning coordinates to the plurality of map blocks in the perturbed generated map field with the sample scene based on the proximity and similarity relationships between the three-dimensional voxel blocks in the updated generated map scene and the three-dimensional voxel blocks in the sample scene, and obtaining the target scene, comprises:

for a target second scene block in the plurality of second scene blocks, determining m first scene blocks nearest to the target second scene block, and determining scene similarity of the target second scene block and the m first scene blocks respectively corresponding to each other on voxel attributes, wherein m is a positive integer;

Determining a first scene block corresponding to the maximum scene similarity as a second updated scene block;

determining a target mapping block corresponding to the target second scene block from the perturbed generated mapping field;

and taking the coordinate data of the disturbed three-dimensional voxel block in the second updated scene block as the coordinate data of the target mapping block, and obtaining the target scene.

6. The method according to any one of claims 2 to 5, wherein the assigning coordinates to the plurality of mapping blocks in the perturbed generated mapping field in the sample scene based on the proximity relationship and the similarity relationship between the three-dimensional voxel blocks in the updated generated mapping scene and the three-dimensional voxel blocks in the sample scene, and obtaining the target scene includes:

based on the proximity relation and the similarity relation between the three-dimensional voxel blocks in the updated generated mapping scene and the three-dimensional voxel blocks in the sample scene, carrying out coordinate assignment on a plurality of mapping blocks in the perturbed generated mapping field by the sample scene to obtain an updated generated mapping field;

and mapping the sample scene to the updated generated mapping field to obtain the target scene.

7. The method of claim 6, wherein the method further comprises:

determining a designated scene, wherein the designated scene is used for representing a scene generated to obtain the sample scene;

performing scene expression on the specified scene with a plurality of scene resolutions to obtain sample scenes respectively corresponding to the plurality of scene resolutions; acquiring a first sample scene corresponding to a first scene resolution from a plurality of sample scenes, wherein the first scene resolution is the lowest scene resolution in the plurality of scene resolutions;

acquiring a first updated generated mapping field based on the first sample scene;

and obtaining the target scene through the sample scene based on the comparison relation between the first updated generated mapping field and the updating condition, wherein the updating condition is a preset condition.

8. The method of claim 7, wherein the obtaining the target scene from the sample scene based on the comparison between the first updated generated mapping field and the update condition comprises:

under the condition that the first updated generated mapping field does not reach an updating condition, carrying out up-sampling operation on the first updated generated mapping field to obtain a first candidate mapping field, wherein the first candidate mapping field corresponds to a second scene resolution, and the second scene resolution is higher than the first scene resolution;

Taking the first candidate mapping field as the perturbed generated mapping field, and obtaining a second updated generated mapping field based on the first candidate mapping field and a second sample scene, wherein the second sample scene corresponds to the second scene resolution;

and obtaining the target scene through the sample scene based on the comparison relation between the second updated generated mapping field and the updating condition.

9. The method of claim 8, wherein the method further comprises:

responding to the ith updated generated mapping field reaching the updating condition, taking the ith updated generated mapping field as a target generated mapping field, wherein the ith updated generated mapping field corresponds to the (i+1) th scene resolution, and i is a positive integer greater than 1;

and mapping the sample scene to the target to generate a mapping field to obtain the target scene, wherein the target scene comprises a scene obtained by mapping the sample scene corresponding to the (i+1) th scene resolution to the target to generate the mapping field.

10. The method according to claim 9, wherein the method further comprises:

Taking the i-1 candidate mapping field as the perturbed generated mapping field, and obtaining the i updated generated mapping field based on the i-1 candidate mapping field and the i sample scene;

and carrying out iterative updating on the generated mapping field after the ith updating until the target generated mapping field is obtained.

11. The method according to any one of claims 2 to 5, wherein mapping the sample scene onto the perturbed generated mapping field to obtain a generated mapping scene comprises:

performing feature transformation on the sample scene to obtain a transformed sample scene;

mapping the transformed sample scene to the perturbed generated mapping field to obtain the generated mapping scene.

12. The method of claim 11, wherein the performing feature transformation on the sample scene to obtain a transformed sample scene comprises:

performing feature processing on voxel density values respectively corresponding to a plurality of three-dimensional voxel blocks in the sample scene to obtain geometric features corresponding to the sample scene;

performing feature compression on spherical harmonic parameters respectively corresponding to a plurality of three-dimensional voxel blocks in the sample scene to obtain appearance features corresponding to the sample scene;

And constructing and obtaining the transformed sample scene based on the geometric features and the appearance features.

13. The method of claim 11, wherein the performing attribute update on the three-dimensional voxel block in the generated mapping scene based on the proximity relationship and the similarity relationship between the perturbed three-dimensional voxel block in the generated mapping scene and the three-dimensional voxel block in the sample scene to obtain an updated generated mapping scene comprises:

based on the adjacent relation and the similarity relation between the disturbed three-dimensional voxel blocks in the generated mapping scene and the three-dimensional voxel blocks in the transformed sample scene, carrying out attribute updating on the three-dimensional voxel blocks in the generated mapping scene to obtain the updated generated mapping scene;

the step of assigning coordinates to a plurality of mapping blocks in the perturbed generated mapping field by using the sample scene based on the proximity relation and the similarity relation between the three-dimensional voxel blocks in the updated generated mapping scene and the three-dimensional voxel blocks in the sample scene, and obtaining a target scene, includes:

and carrying out coordinate assignment on a plurality of mapping blocks in the perturbed generated mapping field by using the sample scene based on the proximity relation and the similarity relation between the three-dimensional voxel blocks in the updated generated mapping scene and the three-dimensional voxel blocks in the transformed sample scene, and obtaining the target scene.

14. A three-dimensional scene building apparatus, the apparatus comprising:

15. A computer device comprising a processor and a memory, wherein the memory has stored therein at least one program that is loaded and executed by the processor to implement the three-dimensional scene construction method of any of claims 1 to 13.

16. A computer-readable storage medium, wherein at least one program is stored in the storage medium, the at least one program being loaded and executed by a processor to implement the three-dimensional scene construction method according to any one of claims 1 to 13.

17. A computer program product comprising a computer program which, when executed by a processor, implements the three-dimensional scene construction method according to any of claims 1 to 13.