CN117132736B

CN117132736B - Stadium modeling method and system based on meta universe

Info

Publication number: CN117132736B
Application number: CN202311388945.0A
Authority: CN
Inventors: 李晓林; 刘祖福; 曾维朝
Original assignee: Shenzhen Guangtong Software Co ltd
Current assignee: Shenzhen Guangtong Software Co ltd
Priority date: 2023-10-25
Filing date: 2023-10-25
Publication date: 2024-02-13
Anticipated expiration: 2043-10-25
Also published as: CN117132736A

Abstract

The invention relates to the technical field of three-dimensional modeling, and discloses a stadium modeling method based on meta universe, which comprises the following steps: cleaning and denoising venue data set data into venue diagram set and corresponding venue model set; selecting venue diagram groups in the venue diagram group one by one as target venue diagram groups, and performing view geometric alignment, context information fusion, dimension coding and feature mapping operation on the target venue diagram groups by using a preset primary modeling model to obtain primary venue point clouds; reconstructing the primary venue point cloud into a standard venue point cloud, and collecting all the standard venue point clouds into a venue point cloud set; and iteratively training the primary modeling model into a venue modeling model by using the venue point cloud set and the venue model set, obtaining a real-time stadium graph group, and generating a real-time stadium model corresponding to the real-time stadium graph group by using the venue modeling model. The invention further provides a stadium modeling system based on the meta universe. The invention can improve the modeling efficiency of stadium.

Description

Stadium modeling method and system based on meta universe

Technical Field

The invention relates to the technical field of three-dimensional modeling, in particular to a stadium modeling method and system based on meta universe.

Background

The three-dimensional modeling means that objects, scenes, buildings and the like in the virtual world and the real world are digitally modeled to create a virtual environment with extremely strong sense of reality and interactivity, and along with the rise of metauniverse concepts, the three-dimensional modeling technology for the real scene is greatly developed, and stadiums are used as important scene buildings in the real world, and the metauniverse three-dimensional modeling for the stadiums is widely applied.

The existing stadium modeling method based on meta universe is mostly a modeling method based on point cloud data acquisition, namely, the actual point cloud data and building parameters of the stadium are acquired through technologies such as wireless radar, three-dimensional modeling is carried out according to the actual point cloud data and the building parameters, in practical application, the modeling method based on the point cloud data acquisition is high in limitation, high in equipment requirement and long in modeling period, and the efficiency in stadium modeling is possibly low.

Disclosure of Invention

The invention provides a stadium modeling method and system based on the universe, which mainly aim to solve the problem of low efficiency in stadium modeling.

In order to achieve the above purpose, the stadium modeling method based on meta universe provided by the invention comprises the following steps:

performing data cleaning and denoising on a pre-acquired venue data set to obtain a standard venue data set, and splitting the standard venue data set into a venue diagram set and a corresponding venue model set;

selecting the venue diagram groups in the venue diagram group one by one as a target venue diagram group, performing visual angle geometric alignment operation on the target venue diagram group by using a preset primary modeling model to obtain aligned venue pictures, wherein the performing visual angle geometric alignment operation on the target venue diagram group by using the preset primary modeling model to obtain aligned venue pictures comprises the following steps: performing primary convolution operation on the target venue graph group by using a preset primary modeling model to obtain a venue feature group; performing geometric feature extraction operation on the venue feature set to obtain a venue geometric feature set; performing grid coordinate transformation on the venue feature group according to the venue geometric feature group to obtain venue geometric grids; updating the venue geometry grid to an interpolated geometry grid using a grid interpolation algorithm:

wherein I is _(x,y) Refers to the gray value of the pixel point with the coordinates of (x, y) in the interpolation geometric grid, I ₍₎ As a function of the gray value of the gray scale,is the lateral pixel distance between the pixel point (x, y) and the nearest pixel point in the venue geometric grid,/the pixel distance is equal to the sum of the pixel points (x, y)>Is the pixel longitudinal distance between a pixel point (x, y) and the nearest pixel point in the venue geometric grid, l (x) is the abscissa of the pixel point which is positioned on the left side of the pixel point (x, y) and is nearest to the pixel point (x, y), f (y) is the ordinate of the pixel point which is positioned on the upper side of the pixel point (x, y) and is nearest to the pixel point (x, y) in the venue geometric grid, c (y) is the ordinate of the pixel point which is positioned on the lower side of the pixel point (x, y) and is nearest to the pixel point (x, y) in the venue geometric grid, r (x) is the abscissa of the pixel point which is positioned on the right side of the pixel point (x, y) and is nearest to the pixel point (x, y) in the venue geometric grid; performing pixel mapping operation on the venue feature group by using the interpolation geometric grid to obtain venue transformation features; performing transposition convolution operation on the venue transformation characteristics to obtain aligned venue pictures;

sequentially carrying out context information fusion and dimension coding operation on the aligned venue pictures to obtain coded venue features, and mapping the coded venue features into primary venue point clouds;

Extracting building distribution characteristics from the primary venue point cloud, reconstructing the primary venue point cloud into standard venue point cloud by utilizing the building distribution characteristics, and collecting all standard venue point clouds into venue point cloud sets;

and performing iterative training on the primary modeling model by using the venue point cloud set and the venue model set to obtain a venue modeling model, obtaining a real-time stadium graph group, and generating a real-time stadium model corresponding to the real-time stadium graph group by using the venue modeling model.

Optionally, the performing data cleaning and denoising on the pre-acquired venue data set to obtain a standard venue data set includes:

screening out equal-size data sets from the venue data set according to the data size;

mapping each data in the equal-size data set by utilizing a hash function to obtain a hash value set;

extracting a repeated hash value item from the hash value group, and carrying out data deduplication on the venue data set by using the repeated hash value item to obtain a deduplication venue data set;

splitting the deduplication venue dataset into a deduplication venue atlas and a deduplication venue model set;

sequentially carrying out denoising enhancement and scene separation operation on each venue picture in the duplicate venue picture set to obtain a separated venue picture set;

Performing scale unified operation on each venue model in the de-duplication venue model set to obtain a de-noised venue model set;

and collecting the separated venue atlas and the denoising venue model set into a standard venue data set.

Optionally, the sequentially performing denoising enhancement and scene separation operations on each venue picture in the deduplication venue picture set to obtain a separated venue picture set, including:

selecting venue pictures in the duplicate removal venue picture set one by one as target duplicate removal venue pictures, and performing median filtering on the target duplicate removal venue pictures to obtain target duplicate removal venue pictures;

performing gray enhancement operation on the target denoising stadium picture to obtain a target enhancement stadium picture;

performing multi-level convolution operation on the target enhanced venue picture to obtain enhanced venue characteristics;

sequentially performing linear activation and full connection operation on the enhanced venue features to obtain venue position information;

performing edge fitting on the target enhanced venue picture to obtain a target venue outline, and generating a target venue mask according to the venue position information and the target venue outline;

and performing mask operation on the target enhanced picture by using the target venue mask to obtain target separated venue pictures, and collecting all the target separated venue pictures into a separated venue atlas.

Optionally, the sequentially performing context information fusion and dimension coding operations on the aligned venue pictures to obtain coded venue features, including:

performing equal-ratio segmentation on the aligned venue pictures according to the sequence from top to bottom to obtain an aligned venue block sequence;

performing multi-level convolution on the aligned venue block sequences to obtain venue texture feature sets, and performing semantic coding on the venue texture feature sets to obtain venue semantic feature sets;

fusing the venue semantic feature set and the venue texture feature set into a primary venue feature set, and performing position coding on the primary venue feature set according to the aligned venue block sequence to obtain a standard venue feature set;

and calculating a fusion venue feature set corresponding to the standard venue feature set according to the primary venue feature set by using the following context fusion formula:

wherein R is _i Refers to the ith fusion venue feature in the fusion venue feature set, i is a feature sequence number, softmax is a normalization function, Q _i Is the i standard venue feature in the standard venue feature set, is a dot product symbol, alpha, beta, gamma are attention coefficient matrices preset in the context fusion formula, w () is a dimension function, w _(Qiβ) Refers to Q _i Feature dimension of beta vector, T is transposed symbol, C _i Is an ith primary venue feature in the set of primary venue features;

and splicing the fusion venue feature set into venue integral features, and sequentially performing multistage full-connection and implicit coding operation on the venue integral features to obtain coded venue features.

Optionally, the mapping the encoded venue feature to a primary venue point cloud comprises:

performing multistage convolution operation on the coded venue features by utilizing convolution cores with different sizes to obtain decoded venue features;

performing multistage deconvolution operation on the decoded venue features by utilizing convolution cores with different channel numbers to obtain extended venue features;

performing multi-level up-sampling operation on the extended venue features to obtain sampled venue features;

performing voxel activation on the sampling venue feature to obtain an activated venue feature;

and performing channel mapping on the activated venue feature to obtain a primary venue point cloud.

Optionally, the extracting building distribution features from the primary venue point cloud includes:

performing outlier detection and coordinate normalization on the primary venue point cloud in sequence to obtain a secondary venue point cloud;

Respectively extracting point cloud curvature characteristics, point cloud normal characteristics and point cloud density characteristics from the secondary venue point cloud;

performing global dimension reduction fusion on the point cloud curvature characteristic, the point cloud normal characteristic and the point cloud density characteristic to obtain a standard point cloud characteristic;

performing feature clustering on the secondary venue point clouds according to the standard point cloud features to obtain venue point cloud class sets, and splitting the secondary venue point clouds into venue structure point cloud sets according to the venue point cloud class sets;

respectively extracting a building center feature set, a building boundary feature set and a building shape feature set from the venue structure point cloud set;

and fusing the building center feature set, the building boundary feature set and the building shape feature set into a building feature set, and splicing the building feature set into building distribution features.

Optionally, the reconstructing the primary venue point cloud into a standard venue point cloud using the building distribution feature comprises:

performing defect labeling on the primary venue point cloud to obtain a labeled venue point cloud;

sequentially performing full connection, multistage convolution and multistage deconvolution operation on the marked venue point cloud according to the building distribution characteristics to obtain a reconstructed venue point cloud;

Performing point cloud interpolation on the reconstructed venue point cloud to obtain an interpolation venue point cloud;

and carrying out Gaussian smoothing and grid topology operation on the interpolation venue point cloud in sequence to obtain a standard venue point cloud.

Optionally, the performing iterative training on the primary modeling model by using the venue point cloud set and the venue model set to obtain a venue modeling model includes:

selecting venue point clouds in the venue point cloud set one by one as target venue point clouds, and converting the target venue point clouds into a target primary model by utilizing the primary modeling model;

screening out a venue model corresponding to the target venue point cloud from the venue model set to serve as a target venue model, and scaling the target primary model according to the target venue model to obtain a target reconstruction model;

extracting a venue vertex group from the target venue model, and extracting a reconstruction vertex group corresponding to the venue vertex set from the target reconstruction model;

extracting a target venue center from the target venue model, and extracting a target reconstruction center from the target reconstruction model;

performing model matching on the target venue model and the target reconstruction model by using the target venue center and the target reconstruction center to obtain a target overlapping model;

Assembling the venue vertex set, the reconstruction vertex set, the target overlap model, the target venue model and the target reconstruction model into a venue parameter set;

and carrying out iterative training on the primary modeling model according to all venue parameter groups by using a preset reconstruction loss value algorithm to obtain a venue modeling model.

Optionally, the performing iterative training on the primary modeling model by using a preset reconstruction loss value algorithm according to all venue parameter sets to obtain a venue modeling model includes:

calculating the reconstruction loss value of the primary modeling model according to all venue parameter groups by using the following reconstruction loss value algorithm:

wherein D is the reconstruction loss value, E is the model number, E is the total number of models of the stadium model set, sigma is a preset countermeasure coefficient, m () is a volume function, A _e Refers to the e-th target venue model in the venue parameter set, B _e Refers to the e-th target reconstruction model in the venue parameter set, P _e Refers to the e-th target overlapped model in the venue parameter set, N is the point number, N is the total number of vertexes of the venue vertex set, and the total number of vertexes of the venue vertex set is equal to the total number of vertexes of the reconstructed vertex set, d () is the Euclidean distance sign, A _e ⁿ Refers to the nth vertex in the venue vertex group of the target venue model in the e th venue parameter group, B _e ⁿ Refer to the nth vertex in the reconstructed vertex set of the target reconstruction model in the ith set of venue parameters;

judging whether the reconstruction loss value is larger than a preset reconstruction loss threshold value or not;

if yes, updating model parameters of the primary modeling model by using a gradient descent algorithm according to the reconstruction loss value, and returning to the step of performing visual angle geometric alignment operation on the target venue image group by using a preset primary modeling model to obtain aligned venue images;

and if not, taking the updated primary modeling model as a venue modeling model.

In order to solve the above problems, the present invention also provides a stadium modeling system based on metauniverse, the system comprising:

the data cleaning module is used for performing data cleaning and denoising on a pre-acquired venue data set to obtain a standard venue data set, and splitting the standard venue data set into a venue diagram set and a corresponding venue model set;

the view angle alignment module is configured to select venue diagram groups in the venue diagram group one by one as a target venue diagram group, perform view angle geometric alignment operation on the target venue diagram group by using a preset primary modeling model, and obtain aligned venue pictures, where the performing view angle geometric alignment operation on the target venue diagram group by using the preset primary modeling model, and obtain aligned venue pictures, includes: performing primary convolution operation on the target venue graph group by using a preset primary modeling model to obtain a venue feature group; performing geometric feature extraction operation on the venue feature set to obtain a venue geometric feature set; performing grid coordinate transformation on the venue feature group according to the venue geometric feature group to obtain venue geometric grids; updating the venue geometry grid to an interpolated geometry grid using a grid interpolation algorithm:

Wherein I is _(x,y) Refers to the gray value of the pixel point with the coordinates of (x, y) in the interpolation geometric grid, I ₍₎ As a function of the gray value of the gray scale,is the lateral pixel distance between the pixel point (x, y) and the nearest pixel point in the venue geometric grid,/the pixel distance is equal to the sum of the pixel points (x, y)>Is what is shown asA pixel longitudinal distance between a pixel point (x, y) and a nearest pixel point in the venue geometric grid, l (x) is an abscissa of a pixel point which is positioned at the left side of the pixel point (x, y) and is nearest to the pixel point (x, y) in the venue geometric grid, f (y) is an ordinate of a pixel point which is positioned at the upper side of the pixel point (x, y) and is nearest to the pixel point (x, y) in the venue geometric grid, c (y) is an ordinate of a pixel point which is positioned at the lower side of the pixel point (x, y) and is nearest to the pixel point (x, y) in the venue geometric grid, and r (x) is an abscissa of a pixel point which is positioned at the right side of the pixel point (x, y) and is nearest to the pixel point (x, y) in the venue geometric grid; performing pixel mapping operation on the venue feature group by using the interpolation geometric grid to obtain venue transformation features; performing transposition convolution operation on the venue transformation characteristics to obtain aligned venue pictures;

the point cloud generation module is used for sequentially carrying out context information fusion and dimension coding operation on the aligned venue pictures to obtain coded venue features, and mapping the coded venue features into primary venue point clouds;

The point cloud reconstruction module is used for extracting building distribution characteristics from the primary venue point clouds, reconstructing the primary venue point clouds into standard venue point clouds by utilizing the building distribution characteristics, and collecting all the standard venue point clouds into venue point clouds;

and the venue modeling module is used for performing iterative training on the primary modeling model by utilizing the venue point cloud set and the venue model set to obtain a venue modeling model, acquiring a real-time stadium graph group and generating a real-time stadium model corresponding to the real-time stadium graph group by utilizing the venue modeling model.

According to the invention, the standard venue data set is obtained by carrying out data cleaning and denoising on the pre-acquired venue data set, the standard venue data set is split into the venue image group set and the corresponding venue model group, the accuracy of training set data can be improved, the accuracy of subsequent model training is further improved, the venue image groups in the venue image group are selected one by one to serve as target venue image groups, the preset primary modeling model is utilized to carry out visual angle geometric alignment operation on the target venue image groups, aligned venue images are obtained, the consistency of the images under the same visual angle can be ensured, the accuracy of the subsequently reconstructed venue model is further improved, the context information fusion and dimension coding operation are sequentially carried out on the aligned venue images, the coded venue features are obtained, the coded venue features are mapped into primary venue cloud, the context features of the aligned venue images can be combined to generate primary venue cloud, and the fineness and accuracy of the primary venue cloud are improved.

The building distribution characteristics are extracted from the primary stadium point clouds, the primary stadium point clouds are rebuilt into standard stadium point clouds by utilizing the building distribution characteristics, defect reconstruction can be carried out according to the distribution characteristics of each stadium point cloud, a three-dimensional stadium model generated by reconstruction is more reasonable and accurate, model training efficiency and stadium modeling accuracy are improved, the stadium modeling model is obtained by utilizing the stadium point clouds and the stadium model set to carry out iterative training on the primary modeling model, a real-time stadium map group is obtained, the real-time stadium model corresponding to the real-time stadium map group is generated by utilizing the stadium modeling model, and stadium modeling speed and stadium model accuracy can be improved, so that stadium modeling efficiency is improved. Therefore, the stadium modeling method and system based on the meta universe can solve the problem of low efficiency in stadium modeling.

Drawings

FIG. 1 is a schematic flow chart of a stadium modeling method based on meta-universe according to an embodiment of the present invention;

FIG. 2 is a flow chart of a method for extracting and separating venue atlas according to an embodiment of the present invention;

FIG. 3 is a schematic flow chart of extracting building distribution characteristics according to an embodiment of the present invention;

FIG. 4 is a functional block diagram of a meta-universe based stadium modeling system according to an embodiment of the present invention;

the achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.

Detailed Description

It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.

The embodiment of the application provides a stadium modeling method based on meta universe. The execution subject of the metauniverse-based stadium modeling method includes, but is not limited to, at least one of a server, a terminal, and the like, which can be configured to execute the method provided by the embodiment of the application. In other words, the meta-universe based stadium modeling method may be performed by software or hardware installed at a terminal device or a server device, the software may be a blockchain platform. The service end includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, and the like. The server may be an independent server, or may be a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and basic cloud computing services such as big data and artificial intelligence platforms.

Referring to fig. 1, a flow chart of a stadium modeling method based on meta-universe according to an embodiment of the present invention is shown. In this embodiment, the stadium modeling method based on meta-universe includes:

s1, performing data cleaning and denoising on a pre-acquired venue data set to obtain a standard venue data set, and splitting the standard venue data set into a venue diagram set and a corresponding venue model set.

In an embodiment of the present invention, the venue data set includes a plurality of venue data, each venue data includes a photograph of a plurality of angles taken for one venue collected in advance and a corresponding three-dimensional modeling model, the venue group set is a atlas composed of a plurality of venue groups, each venue group is a group of pictures taken for a plurality of angles for one venue, each scene model in the scene model set is a three-dimensional modeling model for one venue, and each scene model in the scene model set corresponds to one venue group in the venue group.

In the embodiment of the present invention, the data cleaning and denoising of the pre-acquired venue data set to obtain a standard venue data set includes:

In detail, the data size refers to a data capacity size of each data item in the venue data set, for example, a venue picture has a data size of 356KB, the equal-sized data set is a data set formed by a plurality of data pairs with equal data sizes, and the hash function may be a hash encoding function of MD5, SHA-1, SHA-256, and the like.

Specifically, extracting the repeated hash value item from the hash value group refers to selecting one item of data in a hash value pair with equal hash values in the hash value group to be collected into the repeated hash value item, and performing data deduplication on the venue data set by using the repeated hash value item to obtain a deduplication venue data set refers to deleting all data corresponding to the repeated hash value item from the venue data set to obtain the deduplication venue data set.

Specifically, referring to fig. 2, the performing denoising enhancement and scene separation operations on each venue picture in the deduplication venue picture set sequentially to obtain a separated venue picture set includes:

s21, selecting venue pictures in the duplicate removal venue picture set one by one as target duplicate removal venue pictures, and performing median filtering on the target duplicate removal venue pictures to obtain target duplicate removal venue pictures;

s22, carrying out gray enhancement operation on the target denoising stadium picture to obtain a target enhancement stadium picture;

s23, performing multi-level convolution operation on the target enhanced venue picture to obtain enhanced venue features;

s24, sequentially performing linear activation and full connection operation on the enhanced venue features to obtain venue position information;

S25, performing edge fitting on the target enhanced venue picture to obtain a target venue outline, and generating a target venue mask according to the venue position information and the target venue outline;

s26, performing mask operation on the target enhanced picture by using the target venue mask to obtain target separated venue pictures, and collecting all the target separated venue pictures into a separated venue atlas.

In detail, the gray level enhancement operation can be performed on the target denoising venue picture by using a nonlinear gray level transformation or histogram equalization method to obtain a target enhancement venue picture, wherein the gray level enhancement refers to that the gray level range of each venue picture in the denoising venue picture is flatter, so that the contrast of the picture is enhanced, the details of the venue picture are improved, and the quality of the subsequently generated venue point cloud is further improved.

Specifically, the enhancement venue features can be sequentially subjected to linear activation and full connection operation by using a full connection layer and a linear activation layer of a multi-layer perceptron model (Multilayer Perceptron, abbreviated as MLP) to obtain venue position information, the target enhancement venue pictures can be subjected to edge fitting by using a canny operator or a sobel operator to obtain a target venue outline, and the scale unification is to convert scales of each model in the deduplication venue model to the same size.

Specifically, the standard venue data set can be split into a venue atlas and a venue model set according to suffix names by using a keyword matching method, venue models in the venue model set are selected one by one to serve as target venue models, target venue name keywords are extracted from the target venue models, venue atlas sets are screened from the venue atlas according to the target venue name keywords, and all venue atlas sets form a venue atlas set.

In the embodiment of the invention, the standard venue data set is obtained by carrying out data cleaning and denoising on the venue data set which is acquired in advance, and the standard venue data set is split into the venue chart set and the corresponding venue model set, so that the accuracy of the training set data can be improved, and the accuracy of subsequent model training is further improved.

S2, selecting the venue diagram groups in the venue diagram group one by one as a target venue diagram group, and performing visual angle geometric alignment operation on the target venue diagram group by using a preset primary modeling model to obtain aligned venue pictures.

In the embodiment of the invention, the primary modeling model is a model for generating a three-dimensional model according to a picture, and the primary modeling model comprises a convolutional neural network layer (Convolutional Neural Network, abbreviated as CNN), a spatial transformation network layer (Spatial Transformer Network, abbreviated as STN), a context-aware coding layer, a speeding-up generation network layer (Voxel Generation Network, abbreviated as VGN), a context-aware decoding layer and a generation countermeasure network layer (Generative Adversarial Network, abbreviated as GAN), wherein the spatial transformation network layer comprises a positioning network (Localization Network), a Grid Generator (Grid Generator) and a Sampler (Sampler).

In the embodiment of the present invention, the performing, by using a preset primary modeling model, a visual angle geometric alignment operation on the target venue image group to obtain an aligned venue image includes:

performing primary convolution operation on the target venue graph group by using a preset primary modeling model to obtain a venue feature group;

performing geometric feature extraction operation on the venue feature set to obtain a venue geometric feature set;

performing grid coordinate transformation on the venue feature group according to the venue geometric feature group to obtain venue geometric grids;

updating the venue geometry grid to an interpolated geometry grid using a grid interpolation algorithm:

wherein I is _(x,y) Refers to the gray value of the pixel point with the coordinates of (x, y) in the interpolation geometric grid, I ₍₎ As a function of the gray value of the gray scale,is the lateral pixel distance between the pixel point (x, y) and the nearest pixel point in the venue geometric grid,/the pixel distance is equal to the sum of the pixel points (x, y)>Is the pixel longitudinal distance between a pixel point (x, y) and the nearest pixel point in the venue geometric grid, l (x) is the abscissa of the pixel point which is positioned on the left side of the pixel point (x, y) and is nearest to the pixel point (x, y), f (y) is the ordinate of the pixel point which is positioned on the upper side of the pixel point (x, y) and is nearest to the pixel point (x, y) in the venue geometric grid, c (y) is the ordinate of the pixel point which is positioned on the lower side of the pixel point (x, y) and is nearest to the pixel point (x, y) in the venue geometric grid, r (x) is the abscissa of the pixel point which is positioned on the right side of the pixel point (x, y) and is nearest to the pixel point (x, y) in the venue geometric grid;

Performing pixel mapping operation on the venue feature group by using the interpolation geometric grid to obtain venue transformation features;

and performing transposition convolution operation on the venue transformation characteristics to obtain an aligned venue picture.

In detail, the venue feature set may be subjected to a geometric feature extraction operation by using multiple full-connection layers or multiple convolution layers in a positioning network (Localization Network), so as to obtain a venue geometric feature set, where each venue geometric feature in the venue geometric feature set includes vector features of geometric transformation parameters such as translation, rotation, scaling, and the like of each venue feature in the venue feature set.

Specifically, the grid interpolation algorithm is utilized to update the venue geometric grid into an interpolation geometric grid, so that a plurality of pictures can be subjected to linear interpolation sampling on the sampling grid, further, the geometric transformation and alignment of images are realized, and the pictures at different angles can be conveniently analyzed and processed under a common viewing angle.

Specifically, a Grid Generator (Grid Generator) may be used to perform Grid coordinate transformation on the venue feature set according to the venue geometric feature set to obtain a venue geometric Grid, and the performing pixel mapping operation on the venue feature set by using the interpolation geometric Grid to obtain a venue transformation feature refers to mapping and arranging individual pixels of the venue feature set according to each pixel coordinate on the interpolation geometric Grid to obtain a venue transformation feature.

In the embodiment of the invention, the venue diagram groups in the venue diagram group set are selected one by one as the target venue diagram group, and the visual angle geometric alignment operation is carried out on the target venue diagram group by utilizing the preset primary modeling model to obtain the aligned venue diagram, so that the consistency of the diagram under the same visual angle can be ensured, and the accuracy of the subsequently reconstructed venue model is further improved.

And S3, carrying out context information fusion and dimension coding operation on the aligned venue pictures in sequence to obtain coded venue features, and mapping the coded venue features into primary venue point clouds.

In the embodiment of the invention, the encoding venue feature is a two-dimensional encoding of an aligned venue picture after three-dimensional encoding, the aligned venue picture is a two-dimensional image feature, and in order to construct a three-dimensional model of a stadium, the two-dimensional image feature is required to be encoded into a three-dimensional point cloud feature.

In the embodiment of the present invention, the performing context information fusion and dimension coding operations on the aligned venue pictures in sequence to obtain coded venue features includes:

In detail, the step of performing equal-ratio segmentation on the aligned venue pictures according to the sequence from top to bottom to obtain an aligned venue block sequence refers to transversely segmenting the aligned venue pictures into a plurality of aligned venue blocks with strip-shaped patterns and consistent sizes from top to bottom, and collecting all the aligned venue blocks into the aligned venue block sequence according to the sequence from top to bottom.

Specifically, the venue texture feature set may be semantically encoded by using a trained multi-layer perceptron model (Multilayer Perceptron, abbreviated as MLP) or a transform model to obtain a venue semantic feature set, the venue semantic feature set and the venue texture feature set may be fused into a primary venue feature set by using a global convolution layer or a weighted fusion algorithm, and the primary venue feature set is subjected to position encoding according to the aligned venue tile sequence to obtain a standard venue feature set, which means that the position encoding is performed according to the sequence number of each primary venue feature in the aligned venue tile sequence in the primary venue feature set.

In detail, by calculating the fusion venue feature set corresponding to the standard venue feature set according to the primary venue feature set by using the context fusion formula, the context feature and the weight of the primary venue feature set in fusion can be dynamically determined according to the context feature of the aligned venue picture, so that more accurate feature fusion and better model performance are realized.

In an embodiment of the present invention, the mapping the encoded venue feature to a primary venue point cloud includes:

In detail, the extended venue feature may be subjected to multi-level upsampling operation by deconvolution or pixel reconstruction to obtain a sampled venue feature, voxel activation may be performed on the sampled venue feature by using activation functions such as a leak ReLU, sigmoid or Tanh to obtain an activated venue feature, and channel mapping may be performed on the activated venue feature by using a plurality of convolution layers to obtain a primary venue point cloud.

In the embodiment of the invention, the context information fusion and the dimension coding operation are sequentially carried out on the aligned venue pictures to obtain the coded venue features, and the coded venue features are mapped into the primary venue point cloud, so that the primary venue point cloud can be generated by combining the context features of the aligned venue pictures, and the fineness and the accuracy of the primary venue point cloud are improved.

And S4, extracting building distribution characteristics from the primary venue point cloud, reconstructing the primary venue point cloud into a standard venue point cloud by utilizing the building distribution characteristics, and collecting all the standard venue point clouds into a venue point cloud set.

In the embodiment of the invention, the primary stadium point cloud corresponds to only part of building point clouds of a corresponding stadium, and in order to complement all point clouds of the stadium, the primary stadium point clouds are required to be expanded according to building distribution characteristics, wherein the building distribution characteristics refer to characteristics such as symmetry, structural distribution and the like of buildings.

In an embodiment of the present invention, referring to fig. 3, the extracting building distribution features from the primary venue point cloud includes:

s31, performing outlier detection and coordinate normalization on the primary venue point cloud in sequence to obtain a secondary venue point cloud;

s32, respectively extracting point cloud curvature characteristics, point cloud normal characteristics and point cloud density characteristics from the secondary venue point clouds;

s33, performing global dimension reduction fusion on the point cloud curvature characteristic, the point cloud normal characteristic and the point cloud density characteristic to obtain a standard point cloud characteristic;

s34, carrying out feature clustering on the secondary venue point clouds according to the standard point cloud features to obtain venue point cloud class sets, and splitting the secondary venue point clouds into venue structure point cloud sets according to the venue point cloud class sets;

S35, respectively extracting a building center feature set, a building boundary feature set and a building shape feature set from the venue structure point cloud set;

s36, fusing the building center feature set, the building boundary feature set and the building shape feature set into a building feature set, and splicing the building feature set into building distribution features.

Specifically, the method of local outlier factor or Z-Score detection may be used to perform outlier detection on the primary venue point cloud, the method of mean variance method or linear scaling may be used to perform coordinate normalization operation on the primary venue point cloud to obtain a secondary venue point cloud, a least square method or Normal Change method (Normal Change) may be used to extract point cloud curvature features from the secondary venue point cloud, a principal component analysis method or Normal estimation method may be used to extract point cloud Normal features from the secondary venue point cloud, and a K-nearest neighbor method or gridding method may be used to extract point cloud density features from the secondary venue point cloud.

In detail, the secondary venue point clouds may be subjected to feature clustering according to the standard point cloud features by using a kmeans clustering algorithm to obtain a venue point cloud class set, a building center feature set may be extracted from the venue structure point cloud set by using a region growing method or a clustering center method, a building boundary feature set may be extracted from the venue structure point cloud set by using a Convex Hull method (Convex Hull) or an Alpha Shape method (Alpha Shape), and a building Shape feature set may be extracted from the venue structure point cloud set by using a Shape descriptor method (Shape Descriptors) or a face shaping method (Meshin).

In detail, the reconstructing the primary venue point cloud into a standard venue point cloud using the building distribution feature includes:

Specifically, a multi-layer perceptron model (Multilayer Perceptron, abbreviated as MLP) may be used to perform defect labeling on the primary venue point cloud to obtain a labeled venue point cloud, a generating countermeasure network layer (Generative Adversarial Network, abbreviated as GAN) may be used to perform full connection, multi-stage convolution and multi-stage deconvolution operations on the labeled venue point cloud according to the building distribution characteristics in sequence to obtain a reconstructed venue point cloud, that is, the reconstructed venue point cloud is positioned according to the building center characteristics of the building distribution characteristics, defect matching is performed on the reconstructed venue point cloud according to the building boundary characteristics, multi-stage deconvolution operation is performed on the labeled venue point cloud according to the building shape characteristics, and point cloud interpolation may be performed on the reconstructed venue point cloud by using a nearest neighbor interpolation method or a bilinear interpolation method to obtain an interpolation venue point cloud.

In the embodiment of the invention, the building distribution characteristics are extracted from the primary venue point clouds, the primary venue point clouds are rebuilt into the standard venue point clouds by utilizing the building distribution characteristics, and defect reconstruction can be carried out according to the distribution characteristics of each venue point cloud, so that the three-dimensional venue model generated by reconstruction is more reasonable and accurate, and the training efficiency of the model and the modeling accuracy of the stadium are improved.

S5, performing iterative training on the primary modeling model by using the venue point cloud set and the venue model set to obtain a venue modeling model, obtaining a real-time stadium graph group, and generating a real-time venue model corresponding to the real-time stadium graph group by using the venue modeling model.

In the embodiment of the invention, each venue point cloud in the venue point cloud set corresponds to each venue model in the venue model set one by one, and in order to improve the coincidence degree of the generated venue point cloud and the corresponding venue model, the primary modeling model needs to be trained, so that the modeling accuracy is improved.

In the embodiment of the present invention, the performing iterative training on the primary modeling model by using the venue point cloud set and the venue model set to obtain a venue modeling model includes:

Specifically, the target venue point cloud can be converted into a target primary model by using algorithms such as Poisson reconstruction and Marching cube reconstruction of the primary modeling model, scaling is performed on the target primary model according to the target venue model, and scaling is performed on the target primary model according to a scale of the target venue model to obtain a target reconstruction model.

In detail, all model vertexes can be extracted from the target venue model by using a triangular mesh reconstruction method and collected into a venue vertex group, and vertexes corresponding to all vertexes in the venue vertexes are extracted from the target reconstruction model and collected into a reconstruction vertex set.

Specifically, a bounding box method or a set averaging method may be used to extract a target venue center from the target venue model, extract a target reconstruction center from the target reconstruction model, and perform model matching on the target venue model and the target reconstruction model by using the target venue center and the target reconstruction center, where the target overlapping model is obtained by overlapping the target venue center and the target reconstruction center, and an overlapping portion of the target venue model and the target reconstruction model is used as a target overlapping model.

In detail, the performing iterative training on the primary modeling model by using a preset reconstruction loss value algorithm according to all venue parameter sets to obtain a venue modeling model includes:

wherein D is the reconstruction loss value, E is the model number, E is the total number of models of the stadium model set, sigma is a preset countermeasure coefficient, m () is a volume function, A _e Refers to the e-th target venue model in the venue parameter set, B _e Refers to the e-th target reconstruction model in the venue parameter set, P _e Refers to the e-th target overlapped model in the venue parameter set, N is the point number, N is the total number of vertexes of the venue vertex set, and the total number of vertexes of the venue vertex set is equal to the total number of vertexes of the reconstructed vertex set, d () is the Euclidean distance sign, A _e ⁿ Refers to the venue of the target venue model in the e-th venue parameter setThe nth vertex in the vertex group, B _e ⁿ Refer to the nth vertex in the reconstructed vertex set of the target reconstruction model in the ith set of venue parameters;

Specifically, the reconstruction loss value of the primary modeling model is calculated according to all venue parameter groups by using the reconstruction loss value algorithm, so that the reconstruction loss value of the primary modeling model can be calculated according to the corresponding point deviation distance between each target venue model and the corresponding target reconstruction model and the proportion of the model overlapping part, the representation range of the reconstruction loss value is further improved, and the accuracy of model training is improved.

Specifically, the real-time stadium map group refers to a map group of a stadium shot in real time, and the real-time stadium model is a three-dimensional model of the stadium corresponding to the real-time stadium map group.

In the embodiment of the invention, the venue modeling model is obtained by performing iterative training on the primary modeling model by utilizing the venue point cloud set and the venue model set, a real-time stadium graph group is obtained, and the real-time stadium model corresponding to the real-time stadium graph group is generated by utilizing the venue modeling model, so that the speed of stadium modeling and the accuracy of the generated stadium model can be improved, and the efficiency of stadium modeling is improved.

The building distribution characteristics are extracted from the primary stadium point clouds, the primary stadium point clouds are rebuilt into standard stadium point clouds by utilizing the building distribution characteristics, defect reconstruction can be carried out according to the distribution characteristics of each stadium point cloud, a three-dimensional stadium model generated by reconstruction is more reasonable and accurate, model training efficiency and stadium modeling accuracy are improved, the stadium modeling model is obtained by utilizing the stadium point clouds and the stadium model set to carry out iterative training on the primary modeling model, a real-time stadium map group is obtained, the real-time stadium model corresponding to the real-time stadium map group is generated by utilizing the stadium modeling model, and stadium modeling speed and stadium model accuracy can be improved, so that stadium modeling efficiency is improved. Therefore, the stadium modeling method based on the meta universe can solve the problem of low efficiency in stadium modeling.

As shown in FIG. 4, a functional block diagram of a meta-universe based stadium modeling system is provided in accordance with an embodiment of the present invention.

The stadium modeling system 100 based on meta-universe of the present invention may be installed in an electronic device. Depending on the functionality implemented, the metauniverse-based stadium modeling system 100 may include a data cleansing module 101, a perspective alignment module 102, a point cloud generation module 103, a point cloud reconstruction module 104, and a stadium modeling module 105. The module of the invention, which may also be referred to as a unit, refers to a series of computer program segments, which are stored in the memory of the electronic device, capable of being executed by the processor of the electronic device and of performing a fixed function.

In the present embodiment, the functions concerning the respective modules/units are as follows:

the data cleaning module 101 is configured to perform data cleaning and denoising on a previously acquired venue data set to obtain a standard venue data set, and split the standard venue data set into a venue diagram set and a corresponding venue model set;

the view angle alignment module 102 is configured to select, one by one, a venue group in the venue group set as a target venue group, perform a view angle geometric alignment operation on the target venue group by using a preset primary modeling model, and obtain an aligned venue picture, where the performing, by using the preset primary modeling model, the view angle geometric alignment operation on the target venue group, and obtain the aligned venue picture, includes: performing primary convolution operation on the target venue graph group by using a preset primary modeling model to obtain a venue feature group; performing geometric feature extraction operation on the venue feature set to obtain a venue geometric feature set; performing grid coordinate transformation on the venue feature group according to the venue geometric feature group to obtain venue geometric grids; updating the venue geometry grid to an interpolated geometry grid using a grid interpolation algorithm:

the point cloud generating module 103 is configured to sequentially perform context information fusion and dimension encoding operations on the aligned venue pictures to obtain encoded venue features, and map the encoded venue features into primary venue point clouds;

The point cloud reconstruction module 104 is configured to extract building distribution features from the primary venue point clouds, reconstruct the primary venue point clouds into standard venue point clouds by using the building distribution features, and collect all the standard venue point clouds into venue point clouds;

the venue modeling module 105 is configured to perform iterative training on the primary modeling model by using the venue point cloud set and the venue model set to obtain a venue modeling model, obtain a real-time stadium map set, and generate a real-time venue model corresponding to the real-time stadium map set by using the venue modeling model.

In detail, each module in the meta-universe-based stadium modeling system 100 in the embodiment of the present invention adopts the same technical means as the above-mentioned meta-universe-based stadium modeling method in fig. 1 to 3, and can produce the same technical effects, which are not repeated here.

In the several embodiments provided by the present invention, it should be understood that the disclosed apparatus, system and method may be implemented in other manners. For example, the system embodiments described above are merely illustrative, e.g., the division of the modules is merely a logical function division, and other manners of division may be implemented in practice.

The modules described as separate components may or may not be physically separate, and components shown as modules may or may not be physical units, may be located in one place, or may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional module in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units can be realized in a form of hardware or a form of hardware and a form of software functional modules.

It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof.

The present embodiments are, therefore, to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference signs in the claims shall not be construed as limiting the claim concerned.

The embodiment of the application can acquire and process the related data based on the artificial intelligence technology. Among these, artificial intelligence (Artificial Intelligence, AI) is the theory, method, technique and application system that uses a digital computer or a digital computer-controlled machine to simulate, extend and extend human intelligence, sense the environment, acquire knowledge and use knowledge to obtain optimal results.

Furthermore, it is evident that the word "comprising" does not exclude other elements or steps, and that the singular does not exclude a plurality. Multiple units or systems set forth in the system embodiments may also be implemented by one unit or system in software or hardware. The terms first, second, etc. are used to denote a name, but not any particular order.

Finally, it should be noted that the above-mentioned embodiments are merely for illustrating the technical solution of the present invention and not for limiting the same, and although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications and equivalents may be made to the technical solution of the present invention without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A stadium modeling method based on meta-universe, the method comprising:

S1: performing data cleaning and denoising on a pre-acquired venue data set to obtain a standard venue data set, and splitting the standard venue data set into a venue diagram set and a corresponding venue model set;

s2: selecting the venue diagram groups in the venue diagram group one by one as a target venue diagram group, performing visual angle geometric alignment operation on the target venue diagram group by using a preset primary modeling model to obtain aligned venue pictures, wherein the performing visual angle geometric alignment operation on the target venue diagram group by using the preset primary modeling model to obtain aligned venue pictures comprises the following steps:

s21: performing primary convolution operation on the target venue graph group by using a preset primary modeling model to obtain a venue feature group;

s22: performing geometric feature extraction operation on the venue feature set to obtain a venue geometric feature set;

s23: performing grid coordinate transformation on the venue feature group according to the venue geometric feature group to obtain venue geometric grids;

s24: updating the venue geometry grid to an interpolated geometry grid using a grid interpolation algorithm:

s25: performing pixel mapping operation on the venue feature group by using the interpolation geometric grid to obtain venue transformation features;

s26: performing transposition convolution operation on the venue transformation characteristics to obtain aligned venue pictures;

s3: sequentially carrying out context information fusion and dimension coding operation on the aligned venue pictures to obtain coded venue features, and mapping the coded venue features into primary venue point clouds;

S4: extracting building distribution characteristics from the primary venue point cloud, reconstructing the primary venue point cloud into standard venue point cloud by utilizing the building distribution characteristics, and collecting all standard venue point clouds into venue point cloud sets;

s5: and performing iterative training on the primary modeling model by using the venue point cloud set and the venue model set to obtain a venue modeling model, obtaining a real-time stadium graph group, and generating a real-time stadium model corresponding to the real-time stadium graph group by using the venue modeling model.

2. The meta-universe based stadium modeling method of claim 1, wherein the performing data cleaning and denoising on the pre-acquired stadium data set to obtain a standard stadium data set comprises:

3. The meta-universe based stadium modeling method of claim 2, wherein the sequentially performing denoising enhancement and scene separation operations on each stadium picture in the deduplication stadium picture set to obtain a separated stadium picture set comprises:

4. The meta-universe based stadium modeling method of claim 1, wherein the sequentially performing context information fusion and dimension encoding operations on the aligned stadium pictures to obtain encoded stadium features comprises:

5. The meta-universe based stadium modeling method of claim 1, wherein the mapping the encoded stadium features into a primary stadium point cloud comprises:

6. The meta-universe based stadium modeling method of claim 1, wherein the extracting building distribution features from the primary stadium point cloud comprises:

7. The meta-universe based stadium modeling method of claim 1, wherein the reconstructing the primary stadium point cloud into a standard stadium point cloud using the building distribution features comprises:

8. The meta-universe based stadium modeling method of claim 1, wherein the iteratively training the primary modeling model using the stadium point cloud set and the stadium model set to obtain a stadium modeling model comprises:

9. The meta-universe based stadium modeling method of claim 8, wherein the iterative training of the primary modeling model according to all stadium parameter sets using a preset reconstruction loss value algorithm to obtain a stadium modeling model includes:

Wherein D is the reconstruction loss value, E is the model number, E is the stadium modelModel total number of model set, sigma is preset countermeasure coefficient, m () is volume function, A _e Refers to the e-th target venue model in the venue parameter set, B _e Refers to the e-th target reconstruction model in the venue parameter set, P _e Refers to the e-th target overlapped model in the venue parameter set, N is the point number, N is the total number of vertexes of the venue vertex set, and the total number of vertexes of the venue vertex set is equal to the total number of vertexes of the reconstructed vertex set, d () is the Euclidean distance sign, A _e ⁿ Refers to the nth vertex in the venue vertex group of the target venue model in the e th venue parameter group, B _e ⁿ Refer to the nth vertex in the reconstructed vertex set of the target reconstruction model in the ith set of venue parameters;

10. A meta-universe based stadium modeling system, the system comprising: