CN114663579A - Twin three-dimensional model generation method and device, electronic device and storage medium - Google Patents
Twin three-dimensional model generation method and device, electronic device and storage medium Download PDFInfo
- Publication number
- CN114663579A CN114663579A CN202210135449.3A CN202210135449A CN114663579A CN 114663579 A CN114663579 A CN 114663579A CN 202210135449 A CN202210135449 A CN 202210135449A CN 114663579 A CN114663579 A CN 114663579A
- Authority
- CN
- China
- Prior art keywords
- module
- dimensional
- twin
- basic
- picture
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Processing Or Creating Images (AREA)
Abstract
The invention provides a twin three-dimensional model generation method, a twin three-dimensional model generation device, an electronic device and a storage medium, wherein the method comprises the following steps: acquiring a picture to be processed and a three-dimensional basic model database; carrying out module segmentation and feature detection on the picture to be processed to obtain a plurality of modules corresponding to the picture to be processed and module picture features corresponding to the modules; recalling a plurality of basic three-dimensional models corresponding to the modules in a three-dimensional basic model database based on the module picture characteristics, and respectively processing and rendering the basic three-dimensional models to obtain a plurality of two-dimensional pictures corresponding to the basic three-dimensional models; determining a twin basic three-dimensional model corresponding to the module based on the module and the plurality of two-dimensional pictures, wherein the twin basic three-dimensional model is the basic three-dimensional model most similar to the module; and obtaining a twin three-dimensional model corresponding to the picture to be processed based on the twin basic three-dimensional models corresponding to the modules. The modeling accuracy of the twin model is improved through the method and the device.
Description
Technical Field
The present invention relates to the field of model generation technologies, and in particular, to a twin three-dimensional model generation method and apparatus, an electronic device, and a storage medium.
Background
With the development of digital twin technology, users often need to model real objects, scenes, and the like to obtain twin models.
The related art can know that the current twin model is often performed for a specific scene, and the generated twin model has low accuracy.
Disclosure of Invention
The invention provides a twin three-dimensional model generation method and device, electronic equipment and a storage medium, which are used for overcoming the defect of low accuracy of a twin model generated in the prior art, realizing the generation of a digital twin three-dimensional model with higher similarity to a picture to be processed through multi-level recall, and greatly improving the modeling accuracy of the twin model.
The invention provides a twin three-dimensional model generation method, which comprises the following steps: acquiring a picture to be processed and a three-dimensional basic model database; respectively carrying out module segmentation and feature detection on the picture to be processed to obtain a plurality of modules corresponding to the picture to be processed and module picture features corresponding to the modules, wherein the module picture features at least comprise text labels, geometric features, view angle features and picture texture features; recalling a plurality of basic three-dimensional models corresponding to the modules in the three-dimensional basic model database based on the module picture characteristics, and respectively carrying out image processing on the basic three-dimensional models to obtain a plurality of two-dimensional pictures corresponding to the basic three-dimensional models; determining a twin basic three-dimensional model corresponding to the module based on the module and a plurality of the two-dimensional pictures, wherein the twin basic three-dimensional model is the basic three-dimensional model most similar to the module; and obtaining a twin three-dimensional model corresponding to the picture to be processed based on each twin basic three-dimensional model corresponding to each module.
According to the twin three-dimensional model generation method provided by the present invention, the module picture features further include an inter-module tree-level relationship, and the module partitioning and feature detection are respectively performed on the to-be-processed picture to obtain a plurality of modules corresponding to the to-be-processed picture and module picture features corresponding to the modules, including: and respectively carrying out hierarchical module segmentation and feature detection on the picture to be processed to obtain a plurality of modules corresponding to the picture to be processed and inter-module tree level relations corresponding to the modules, wherein the inter-module tree level relations represent the position relations among different modules.
According to a twin three-dimensional model generation method provided by the present invention, the recalling a plurality of basic three-dimensional models corresponding to the module in the three-dimensional basic model database based on the module picture feature includes: recalling a plurality of base three-dimensional models corresponding to the module in the three-dimensional base model database based on the text labels and the geometric features; the image processing of the basic three-dimensional model to obtain a plurality of two-dimensional pictures corresponding to the plurality of basic three-dimensional models includes: and respectively carrying out visual angle adjustment and texture rendering on the basic three-dimensional model based on the visual angle characteristics and the picture texture characteristics to obtain a plurality of two-dimensional pictures corresponding to the plurality of basic three-dimensional models.
According to the twin three-dimensional model generation method provided by the invention, the determining the twin basic three-dimensional model corresponding to the module based on the module and the two-dimensional pictures comprises the following steps: respectively determining similarity values of the module and the two-dimensional pictures through similarity calculation and respectively determining similarity recommendation values of the module and the two-dimensional pictures through collaborative filtering calculation based on the module and the two-dimensional pictures; generating a candidate three-dimensional model probability distribution corresponding to the module based on the similarity value and the similarity recommendation value; determining a twin base three-dimensional model corresponding to the module based on the candidate three-dimensional model probability distribution.
According to the twin three-dimensional model generation method provided by the invention, after obtaining a plurality of two-dimensional pictures corresponding to a plurality of basic three-dimensional models, the method further comprises the following steps: determining association probability between associated modules based on an inter-module co-occurrence constraint model of the conditional random field; generating a candidate three-dimensional model probability distribution corresponding to the module based on the similarity value and the similarity recommendation value, including: and generating a candidate three-dimensional model probability distribution corresponding to the module based on the similarity value, the similarity recommendation value and the association probability.
According to the twin three-dimensional model generation method provided by the present invention, the generating a candidate three-dimensional model probability distribution corresponding to the module based on the similarity value, the similarity recommendation value, and the association probability includes: and determining the probability distribution of the candidate three-dimensional model corresponding to the module based on the similarity value, the similarity recommendation value and the weighted average value of the association probability, wherein the weight value in the weighting calculation process is a hyperparameter.
According to the twin three-dimensional model generation method provided by the invention, the determining the twin basic three-dimensional model corresponding to the module based on the module and the two-dimensional pictures comprises the following steps: and in response to the user selecting a target two-dimensional picture from the plurality of two-dimensional pictures, determining a target basic three-dimensional model as a twin basic three-dimensional model corresponding to the module, wherein the target basic three-dimensional model is a basic three-dimensional model corresponding to the target two-dimensional picture.
According to the twin three-dimensional model generation method provided by the invention, the twin three-dimensional model corresponding to the picture to be processed is obtained based on each twin basic three-dimensional model corresponding to each module, and the method comprises the following steps: and obtaining a twin three-dimensional model corresponding to the picture to be processed based on the tree level relation among the modules and the twin basic three-dimensional models corresponding to the modules.
The present invention also provides a twin three-dimensional model generation apparatus, the apparatus comprising: the acquisition module is used for acquiring a picture to be processed and a three-dimensional basic model database; the detection module is used for respectively carrying out module segmentation and feature detection on the picture to be processed to obtain a plurality of modules corresponding to the picture to be processed and module picture features corresponding to the modules, wherein the module picture features at least comprise text labels, geometric features, view angle features and picture texture features; the recall module is used for recalling a plurality of basic three-dimensional models corresponding to the module in the three-dimensional basic model database based on the module picture characteristics, and respectively carrying out image processing on the basic three-dimensional models to obtain a plurality of two-dimensional pictures corresponding to the basic three-dimensional models; a processing module for determining a twin basic three-dimensional model corresponding to the module based on the module and the plurality of two-dimensional pictures, wherein the twin basic three-dimensional model is the basic three-dimensional model most similar to the module; and the generating module is used for obtaining the twin three-dimensional model corresponding to the picture to be processed based on each twin basic three-dimensional model corresponding to each module.
The invention also provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the twin three-dimensional model generation method as described in any one of the above when executing the program.
The present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a twin three-dimensional model generation method as recited in any of the above.
The invention also provides a computer program product comprising a computer program which, when executed by a processor, implements a twin three-dimensional model generation method as described in any one of the above.
The twin three-dimensional model generation method, the twin three-dimensional model generation device, the electronic equipment and the storage medium provided by the invention have the advantages that the plurality of basic three-dimensional models corresponding to the modules are recalled in a multi-layer mode through the module picture characteristics, the twin basic three-dimensional model corresponding to the modules is determined based on the plurality of two-dimensional pictures corresponding to the plurality of basic three-dimensional models, and the twin three-dimensional model corresponding to the picture to be processed is obtained according to the twin basic three-dimensional model. The digital twin three-dimensional model with higher similarity to the picture to be processed can be generated through multi-level recall, and the modeling accuracy of the twin model is greatly improved.
Drawings
In order to more clearly illustrate the present invention or the technical solutions in the prior art, the drawings used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
FIG. 1 is a schematic flow diagram of a twin three-dimensional model generation method provided by the present invention;
FIG. 2 is a second schematic flow chart of a twin three-dimensional model generation method provided by the present invention;
FIG. 3 is a schematic flow chart of determining a twin basic three-dimensional model corresponding to a module based on the module and a plurality of two-dimensional pictures according to the present invention;
FIG. 4 is a third schematic flow chart of a twin three-dimensional model generation method provided by the present invention;
FIG. 5 is a schematic structural diagram of a twin three-dimensional model generating device provided by the present invention;
fig. 6 is a schematic structural diagram of an electronic device provided in the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The twin three-dimensional model generation method provided by the invention is described below with reference to fig. 1-4.
FIG. 1 is a schematic flow chart of a twin three-dimensional model generation method provided by the present invention.
In an exemplary embodiment of the present invention, as shown in fig. 1, the twin three-dimensional model generation method may include steps 110 to 150, which will be described separately below.
In step 110, a picture to be processed and a three-dimensional basic model database are obtained.
In one embodiment, the picture to be processed may be a single picture to be processed, and the three-dimensional basic model database may be a set of three-dimensional basic models having tags, wherein the tags may be used to identify why the three-dimensional basic model is. In the application process, a single picture to be processed and a three-dimensional basic model database with labels can be obtained.
In step 120, module segmentation and feature detection are performed on the to-be-processed picture, respectively, to obtain a plurality of modules (module 1, module 2 … …, module n) corresponding to the to-be-processed picture, and module picture features corresponding to the modules, where the module picture features at least include text labels, geometric features, view features, and picture texture features.
In an embodiment, module segmentation and feature detection may be performed on a single processed picture, and a plurality of modules corresponding to the picture to be processed and module picture features corresponding to the modules may be obtained at the same time. The modular picture features include but are not limited to text labels, geometric features, view angle features, picture texture features and other information identified by computer vision detection, identification and segmentation algorithms.
In step 130, a plurality of basic three-dimensional models corresponding to the modules are recalled in the three-dimensional basic model database based on the module picture characteristics, and the basic three-dimensional models are respectively subjected to image processing to obtain a plurality of two-dimensional pictures corresponding to the plurality of basic three-dimensional models.
In one embodiment, a plurality of base three-dimensional models corresponding to a module may be recalled in a three-dimensional base model database based on multiple recalls, e.g., based on a plurality of types of module picture features of the module. Among other things, the various types of modular picture features may include text labels and geometric features. In the embodiment, the comprehensiveness and the accuracy of the recalled basic three-dimensional models can be ensured by multi-path recall of the plurality of basic three-dimensional models, and a foundation is laid for improving the modeling accuracy of the twin model.
Further, the basic three-dimensional models can be subjected to image processing respectively to obtain a plurality of two-dimensional pictures corresponding to the plurality of basic three-dimensional models.
In step 140, a twin elementary three-dimensional model corresponding to the module is determined based on the module and the plurality of two-dimensional pictures, wherein the twin elementary three-dimensional model is the most similar elementary three-dimensional model to the module.
In an exemplary embodiment of the present invention, the twin basic three-dimensional model corresponding to each module may be determined based on a plurality of two-dimensional pictures corresponding to the module. Wherein the twin base three-dimensional model represents the base three-dimensional model most similar to the module. Through the embodiment, the most similar basic three-dimensional model can be determined for each module in the picture to be processed, and a foundation is laid for improving the modeling accuracy of the twin model.
In an exemplary embodiment of the present invention, based on the module and the plurality of two-dimensional pictures, determining the twin basic three-dimensional model corresponding to the module may further be implemented by:
and in response to the user selecting a target two-dimensional picture from the plurality of two-dimensional pictures, determining the target basic three-dimensional model as a twin basic three-dimensional model corresponding to the module, wherein the target basic three-dimensional model is the basic three-dimensional model corresponding to the target two-dimensional picture. By the embodiment, user interaction can be realized, the twin basic three-dimensional model corresponding to the module can be rapidly determined through the user interaction, and further the twin model modeling efficiency can be greatly improved.
In step 150, a twin three-dimensional model corresponding to the picture to be processed is obtained based on the twin basic three-dimensional models corresponding to the modules.
In an embodiment, the basic three-dimensional model (corresponding to the twin basic three-dimensional model) most similar to each module can be pieced together according to the hierarchical relation of the pictures and the connection relation can be adjusted, so that the twin three-dimensional model most similar to the shape and the posture texture of the picture to be processed is obtained.
The twin three-dimensional model generation method provided by the invention recalls a plurality of basic three-dimensional models corresponding to the module in a multi-level mode through the characteristics of the module pictures, determines the twin basic three-dimensional model corresponding to the module based on a plurality of two-dimensional pictures corresponding to the plurality of basic three-dimensional models, and obtains the twin three-dimensional model corresponding to the picture to be processed according to the twin basic three-dimensional model. The digital twin three-dimensional model with higher similarity to the picture to be processed can be generated through multi-level recall, and the modeling accuracy of the twin model is greatly improved.
The present invention will be described with reference to the following embodiments, in which a plurality of modules corresponding to pictures to be processed and the process of obtaining picture features of the modules corresponding to the modules are obtained.
In an exemplary embodiment of the present invention, the module picture feature may further include an inter-module tree level relationship, where the inter-module tree level relationship may represent a positional relationship between different modules. In an example, module segmentation and feature detection are respectively performed on a picture to be processed to obtain a plurality of modules corresponding to the picture to be processed, and the module picture features corresponding to the modules can be implemented in the following manner:
and respectively carrying out hierarchical module segmentation and feature detection on the picture to be processed to obtain a plurality of modules corresponding to the picture to be processed and the tree-level relation between the modules corresponding to the modules. In this embodiment, by obtaining the tree-level relationship between the modules corresponding to the modules, a hierarchical relationship guidance is provided for obtaining the twin three-dimensional model corresponding to the picture to be processed based on the twin basic three-dimensional model corresponding to the modules, so that the modeling accuracy of the twin model is improved.
In an exemplary embodiment of the present invention, obtaining the twin three-dimensional model corresponding to the picture to be processed based on each twin basic three-dimensional model corresponding to each module may be implemented in the following manner:
and obtaining a twin three-dimensional model corresponding to the picture to be processed based on the tree-level relation among the modules and the twin basic three-dimensional models corresponding to the modules. In the application process, the twin basic three-dimensional models of the modules can be spliced according to the picture layer hierarchical relation and the connection relation is adjusted, so that the twin three-dimensional model most similar to the shape and the posture texture of the picture to be processed is obtained. In this embodiment, through the tree-level relationship between the modules, a hierarchical relationship guidance is provided for obtaining a twin three-dimensional model corresponding to a picture to be processed based on the twin basic three-dimensional model corresponding to the modules, and further, the modeling accuracy of the twin model is improved.
To further describe the twin three-dimensional model generation method provided by the present invention, the following embodiments will be described.
FIG. 2 is a second schematic flow chart of the twin three-dimensional model generation method provided by the present invention.
In an exemplary embodiment of the present invention, the twin three-dimensional model generating method may include steps 210 to 260, wherein steps 210 to 220 are the same as or similar to steps 110 to 120, and steps 250 to 260 are the same as or similar to steps 140 to 150, and the detailed description and advantages thereof refer to the foregoing description, and steps 230 and 240 will be described below.
In step 230, a plurality of base three-dimensional models corresponding to the module are recalled in a three-dimensional base model database based on the text labels and the geometric features.
In one embodiment, a plurality of basic three-dimensional models corresponding to the modules can be recalled in the three-dimensional basic model database according to the text labels and the geometric characteristics of the modules in the picture to be processed. It is understood that a module may correspond to a plurality of elementary three-dimensional models. During application, one basic three-dimensional model most similar to the module can be selected from a plurality of basic three-dimensional models. Wherein, the recall model can be a deep recall model based on a deep neural network (such as YouTubeDNN, DSSM, etc.). In the embodiment, the comprehensiveness and the accuracy of the recalled basic three-dimensional models can be ensured by multi-path recall of the plurality of basic three-dimensional models, and a foundation is laid for improving the modeling accuracy of the twin model.
In step 240, based on the view angle features and the image texture features, view angle adjustment and texture rendering are performed on the basic three-dimensional model, respectively, so as to obtain a plurality of two-dimensional images corresponding to the plurality of basic three-dimensional models.
In an embodiment, in order to enable a basic three-dimensional model (corresponding to a twin basic three-dimensional model) most similar to a module to be selected from a plurality of basic three-dimensional models more simply, clearly and accurately, the basic three-dimensional model may be subjected to view angle adjustment and texture rendering respectively based on view angle characteristics and picture texture characteristics, so as to obtain a basic three-dimensional model having the same view angle or texture as a picture to be processed, and a two-dimensional picture corresponding to the basic three-dimensional model is obtained based on the adjusted basic three-dimensional model. It can be understood that, since the adjusted basic three-dimensional model has the same view angle or texture as the picture to be processed, the two-dimensional picture also has the same view angle or texture as the picture to be processed. It can be understood that which two-dimensional picture is most similar to the module of the picture to be processed can be more accurately determined based on the two-dimensional picture.
A process of determining the twin basic three-dimensional model corresponding to the module based on the module and the plurality of two-dimensional pictures will be described with reference to the following embodiments.
FIG. 3 is a schematic flow chart of determining a twin basic three-dimensional model corresponding to a module based on the module and a plurality of two-dimensional pictures.
In an exemplary embodiment of the present invention, as shown in fig. 3, determining the twin basic three-dimensional model corresponding to the module based on the module and the plurality of two-dimensional pictures may include steps 310 to 330, which will be described separately below.
In step 310, based on the module and the plurality of two-dimensional pictures, the similarity values between the module and each two-dimensional picture are respectively determined through similarity calculation, and the similarity recommendation values between the module and each two-dimensional picture are respectively determined through collaborative filtering calculation.
In step 320, a candidate three-dimensional model probability distribution corresponding to the module is generated based on the similarity value and the similarity recommendation value.
In one embodiment, the similarity value between each calculation module and each two-dimensional picture corresponding to the calculation module can be calculated through similarity calculation, and the similarity recommendation value between each calculation module and each two-dimensional picture corresponding to the calculation module can be calculated through collaborative filtering calculation. In the application process, the similarity degree between the module and each two-dimensional picture can be obtained through the similarity value and the similarity recommendation value, the similarity degrees are ranked, and the probability distribution of the candidate three-dimensional model corresponding to the module can be obtained. In one example, in the sorting process, a recommendation algorithm based on user characteristics, such as an FM recommendation algorithm, a GBDT + LR recommendation algorithm, a Wide & Deep recommendation algorithm, a Deep FM recommendation algorithm, and the like, may be used.
In step 330, a twin base three-dimensional model corresponding to the module is determined based on the candidate three-dimensional model probability distributions.
In one embodiment, a twin base three-dimensional model corresponding to a module may be determined based on a candidate three-dimensional model probability distribution. Further, the twin three-dimensional model corresponding to the picture to be processed can be obtained through the determined twin basic three-dimensional model of each module.
To further describe the twin three-dimensional model generation method provided by the present invention, the following embodiments will be described.
FIG. 4 is a third schematic flow chart of a twin three-dimensional model generation method provided by the present invention.
In an exemplary embodiment of the present invention, as shown in fig. 4, the twin three-dimensional model generation method may include steps 410 to 490, wherein steps 410 to 440 are the same as or similar to steps 210 to 240, step 460 is the same as or similar to step 310, step 480 is the same as or similar to step 330, and step 490 is the same as or similar to step 260, and the detailed description and advantageous effects thereof refer to the foregoing description, and step 450 and step 470 will be described below, respectively.
In step 450, a probability of association between the associated modules is determined based on the inter-module co-occurrence constraint model of the conditional random field.
In step 470, a candidate three-dimensional model probability distribution corresponding to the module is generated based on the similarity value, the similarity recommendation value, and the association probability.
In one embodiment, the effect of the relationships between the modules on the twin base three-dimensional model may be considered. During application, the association probability between the associated modules can be determined based on the inter-module co-occurrence constraint model of the conditional random field. The related module refers to a module with a connection relation, a position relation or a logic relation. In an example, the association probabilities between the associated modules may also be determined by other matching probabilistic graphical models, such as a Markov random field probabilistic graphical model and a Bayesian network probabilistic graphical model.
Further, the similarity degree between the module and each two-dimensional picture can be obtained based on the similarity value between the module and each two-dimensional picture, the similarity recommended value between the module and each two-dimensional picture, and the association probability between the associated modules. And sequencing the similarity degrees to obtain the probability distribution of the candidate three-dimensional model corresponding to the module. In one example, in the sorting process, a recommendation algorithm based on user characteristics, such as an FM recommendation algorithm, a GBDT + LR recommendation algorithm, a Wide & Deep recommendation algorithm, a Deep FM recommendation algorithm, and the like, may be used. In the embodiment, the probability distribution of the candidate three-dimensional models corresponding to the modules is determined based on the association probability among the associated modules, and obviously unreasonable three-dimensional models can be eliminated, so that the calculation amount is reduced.
In an exemplary embodiment of the present invention, generating a probability distribution of a candidate three-dimensional model corresponding to a module based on the similarity value, the similarity recommendation value, and the association probability may be implemented as follows:
and determining the probability distribution of the candidate three-dimensional model corresponding to the module based on the similarity value, the similarity recommendation value and the weighted average value of the association probability, wherein the weight value in the weighting calculation process is a hyperparameter. In this embodiment, determining the weight value as a hyper-parameter can increase the rationality of the probability distribution of the candidate three-dimensional model corresponding to the module by making full use of the historical data.
To further describe the twin three-dimensional model generation method provided by the present invention, the following embodiments will be described.
In an embodiment, a single to-be-processed picture may be parsed by a picture parsing module, so as to obtain a plurality of modules (module 1, module 2 … …, module n) corresponding to the to-be-processed picture, and module picture features corresponding to the modules, where the module picture features may include, but are not limited to, text labels, geometric features, view angle features, and picture texture features. Further, a plurality of basic three-dimensional models corresponding to the modules (module 1, module 2 … … module n) can be obtained by recalling candidate sets of text labels and geometric features corresponding to the modules (module 1, module 2 … … module n) based on the parameterized basic three-dimensional model database (corresponding to the three-dimensional basic model database).
In order to select a basic three-dimensional model (corresponding to the twin basic three-dimensional model) most similar to the module from the plurality of basic three-dimensional models more simply, clearly and accurately, the basic three-dimensional models can be subjected to visual angle adjustment and texture rendering respectively based on the visual angle characteristics and the image texture characteristics to obtain the basic three-dimensional models with the same visual angle or texture as the image to be processed, and the two-dimensional images corresponding to the basic three-dimensional models are obtained based on the adjusted basic three-dimensional models.
In the application process, the candidate building block probability distribution (corresponding candidate three-dimensional model probability distribution) of each module (module 1, module 2 … … module n) can be obtained based on the similarity between each module and each corresponding two-dimensional picture and the recommendation ranking of collaborative filtering between each module and each corresponding two-dimensional picture. During application, the twin base three-dimensional model corresponding to the module may be determined based on the candidate three-dimensional model probability distribution. Further, a twin three-dimensional model corresponding to the picture to be processed is obtained through the determined twin basic three-dimensional models of the modules.
In yet another example, the impact of the relationships between the modules on the twin base three-dimensional model may also be considered. During application, the association probability between the associated modules can be determined based on the co-occurrence constraint model between the modules of the conditional random field. And determining candidate twin three-dimensional models (corresponding to the twin three-dimensional models) about the picture to be processed formed by the modules according to the association probability among the associated modules and the probability distribution of the candidate building blocks.
In yet another example, the target base three-dimensional model may be determined to be a twin base three-dimensional model corresponding to the module in response to the user selecting the target two-dimensional picture among the plurality of two-dimensional pictures, wherein the target base three-dimensional model is the base three-dimensional model corresponding to the target two-dimensional picture. By the embodiment, user interaction can be realized, the twin basic three-dimensional model corresponding to the module can be rapidly determined through the user interaction, and further the twin model modeling efficiency can be greatly improved.
According to the description, the twin three-dimensional model generation method provided by the invention recalls the multiple basic three-dimensional models corresponding to the module in a multi-layer manner through the module picture characteristics, determines the twin basic three-dimensional model corresponding to the module based on the multiple two-dimensional pictures corresponding to the multiple basic three-dimensional models, and obtains the twin three-dimensional model corresponding to the picture to be processed according to the twin basic three-dimensional model. The digital twin three-dimensional model with higher similarity to the picture to be processed can be generated through multi-level recall, and the modeling accuracy of the twin model is greatly improved.
Based on the same conception, the invention also provides a twin three-dimensional model generation device.
The twin three-dimensional model generation device provided by the present invention is described below, and the twin three-dimensional model generation device described below and the twin three-dimensional model generation method described above may be referred to in correspondence with each other.
FIG. 5 is a schematic structural diagram of a twin three-dimensional model generating apparatus provided in the present invention.
In an exemplary embodiment of the present invention, as shown in fig. 5, the twin three-dimensional model generating apparatus may include an obtaining module 510, a detecting module 520, a recalling module 530, a processing module 540 and a generating module 550, which will be described below respectively.
The acquisition module 510 may be configured to acquire a picture to be processed and a three-dimensional base model database.
The detection module 520 may be configured to perform module segmentation and feature detection on the to-be-processed picture, respectively, to obtain a plurality of modules corresponding to the to-be-processed picture and module picture features corresponding to the modules, where the module picture features at least include a text label, a geometric feature, a view angle feature, and a picture texture feature.
The recalling module 530 may be configured to recall a plurality of basic three-dimensional models corresponding to the module in the three-dimensional basic model database based on the module picture features, and perform image processing on the basic three-dimensional models respectively to obtain a plurality of two-dimensional pictures corresponding to the plurality of basic three-dimensional models.
The processing module 540 may be configured for determining a twin elementary three-dimensional model corresponding to a module based on the module and the plurality of two-dimensional pictures, wherein the twin elementary three-dimensional model is the most similar elementary three-dimensional model to the module.
The generating module 550 may be configured to obtain a twin three-dimensional model corresponding to the picture to be processed based on the twin basic three-dimensional models corresponding to the modules.
In an exemplary embodiment of the present invention, the module picture features may further include a tree-level relationship between modules, and the detection module 520 may perform module segmentation and feature detection on the picture to be processed respectively in the following manners to obtain a plurality of modules corresponding to the picture to be processed and module picture features corresponding to the modules: hierarchical module segmentation and feature detection are respectively carried out on the picture to be processed, so that a plurality of modules corresponding to the picture to be processed and inter-module tree-level relations corresponding to the modules are obtained, wherein the inter-module tree-level relations represent position relations among different modules.
In an exemplary embodiment of the present invention, the recalling module 530 can recall a plurality of basic three-dimensional models corresponding to the module in the three-dimensional basic model database based on the module picture features in the following manner: recalling a plurality of basic three-dimensional models corresponding to the modules in a three-dimensional basic model database based on the text labels and the geometric features; the recall module 530 may further perform image processing on the basic three-dimensional model in the following manner to obtain a plurality of two-dimensional pictures corresponding to the plurality of basic three-dimensional models: and respectively carrying out visual angle adjustment and texture rendering on the basic three-dimensional model based on the visual angle characteristics and the picture texture characteristics to obtain a plurality of two-dimensional pictures corresponding to the plurality of basic three-dimensional models.
In an exemplary embodiment of the present invention, the processing module 540 may determine the twin basic three-dimensional model corresponding to the module based on the module and the plurality of two-dimensional pictures in the following manner: respectively determining similarity values of the module and the two-dimensional pictures through similarity calculation based on the module and the two-dimensional pictures, and respectively determining similarity recommendation values of the module and the two-dimensional pictures through collaborative filtering calculation; generating candidate three-dimensional model probability distribution corresponding to the module based on the similarity value and the similarity recommendation value; and determining the twin basic three-dimensional model corresponding to the module based on the probability distribution of the candidate three-dimensional model.
In an exemplary embodiment of the invention, the processing module 540 may be further configured to determine a probability of association between associated modules based on an inter-module co-occurrence constraint model of the conditional random field; the processing module 540 may also generate a candidate three-dimensional model probability distribution corresponding to the module based on the similarity value and the similarity recommendation value in the following manner: and generating candidate three-dimensional model probability distribution corresponding to the module based on the similarity value, the similarity recommendation value and the association probability.
In an exemplary embodiment of the present invention, the processing module 540 may further generate a probability distribution of the candidate three-dimensional model corresponding to the module based on the similarity value, the similarity recommendation value, and the association probability in the following manner: and determining the probability distribution of the candidate three-dimensional model corresponding to the module based on the similarity value, the similarity recommendation value and the weighted average value of the association probability, wherein the weight value in the weighting calculation process is a hyperparameter.
In an exemplary embodiment of the present invention, the processing module 540 may determine the twin basic three-dimensional model corresponding to the module based on the module and the plurality of two-dimensional pictures in the following manner: and in response to the user selecting a target two-dimensional picture from the plurality of two-dimensional pictures, determining the target basic three-dimensional model as a twin basic three-dimensional model corresponding to the module, wherein the target basic three-dimensional model is the basic three-dimensional model corresponding to the target two-dimensional picture.
In an exemplary embodiment of the present invention, the generating module 550 may obtain the twin three-dimensional model corresponding to the picture to be processed based on the twin basic three-dimensional models corresponding to the modules in the following manner: and obtaining a twin three-dimensional model corresponding to the picture to be processed based on the tree level relation among the modules and the twin basic three-dimensional models corresponding to the modules.
Fig. 6 illustrates a physical structure diagram of an electronic device, which may include, as shown in fig. 6: a processor (processor)610, a communication Interface (Communications Interface)620, a memory (memory)630 and a communication bus 640, wherein the processor 610, the communication Interface 620 and the memory 630 communicate with each other via the communication bus 640. The processor 610 may invoke logic instructions in the memory 630 to perform a twin three-dimensional model generation method comprising: acquiring a picture to be processed and a three-dimensional basic model database; respectively carrying out module segmentation and feature detection on the picture to be processed to obtain a plurality of modules corresponding to the picture to be processed and module picture features corresponding to the modules, wherein the module picture features at least comprise text labels, geometric features, view angle features and picture texture features; recalling a plurality of basic three-dimensional models corresponding to the modules in the three-dimensional basic model database based on the module picture characteristics, and respectively carrying out image processing on the basic three-dimensional models to obtain a plurality of two-dimensional pictures corresponding to the basic three-dimensional models; determining a twin elementary three-dimensional model corresponding to the module based on the module and a plurality of the two-dimensional pictures, wherein the twin elementary three-dimensional model is the elementary three-dimensional model most similar to the module; and obtaining a twin three-dimensional model corresponding to the picture to be processed based on each twin basic three-dimensional model corresponding to each module.
In addition, the logic instructions in the memory 630 may be implemented in software functional units and stored in a computer readable storage medium when the logic instructions are sold or used as independent products. Based on such understanding, the technical solution of the present invention or a part thereof which substantially contributes to the prior art may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product, the computer program product comprising a computer program, the computer program being storable on a non-transitory computer-readable storage medium, the computer program, when executed by a processor, being capable of executing the twin three-dimensional model generation method provided by the above methods, the method comprising: acquiring a picture to be processed and a three-dimensional basic model database; respectively carrying out module segmentation and feature detection on the picture to be processed to obtain a plurality of modules corresponding to the picture to be processed and module picture features corresponding to the modules, wherein the module picture features at least comprise text labels, geometric features, view angle features and picture texture features; recalling a plurality of basic three-dimensional models corresponding to the modules in the three-dimensional basic model database based on the module picture characteristics, and respectively carrying out image processing on the basic three-dimensional models to obtain a plurality of two-dimensional pictures corresponding to the basic three-dimensional models; determining a twin basic three-dimensional model corresponding to the module based on the module and a plurality of the two-dimensional pictures, wherein the twin basic three-dimensional model is the basic three-dimensional model most similar to the module; and obtaining a twin three-dimensional model corresponding to the picture to be processed based on each twin basic three-dimensional model corresponding to each module.
In yet another aspect, the present invention also provides a non-transitory computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements a twin three-dimensional model generation method provided by the above methods, the method comprising: acquiring a picture to be processed and a three-dimensional basic model database; respectively carrying out module segmentation and feature detection on the picture to be processed to obtain a plurality of modules corresponding to the picture to be processed and module picture features corresponding to the modules, wherein the module picture features at least comprise text labels, geometric features, view angle features and picture texture features; recalling a plurality of basic three-dimensional models corresponding to the modules in the three-dimensional basic model database based on the module picture characteristics, and respectively carrying out image processing on the basic three-dimensional models to obtain a plurality of two-dimensional pictures corresponding to the basic three-dimensional models; determining a twin basic three-dimensional model corresponding to the module based on the module and a plurality of the two-dimensional pictures, wherein the twin basic three-dimensional model is the basic three-dimensional model most similar to the module; and obtaining a twin three-dimensional model corresponding to the picture to be processed based on each twin basic three-dimensional model corresponding to each module.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
It will be further appreciated that while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in serial order, or that all illustrated operations be performed, to achieve desirable results. In certain environments, multitasking and parallel processing may be advantageous.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (12)
1. A twin three-dimensional model generation method, characterized in that the method comprises:
acquiring a picture to be processed and a three-dimensional basic model database;
respectively carrying out module segmentation and feature detection on the picture to be processed to obtain a plurality of modules corresponding to the picture to be processed and module picture features corresponding to the modules, wherein the module picture features at least comprise text labels, geometric features, view angle features and picture texture features;
recalling a plurality of basic three-dimensional models corresponding to the modules in the three-dimensional basic model database based on the module picture characteristics, and respectively carrying out image processing on the basic three-dimensional models to obtain a plurality of two-dimensional pictures corresponding to the basic three-dimensional models;
determining a twin basic three-dimensional model corresponding to the module based on the module and a plurality of the two-dimensional pictures, wherein the twin basic three-dimensional model is the basic three-dimensional model most similar to the module;
and obtaining a twin three-dimensional model corresponding to the picture to be processed based on each twin basic three-dimensional model corresponding to each module.
2. The twin three-dimensional model generation method according to claim 1, wherein the module picture features further include inter-module tree-level relationships, and the performing module segmentation and feature detection on the to-be-processed picture respectively to obtain a plurality of modules corresponding to the to-be-processed picture and module picture features corresponding to the modules includes:
and respectively carrying out hierarchical module segmentation and feature detection on the picture to be processed to obtain a plurality of modules corresponding to the picture to be processed and inter-module tree-level relations corresponding to the modules, wherein the inter-module tree-level relations represent the position relations among different modules.
3. The twin three-dimensional model generation method according to claim 1 or 2, wherein the recalling a plurality of basic three-dimensional models corresponding to the module in the three-dimensional basic model database based on the module picture feature comprises:
recalling a plurality of base three-dimensional models corresponding to the module in the three-dimensional base model database based on the text labels and the geometric features;
the image processing of the basic three-dimensional model to obtain a plurality of two-dimensional pictures corresponding to the plurality of basic three-dimensional models includes:
and respectively carrying out visual angle adjustment and texture rendering on the basic three-dimensional model based on the visual angle characteristics and the picture texture characteristics to obtain a plurality of two-dimensional pictures corresponding to the plurality of basic three-dimensional models.
4. The twin three-dimensional model generation method according to claim 1, wherein the determining the twin basic three-dimensional model corresponding to the module based on the module and the plurality of two-dimensional pictures comprises:
respectively determining similarity values of the module and the two-dimensional pictures through similarity calculation and respectively determining similarity recommendation values of the module and the two-dimensional pictures through collaborative filtering calculation based on the module and the two-dimensional pictures;
generating a candidate three-dimensional model probability distribution corresponding to the module based on the similarity value and the similarity recommendation value;
determining a twin base three-dimensional model corresponding to the module based on the candidate three-dimensional model probability distribution.
5. The twin three-dimensional model generation method according to claim 4, wherein after said obtaining a plurality of two-dimensional pictures corresponding to a plurality of said basic three-dimensional models, said method further comprises:
determining association probability between associated modules based on an inter-module co-occurrence constraint model of the conditional random field;
generating a candidate three-dimensional model probability distribution corresponding to the module based on the similarity value and the similarity recommendation value, including:
and generating a candidate three-dimensional model probability distribution corresponding to the module based on the similarity value, the similarity recommendation value and the association probability.
6. The twin three-dimensional model generation method of claim 5, wherein generating a candidate three-dimensional model probability distribution corresponding to the module based on the similarity value, the similarity recommendation value, and the association probability comprises:
and determining candidate three-dimensional model probability distribution corresponding to the module based on the similarity value, the similarity recommendation value and the weighted average value of the association probability, wherein the weight value in the weighting calculation process is a hyperparameter.
7. The twin three-dimensional model generation method according to claim 1, wherein the determining a twin elementary three-dimensional model corresponding to the module based on the module and the plurality of two-dimensional pictures comprises:
and in response to the user selecting a target two-dimensional picture from the plurality of two-dimensional pictures, determining a target basic three-dimensional model as a twin basic three-dimensional model corresponding to the module, wherein the target basic three-dimensional model is a basic three-dimensional model corresponding to the target two-dimensional picture.
8. The twin three-dimensional model generation method according to claim 2, wherein the obtaining a twin three-dimensional model corresponding to the picture to be processed based on each of the twin elementary three-dimensional models corresponding to each of the modules includes:
and obtaining a twin three-dimensional model corresponding to the picture to be processed based on the tree level relation among the modules and the twin basic three-dimensional models corresponding to the modules.
9. An apparatus for generating a twin three-dimensional model, the apparatus comprising:
the acquisition module is used for acquiring a picture to be processed and a three-dimensional basic model database;
the detection module is used for respectively carrying out module segmentation and feature detection on the picture to be processed to obtain a plurality of modules corresponding to the picture to be processed and module picture features corresponding to the modules, wherein the module picture features at least comprise text labels, geometric features, view angle features and picture texture features;
the recall module is used for recalling a plurality of basic three-dimensional models corresponding to the module in the three-dimensional basic model database based on the module picture characteristics, and respectively carrying out image processing on the basic three-dimensional models to obtain a plurality of two-dimensional pictures corresponding to the basic three-dimensional models;
a processing module for determining a twin elementary three-dimensional model corresponding to the module based on the module and a plurality of the two-dimensional pictures, wherein the twin elementary three-dimensional model is the elementary three-dimensional model most similar to the module;
and the generating module is used for obtaining the twin three-dimensional model corresponding to the picture to be processed based on each twin basic three-dimensional model corresponding to each module.
10. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the twin three-dimensional model generation method according to any one of claims 1 to 8 when executing the program.
11. A non-transitory computer-readable storage medium having stored thereon a computer program, wherein the computer program, when executed by a processor, implements the twin three-dimensional model generation method according to any one of claims 1 to 8.
12. A computer program product comprising a computer program, characterized in that the computer program, when being executed by a processor, carries out the twin three-dimensional model generation method according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210135449.3A CN114663579A (en) | 2022-02-14 | 2022-02-14 | Twin three-dimensional model generation method and device, electronic device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210135449.3A CN114663579A (en) | 2022-02-14 | 2022-02-14 | Twin three-dimensional model generation method and device, electronic device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114663579A true CN114663579A (en) | 2022-06-24 |
Family
ID=82028412
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210135449.3A Pending CN114663579A (en) | 2022-02-14 | 2022-02-14 | Twin three-dimensional model generation method and device, electronic device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114663579A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116628829A (en) * | 2023-07-24 | 2023-08-22 | 山东融谷信息科技有限公司 | Intelligent building three-dimensional visualization system based on digital twinning |
| CN116647644A (en) * | 2023-06-06 | 2023-08-25 | 上海优景智能科技股份有限公司 | Campus interactive monitoring method and system based on digital twin technology |
-
2022
- 2022-02-14 CN CN202210135449.3A patent/CN114663579A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116647644A (en) * | 2023-06-06 | 2023-08-25 | 上海优景智能科技股份有限公司 | Campus interactive monitoring method and system based on digital twin technology |
| CN116647644B (en) * | 2023-06-06 | 2024-02-20 | 上海优景智能科技股份有限公司 | Campus interactive monitoring method and system based on digital twin technology |
| CN116628829A (en) * | 2023-07-24 | 2023-08-22 | 山东融谷信息科技有限公司 | Intelligent building three-dimensional visualization system based on digital twinning |
| CN116628829B (en) * | 2023-07-24 | 2023-11-07 | 山东融谷信息科技有限公司 | Intelligent building three-dimensional visualization system based on digital twinning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US6606406B1 (en) | System and method for progressive stereo matching of digital images | |
| Saito et al. | Application of genetic algorithms to stereo matching of images | |
| CN114663579A (en) | Twin three-dimensional model generation method and device, electronic device and storage medium | |
| CN111627065A (en) | Visual positioning method and device and storage medium | |
| US20230206603A1 (en) | High-precision point cloud completion method based on deep learning and device thereof | |
| CN112927353B (en) | Three-dimensional scene reconstruction method, storage medium and terminal based on two-dimensional target detection and model alignment | |
| Han et al. | Stereo matching using genetic algorithm with adaptive chromosomes | |
| US20230043026A1 (en) | Learning-based active surface model for medical image segmentation | |
| CN113095333B (en) | Unsupervised feature point detection method and unsupervised feature point detection device | |
| EP3012779A1 (en) | Method and apparatus for extracting feature correspondences from multiple images | |
| CN115019135A (en) | Model training method, target detection method, device, electronic equipment and storage medium | |
| CN116843834A (en) | Three-dimensional face reconstruction and six-degree-of-freedom pose estimation method, device and equipment | |
| CN115439694A (en) | High-precision point cloud completion method and device based on deep learning | |
| JP2001043368A5 (en) | ||
| CN116012387A (en) | Virtual view selection method and device for three-dimensional semantic segmentation of indoor scene | |
| CN111985616B (en) | Image feature extraction method, image retrieval method, device and equipment | |
| CN116229189B (en) | Image processing method, device, equipment and storage medium based on fluorescence endoscope | |
| CN111723688B (en) | Human body action recognition result evaluation method and device and electronic equipment | |
| CN112668710B (en) | Model training, tubular object extraction and data recognition method and equipment | |
| CN114973098A (en) | Short video repetition removing method based on deep learning | |
| CN111275183A (en) | Visual task processing method and device and electronic system | |
| CN112184797B (en) | Method for spatially positioning key part of kilogram group weight | |
| CN114413910B (en) | Visual target navigation method and device | |
| CN116912486A (en) | Target segmentation method based on edge convolution and multidimensional feature fusion and electronic device | |
| Ming et al. | Modeling and inferring 2.1 D sketch with mixed Markov random field |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |