CN118015190A - Autonomous construction method and device of digital twin model - Google Patents
Autonomous construction method and device of digital twin model Download PDFInfo
- Publication number
- CN118015190A CN118015190A CN202410153135.5A CN202410153135A CN118015190A CN 118015190 A CN118015190 A CN 118015190A CN 202410153135 A CN202410153135 A CN 202410153135A CN 118015190 A CN118015190 A CN 118015190A
- Authority
- CN
- China
- Prior art keywords
- target
- digital twin
- data
- twin model
- feature points
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000010276 construction Methods 0.000 title claims abstract description 19
- 238000005516 engineering process Methods 0.000 claims abstract description 24
- 238000013507 mapping Methods 0.000 claims abstract description 18
- 238000007781 pre-processing Methods 0.000 claims abstract description 14
- 238000000034 method Methods 0.000 claims description 72
- 238000012545 processing Methods 0.000 claims description 30
- 238000004422 calculation algorithm Methods 0.000 claims description 22
- 238000000605 extraction Methods 0.000 claims description 12
- 238000003702 image correction Methods 0.000 claims description 5
- 238000003860 storage Methods 0.000 claims description 5
- 230000008569 process Effects 0.000 description 24
- 238000010586 diagram Methods 0.000 description 5
- 230000000694 effects Effects 0.000 description 5
- 238000003384 imaging method Methods 0.000 description 5
- 230000008859 change Effects 0.000 description 3
- 238000001514 detection method Methods 0.000 description 3
- 238000001914 filtration Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000013178 mathematical model Methods 0.000 description 3
- 241000282414 Homo sapiens Species 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 238000012937 correction Methods 0.000 description 2
- 230000033001 locomotion Effects 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 238000004088 simulation Methods 0.000 description 2
- 239000013598 vector Substances 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 206010063385 Intellectualisation Diseases 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000013075 data extraction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
The invention provides an autonomous construction method and device of a digital twin model, comprising the following steps: acquiring field data of a target scene; preprocessing field data to determine target data; extracting initial feature points from target data; performing feature matching on the initial feature points to determine target feature points; establishing a primary digital twin model corresponding to a target scene according to the target feature points based on a three-dimensional reconstruction technology; identification information identifying each item within the target scene; the identification information is associated with corresponding respective items within the primary digital twin model to determine a target digital twin model. According to the invention, three-dimensional reconstruction is carried out through semantic recognition, the problem that management with finer granularity is not available after live-action reconstruction is solved, and automatic association binding is carried out through identification information, so that the problem of mapping binding of a large number of articles in the later period is solved, and the generated digital twin model can be directly applied to application, thereby realizing real digitization.
Description
Technical Field
The application relates to the technical field of computers, in particular to an autonomous construction method and device of a digital twin model.
Background
The digital twin is to fully utilize data such as a physical model, sensor update, operation history and the like, integrate simulation processes of multiple disciplines, multiple physical quantities, multiple scales and multiple probabilities, and complete mapping in a virtual space, thereby reflecting the full life cycle process of corresponding entity equipment. The mapping process of digital twin is independent of a three-dimensional reconstruction technology, the three-dimensional reconstruction refers to the establishment of a mathematical model suitable for computer representation and processing of a three-dimensional object, is a basis for processing, operating and analyzing the property of the three-dimensional object in a computer environment, and is also a key technology for establishing virtual reality expressing objective world in a computer.
In the prior art, three-dimensional reconstruction is generally completed by scanning a laser radar, but the reconstructed model is mostly a live-action reconstruction model, and although the model can be truly restored, the three-dimensional reconstruction is inconvenient in digital twin application, and the main reasons are as follows: on one hand, the reconstructed model is a whole and cannot be split according to the management granularity; on the other hand, the articles in the model cannot be associated and bound with the data in the service system, so that the digital twin digital application is affected.
Disclosure of Invention
The invention provides an autonomous construction method and device for a digital twin model, which carry out three-dimensional reconstruction through semantic recognition, solve the problem that management of finer granularity is not available after live-action reconstruction, and carry out automatic association binding through identification information, solve the problem of mapping binding of a large number of articles in the later stage, so that the generated digital twin model can be directly applied to application, and realize real digitization.
In a first aspect, the present invention provides a method for autonomous construction of a digital twin model, the method comprising:
acquiring field data of a target scene;
preprocessing the field data to determine target data;
extracting initial feature points from the target data; performing feature matching on the initial feature points to determine target feature points;
establishing a primary digital twin model corresponding to the target scene according to the target feature points based on a three-dimensional reconstruction technology;
Identification information identifying each item within the target scene;
and carrying out association binding on the identification information and corresponding various articles in the primary digital twin model so as to determine a target digital twin model.
Preferably, the acquiring the field data of the target scene includes:
Collecting image data and/or video data of a target site;
The live data is determined from the image data and/or the video data.
Preferably, the preprocessing the field data to determine target data includes:
Performing noise removal processing on the field data to determine first data;
performing image correction processing on the first data to determine second data;
performing camera calibration processing on the second data to determine third data;
And performing image alignment processing on the third data to determine the target data.
Preferably, the extracting of the initial feature points from the target data; performing feature matching on the initial feature points to determine target feature points includes:
extracting initial feature points from the target data by adopting a feature extraction algorithm;
and determining the target feature points from the initial feature points by adopting a feature matching algorithm.
Preferably, the establishing a primary digital twin model corresponding to the target scene according to the target feature points based on the three-dimensional reconstruction technology comprises:
Determining a dense point cloud corresponding to the target scene according to the target feature points based on the three-dimensional reconstruction technology;
sequentially carrying out surface reconstruction and texture mapping on the dense point cloud to determine a virtual scene corresponding to the target scene;
and determining the primary digital twin model according to the virtual scene.
Preferably, said determining said primary digital twin model from said virtual scene comprises:
determining a three-dimensional space structure of the target scene;
Determining semantic information of each component in the virtual scene;
and determining the primary digital twin model according to the semantic information and the three-dimensional space structure.
Preferably, the method further comprises:
Identifying replacement items within the target scene to determine physical information of the replacement items;
reconstructing the target digital twin model according to the physical information so as to update the target digital twin model.
In a second aspect, the present invention provides an autonomous building apparatus of a digital twin model, comprising:
The field data acquisition module is used for acquiring field data of a target scene;
the target data determining module is used for preprocessing the field data to determine target data;
The target feature point determining module is used for extracting initial feature points from the target data; performing feature matching on the initial feature points to determine target feature points;
the primary digital twin model building module is used for building a primary digital twin model corresponding to the target scene according to the target characteristic points based on a three-dimensional reconstruction technology;
the identification information identification module is used for identifying the identification information of each article in the target scene;
and the target digital twin model determining module is used for carrying out association binding on the identification information and corresponding articles in the primary digital twin model so as to determine the target digital twin model.
In a third aspect, the present invention provides a readable medium comprising execution instructions which, when executed by a processor of an electronic device, perform the method according to any of the first aspects.
In a fourth aspect, the present invention provides an electronic device comprising a processor and a memory storing execution instructions, the processor performing the method according to any one of the first aspects when executing the execution instructions stored in the memory.
The invention provides an autonomous construction method and device of a digital twin model, which are characterized in that field data of a target scene are collected, the field data are preprocessed to determine the target data, initial feature points are extracted from the target data, feature matching is carried out on the initial feature points to determine the target feature points, and a primary digital twin model corresponding to the target scene is established according to the target feature points based on a three-dimensional reconstruction technology; identifying the identification information of each article in the target scene, and carrying out association binding on the identification information and each corresponding article in the primary digital twin model to determine the target digital twin model, thereby completing the mapping process of the whole digital twin. According to the invention, three-dimensional reconstruction is carried out through semantic recognition, the problem that management with finer granularity is not available after live-action reconstruction is solved, and automatic association binding is carried out through identification information, so that the problem of mapping binding of a large number of articles in the later period is solved, and the generated digital twin model can be directly applied to application, thereby realizing real digitization.
Further effects of the above-described non-conventional preferred embodiments will be described below in connection with the detailed description.
Drawings
In order to more clearly illustrate the embodiments of the invention or the prior art solutions, the drawings which are used in the description of the embodiments or the prior art will be briefly described below, it being obvious that the drawings in the description below are only some of the embodiments described in the present invention, and that other drawings can be obtained according to these drawings without inventive faculty for a person skilled in the art.
FIG. 1 is a schematic diagram of an autonomous construction method of a digital twin model according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of another method for autonomous construction of a digital twin model according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of an autonomous building apparatus of a digital twin model according to an embodiment of the present invention;
Fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to specific embodiments and corresponding drawings. It will be apparent that the described embodiments are only some, but not all, embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Digital twinning has become a popular technological application in more and more industries today where human society is highly developing digital technology. The digital twin can help human beings digitally copy objects, systems, processes and the like in the physical world of reality so as to achieve the goal of integrating virtual and reality, and improve the business efficiency and the operation value in different industries. Is a concept that combines the physical world with the digital world.
It simulates and reflects a physical entity or process by creating a real-time, interactive digital model. The digital twin can be used in various fields including manufacturing industry, energy, traffic, medical treatment and the like, and the intellectualization and optimization of various devices in a space environment are realized by combining the technology of artificial intelligence, big data analysis, internet of things and the like.
The digital twin is to fully utilize data such as a physical model, sensor update, operation history and the like, integrate simulation processes of multiple disciplines, multiple physical quantities, multiple scales and multiple probabilities, and complete mapping in a virtual space, thereby reflecting the full life cycle process of corresponding entity equipment. The mapping process of digital twin is independent of a three-dimensional reconstruction technology, the three-dimensional reconstruction refers to the establishment of a mathematical model suitable for computer representation and processing of a three-dimensional object, is a basis for processing, operating and analyzing the property of the three-dimensional object in a computer environment, and is also a key technology for establishing virtual reality expressing objective world in a computer.
In the prior art, three-dimensional reconstruction is generally completed by scanning a laser radar, but the reconstructed model is mostly a live-action reconstruction model, and although the model can be truly restored, the three-dimensional reconstruction is inconvenient in digital twin application, and the main reasons are as follows: on one hand, the reconstructed model is a whole and cannot be split according to the management granularity; on the other hand, the articles in the model cannot be associated and bound with the data in the service system, so that the digital twin digital application is affected.
Therefore, how to complete the mapping of the physical world and the virtual world, provide a digital foundation for the subsequent digital twin application, and after the reconstruction of the whole scene is completed, the association binding of all the minimum granularity objects to be managed is of great importance in the construction of the whole twin world.
In view of this, the present invention provides an autonomous construction method of a digital twin model. Referring to fig. 1, a specific embodiment of an autonomous construction method of a digital twin model is provided in the present invention.
In this embodiment, the method includes:
step 101, acquiring field data of a target scene;
The target scene in this embodiment refers to an actual physical scene in which a digital twin body needs to be established to perform digital twin mapping. Such as factory, hospital, school, community, etc. The field data refers to a two-dimensional image of a three-dimensional object in a target scene, and can comprise image data and/or video data, and the field data can be acquired by using a camera, a laser scanner, a depth camera and other devices; or by extracting data from existing images or videos; or the mobile robot integrating the scanning and camera shooting functions can acquire the scanning and camera shooting signals in the moving process along the preset path. It should be noted that the field data of the target scene are collected, the geometric characteristics and the illumination conditions of the collecting device have a great influence on the subsequent construction of the digital twin model, and the collecting device with good environmental conditions and high performance is required to collect the field data, so that the generated digital twin model has better effect.
102, Preprocessing field data to determine target data;
The target data in this embodiment refers to data that can extract features, and because there are often few problems in the process of collecting the field data, such as distortion, noise, etc., this may cause erroneous judgment on the feature extraction, so that preprocessing of the field data is required. The main purpose of preprocessing is to eliminate extraneous information in the image, recover useful real information, enhance the detectability of the relevant information and maximally simplify the data, thereby improving the reliability of feature extraction, image segmentation, matching and recognition. Therefore, the field data preprocessing stage is mainly one process of screening, denoising and correcting the image.
Specifically, noise-removing processing is performed on the field data to determine first data; performing image correction processing on the first data to determine second data; performing camera calibration processing on the second data to determine third data; image alignment processing is performed on the third data to determine target data.
In which unnecessary signals are inevitably introduced during the shooting and transmission of the image, which signals affect the quality of the image and produce corresponding disturbances in the subsequent image processing, these disturbance signals being called noise. The noise removal processing is to remove the unnecessary signals in the image, improve the image quality and better perform the next image processing operation. Noise removal processing is the basis and premise of image processing, and is an important link in the image preprocessing stage. The noise removal process may be performed in the following ways: a. the mean value filtering effectively eliminates some Gaussian noise; b. median filtering, namely taking pixels as centers, taking a designated sliding window shape as a neighborhood, sequencing pixels in the neighborhood, and removing and assigning a median result to the pixels in the neighborhood. The method is easy to remove some isolated noise and can retain most of edge information. c. The Gaussian filtering is not simple averaging or median, but a two-dimensional discrete Gaussian function is called to remove noise, more edge details can be reserved, the image is clearer, and the smoothing effect is softer.
The image correction processing refers to a restorative processing performed on a distorted image. The causes of image distortion are: image distortion caused by aberration, distortion, bandwidth limitation, etc. of the imaging system; image geometric distortion caused by imaging device shooting posture and scanning nonlinearity; image distortion due to motion blur, radiation distortion, introduced noise, and the like. The image correction process can be divided into geometric correction and gray correction, and can be directly operated by existing software or mathematical models.
In image measurement processes and machine vision applications, in order to determine the correlation between the three-dimensional geometric position of a point on the surface of a spatial object and its corresponding point in the image, a geometric model of camera imaging must be established, and these geometric model parameters are camera parameters. Under most conditions, these parameters must be obtained through experiments and calculations, and this process of solving the parameters is called camera calibration (or camera calibration). In image measurement or machine vision application, the calibration of camera parameters is a very critical link, and the accuracy of the calibration result and the stability of the algorithm directly influence the accuracy of the establishment of a subsequent digital twin model. The camera calibration method can be performed by a traditional camera calibration method, an active vision camera calibration method, a camera self-calibration method and a zero-distortion camera calibration method.
The image alignment process refers to arranging and adjusting a plurality of images in such a manner that their positions, directions, and dimensions are kept uniform. The purpose of this approach is to facilitate subsequent processing, such as image fusion, image stitching, image matching, etc. Common image alignment methods include a feature point-based method and a depth information-based method. Feature point-based methods achieve image alignment by finding points with higher features in multiple images and calculating their coordinates in each image. The depth information-based method achieves image alignment by calculating depth information of each pixel point in a three-dimensional space. Because the equipment parameters of the image shooting in the field data are different, the two different images of the same article can be inconsistent in angle, and the pictures shot for multiple times can be spliced together through the image alignment processing, so that the field scene can be accurately restored at different angles.
The target data formed by preprocessing the field data can selectively strengthen and inhibit the information of the image in the target data so as to improve the visual effect of the image, and convert the image into a form more suitable for the processing of the subsequent steps so as to facilitate the data extraction or recognition.
Step 103, extracting initial feature points from target data; performing feature matching on the initial feature points to determine target feature points;
The initial feature points are points with obvious features, convenient detection and matching in the picture, such as corners and edge points of a building, and feature points which can be extracted according to the gray level or corner characteristics of the image in the picture are important components for constructing three-dimensional features. It has descriptors that can uniquely describe the pixel feature, typically the initial feature point consists of a key point and descriptors. Feature extraction and feature matching are the most critical steps of three-dimensional reconstruction based on multiple views, and the accuracy of the feature extraction and feature matching is directly related to the accuracy of the calculated projection matrix, so that the whole reconstruction result is affected. Feature extraction is to digitize certain features of the identified object, while the reflected image features are more similar, such as corners, spots, regions, edges, etc. Feature matching is to find the same feature point of the same object in different images, and each feature point is provided with a descriptor marked with unique identity and characteristics, so that feature matching is to find two feature points with similar descriptors in two images. Thus, the target feature point is the composition of the initial feature points that are successfully matched. Since the target feature point is a portion that is representative among images, the same region thereof can be found in different images. Different regions have different expression. The single initial feature point contains relatively less information, and only the position coordinate information of the single initial feature point in the image can be reflected, so that the matched target feature point is found in a plurality of images, and accurate three-dimensional features can be constructed in an auxiliary mode.
In general, a feature extraction algorithm may be employed to extract initial feature points from the target data; and determining the target feature points from the initial feature points by adopting a feature matching algorithm.
Specifically, the feature extraction algorithm may include, according to the initial feature point category: edge extraction, sobel algorithm, prewitt algorithm, canny algorithm and the like; extracting corner points, a Harris corner point detection algorithm, a CSS corner point detection algorithm and the like; spot extraction, doG algorithm, SIFT, SURF algorithm, etc.; region feature extraction, MSER algorithm, and the like.
The feature matching algorithm performs matching according to the similarity of feature vectors formed by the initial feature points, and the smaller the distance between the feature vectors described by the two initial feature points by adopting a distance function, the greater the probability that the two initial feature points correspond to the same point in the actual space. A particular problem with such feature matching is how to find pairwise nearest matching pairs from the set of two initial feature points that have been detected. The specific feature matching algorithm can be roughly divided into three types, namely, a violent search, namely, an exhaustion method, namely, each point in the initial feature point set is compared with other points in the corresponding set in distance; the second category is by creating data index and then matching, such as K-d tree algorithm; the third category is to reduce the matching range through a certain constraint relation and perform feature matching according to the feature auxiliary normalization cross-correlation method of epipolar geometry.
Because the imaging view field is changed due to camera movement, the same object can appear at different positions in the image, the position of the object in the new image can be quickly positioned through feature matching, a corresponding relation between image pairs is established according to the extracted initial feature points, namely imaging points of the same physical space point in two different images are in one-to-one correspondence to form target feature points, and three-dimensional information of a target scene is gradually restored by taking the target feature points as references.
Step 104, establishing a primary digital twin model corresponding to the target scene according to the target feature points based on a three-dimensional reconstruction technology;
The three-dimensional reconstruction technology is a technology of acquiring a data image of a scene object through image acquisition equipment, analyzing and processing the image by utilizing a related algorithm or software, and deducing three-dimensional information of the object in a real environment by combining with computer vision knowledge. And determining dense point clouds corresponding to the target scene according to the target feature points based on the three-dimensional reconstruction technology, and sequentially carrying out surface reconstruction and texture mapping on the dense point clouds to determine a virtual scene corresponding to the target scene. Determining a three-dimensional space structure of the target scene, determining semantic information of each component in the virtual scene, and determining a primary digital twin model according to the semantic information and the three-dimensional space structure.
Specifically, the target characteristic points in the steps can be formed into sparse point clouds through a three-dimensional reconstruction technology, the sparse point clouds can be formed into dense point clouds by utilizing PMVS algorithm, and after the dense point clouds are formed, the outline, the characteristics and the like of the real object are obviously improved and are basically identifiable. Although the dense point cloud can restore the appearance of the real object more vividly, the dense point cloud is still only a collection of a large amount of isolated three-dimensional space, the real three-dimensional real object needs to be subjected to surface reconstruction, a Power Crust algorithm can be adopted for carrying out surface reconstruction on point cloud data, after reconstruction, the outline and the shape of the real object are clearly visible, but the final step, namely texture mapping, is needed, and the effect of the texture mapping is to enable the reconstructed 3D model to be more similar to the real object, and has the color, texture and detail characteristics of the real object. And (3) carrying out image recognition or point cloud recognition through three-dimensional reconstruction scanning of related point clouds and image information, and analyzing objects in a specific space, such as walls, doors, windows, beds, cabinets, marks and the like.
The objects in the target scene are formed by the three-dimensional reconstruction technology, but the specific placement position, the relative direction and the relative size of the objects need to be distinguished, so that the three-dimensional space structure of the target scene, that is, the space coordinates of the target scene, can be determined, and the three-dimensional space structure can be determined by the camera calibration processing method in step 102. The semantic information in this embodiment refers to semantic tags that are owned by each item in the virtual scene, such as objects (e.g., classes, materials, shapes, and other attributes), scene categories, material types, three-dimensional shapes, and so on. Different items have different semantic information. Based on the inferred semantic information of images, point clouds and the like in the virtual scene, the distance or depth information of objects in the target scene is calculated by combining the three-dimensional space structure, the geometric information of the positions, the sizes, the directions and the like of the objects is inferred, and finally each object is placed to form a primary digital twin model.
Step 105, identifying identification information of each article in the target scene;
A primary digital twin model is determined, and the items in the primary digital twin model cannot be associated and bound with data in a service system, namely digital application of digital twin cannot be formed. It is therefore also necessary to correlate the actual items within the target scene with the corresponding items in the primary digital twin model. Thus, identification information identifying individual items within the target scene is required. Each item in the target scene may be given an identification, which may be a two-dimensional code, a bar code, a digital label, etc. Each identity is associated with information about the associated item, such as the name, size, effect, date, work content, principle, and digital asset bound thereto. The identification of each article is scanned by the relevant scanning equipment, so that the identification information of each article in the target scene is identified.
And 106, performing association binding on the identification information and corresponding various articles in the primary digital twin model to determine a target digital twin model.
The primary digital twin model can reconstruct the identification of each article in the target scene in the primary digital twin model through a three-dimensional reconstruction technology, and the identification information is associated and bound with the corresponding articles in the primary digital twin model, so that each article in the formed target digital twin model has the same representative meaning as each article in the actual target scene. In the process of association binding, on one hand, the identification information of the article and the primary digital twin model can be compared to judge whether the identification is wrong or not; on the other hand, the mapping of the physical world and the virtual world can be completed, and a digital foundation is provided for subsequent digital twin application. After the reconstruction of the whole target scene is completed, all minimum granularity articles to be managed are associated and bound, so that the whole twin world is constructed and can be directly used in subsequent applications, and the target digital twin model can be regarded as the mapping of the whole target scene.
According to the invention, three-dimensional reconstruction is carried out through semantic recognition, the problem that management with finer granularity is not available after live-action reconstruction is solved, and automatic association binding is carried out through identification information, so that the problem of mapping binding of a large number of articles in the later period is solved, and the generated digital twin model can be directly applied to application, thereby realizing real digitization.
Fig. 1 shows only a basic embodiment of the method according to the invention, on the basis of which certain optimizations and developments are made, but other preferred embodiments of the method can also be obtained.
Fig. 2 shows another embodiment of an autonomous construction method of a digital twin model according to the present invention. This embodiment is further described on the basis of the foregoing embodiment, in which the method includes the following steps:
step 201, collecting field data of a target scene;
step 202, preprocessing field data to determine target data;
step 203, extracting initial feature points from target data; performing feature matching on the initial feature points to determine target feature points;
step 204, establishing a primary digital twin model corresponding to the target scene according to the target feature points based on a three-dimensional reconstruction technology;
Step 205, identifying identification information of each item in the target scene;
step 206, associating and binding the identification information with corresponding articles in the primary digital twin model to determine the target digital twin model.
Steps 201 to 206 in this embodiment are identical to steps 101 to 106, and will not be described here again.
Step 207, identifying the replacement object in the target scene to determine physical information of the replacement object;
Items within the target scene may not be a change and once a change is involved, the target digital twin model needs to be updated to ensure the integrity and uniformity of the target digital twin model. Specifically, the replacement object in the target scene is identified to determine the physical information of the replacement object, wherein the physical information can include geometric information such as the position, the size, the direction, the appearance and the like of the replacement object, and the physical information of the replacement object is determined by scanning the replacement object through the corresponding sensor and identifying the replacement object. The replacement item may be a specific replacement item within the target scene or a corresponding structural or dimensional change within the target scene.
And step 208, reconstructing the target digital twin model according to the physical information so as to update the target digital twin model.
And reconstructing the position, the size, the direction and the appearance of the replacement object by taking the target digital twin model as a reference on the basis of the three-dimensional reconstruction technology by combining with the physical information, so as to finish the updating of the target digital twin model. The current target digital twin model is used as a general model, and the replacement object is locally updated according to the physical information, so that the data volume in the data operation process in the model forming process is reduced, and the integrity and the uniformity of the target digital twin model can be ensured.
Referring now to FIG. 3, an embodiment of an autonomous device for constructing a digital twin model according to the present invention is shown. The apparatus described in this embodiment is a physical apparatus for performing the method described in fig. 1-2. The technical solution is essentially identical to the above embodiment, and the corresponding description in the above embodiment is also applicable to this embodiment. The device in this embodiment includes:
a field data acquisition module 301 configured to acquire field data of a target scene;
a target data determination module 302 configured to pre-process field data to determine target data;
a target feature point determination module 303 configured to extract initial feature points from the target data; performing feature matching on the initial feature points to determine target feature points;
a primary digital twin model building module 304 configured to build a primary digital twin model corresponding to a target scene from the target feature points based on a three-dimensional reconstruction technique;
An identification information identifying module 305 configured to identify identification information of each item within the target scene;
a target digital twin model determination module 306 configured to associate and bind the identification information with corresponding respective items within the primary digital twin model to determine the target digital twin model.
Fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present invention. At the hardware level, the electronic device comprises a processor, optionally an internal bus, a network interface, a memory. The Memory may include a Memory, such as a Random-Access Memory (RAM), and may further include a non-volatile Memory (non-volatile Memory), such as at least 1 disk Memory. Of course, the electronic device may also include hardware required for other services.
The processor, network interface, and memory may be interconnected by an internal bus, which may be an ISA (Industry Standard Architecture ) bus, a PCI (PERIPHERAL COMPONENT INTERCONNECT, peripheral component interconnect standard) bus, or EISA (Extended Industry Standard Architecture ) bus, among others. The buses may be classified as address buses, data buses, control buses, etc. For ease of illustration, only one bi-directional arrow is shown in FIG. 4, but not only one bus or type of bus.
And the memory is used for storing the execution instruction. In particular, a computer program that executes instructions may be executed. The memory may include memory and non-volatile storage and provide the processor with instructions and data for execution.
In one possible implementation manner, the processor reads the corresponding execution instruction from the nonvolatile memory into the memory and then executes the execution instruction, and may also acquire the corresponding execution instruction from other devices, so as to form an autonomous construction device of the digital twin model on a logic level. The processor executes the execution instructions stored in the memory to implement an autonomous construction method of the digital twin model provided in any embodiment of the present invention by executing the execution instructions.
The method executed by the autonomous building apparatus for a digital twin model according to the embodiment of the present invention shown in fig. 3 may be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in a processor or by instructions in the form of software. The processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU), a network processor (Network Processor, NP), etc.; but may also be a digital signal Processor (DIGITAL SIGNAL Processor, DSP), application SPECIFIC INTEGRATED Circuit (ASIC), field-Programmable gate array (Field-Programmable GATE ARRAY, FPGA) or other Programmable logic device, discrete gate or transistor logic device, discrete hardware components. The disclosed methods, steps, and logic blocks in the embodiments of the present invention may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The steps of the method disclosed in connection with the embodiments of the present invention may be embodied directly in the execution of a hardware decoding processor, or in the execution of a combination of hardware and software modules in a decoding processor. The software modules may be located in a random access memory, flash memory, read only memory, programmable read only memory, or electrically erasable programmable memory, registers, etc. as well known in the art. The storage medium is located in a memory, and the processor reads the information in the memory and, in combination with its hardware, performs the steps of the above method.
The embodiment of the invention also provides a readable medium, wherein the readable storage medium stores execution instructions, and when the stored execution instructions are executed by a processor of electronic equipment, the electronic equipment can be enabled to execute the autonomous construction method of the digital twin model provided in any embodiment of the invention, and the method is particularly used for executing the method shown in fig. 1 and 2.
The electronic device described in the foregoing embodiments may be a computer.
It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method or a computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or a combination of software and hardware aspects.
The embodiments of the present invention are described in a progressive manner, and the same and similar parts of the embodiments are all referred to each other, and each embodiment is mainly described in the differences from the other embodiments. In particular, for the device embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and reference is made to the description of the method embodiments in part.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises the element.
The foregoing is merely exemplary of the present invention and is not intended to limit the present invention. Various modifications and variations of the present invention will be apparent to those skilled in the art. Any modification, equivalent replacement, improvement, etc. which come within the spirit and principles of the invention are to be included in the scope of the claims of the present invention.
Claims (10)
1. An autonomous construction method of a digital twin model, the method comprising:
acquiring field data of a target scene;
preprocessing the field data to determine target data;
extracting initial feature points from the target data; performing feature matching on the initial feature points to determine target feature points;
establishing a primary digital twin model corresponding to the target scene according to the target feature points based on a three-dimensional reconstruction technology;
Identification information identifying each item within the target scene;
and carrying out association binding on the identification information and corresponding various articles in the primary digital twin model so as to determine a target digital twin model.
2. The method of claim 1, wherein the acquiring the field data of the target scene comprises:
Collecting image data and/or video data of a target site;
The live data is determined from the image data and/or the video data.
3. The method of claim 1, wherein the preprocessing the field data to determine target data comprises:
Performing noise removal processing on the field data to determine first data;
performing image correction processing on the first data to determine second data;
performing camera calibration processing on the second data to determine third data;
And performing image alignment processing on the third data to determine the target data.
4. The method of claim 1, wherein the extracting initial feature points from the target data; performing feature matching on the initial feature points to determine target feature points includes:
extracting initial feature points from the target data by adopting a feature extraction algorithm;
and determining the target feature points from the initial feature points by adopting a feature matching algorithm.
5. The method of claim 1, wherein the establishing a primary digital twin model corresponding to the target scene from the target feature points based on the three-dimensional reconstruction technique comprises:
Determining a dense point cloud corresponding to the target scene according to the target feature points based on the three-dimensional reconstruction technology;
sequentially carrying out surface reconstruction and texture mapping on the dense point cloud to determine a virtual scene corresponding to the target scene;
and determining the primary digital twin model according to the virtual scene.
6. The method of claim 5, wherein said determining the primary digital twin model from the virtual scene comprises:
determining a three-dimensional space structure of the target scene;
Determining semantic information of each component in the virtual scene;
and determining the primary digital twin model according to the semantic information and the three-dimensional space structure.
7. The method according to any one of claims 1 to 6, further comprising:
Identifying replacement items within the target scene to determine physical information of the replacement items;
reconstructing the target digital twin model according to the physical information so as to update the target digital twin model.
8. An autonomous construction device of a digital twin model, comprising:
The field data acquisition module is used for acquiring field data of a target scene;
the target data determining module is used for preprocessing the field data to determine target data;
The target feature point determining module is used for extracting initial feature points from the target data; performing feature matching on the initial feature points to determine target feature points;
the primary digital twin model building module is used for building a primary digital twin model corresponding to the target scene according to the target characteristic points based on a three-dimensional reconstruction technology;
the identification information identification module is used for identifying the identification information of each article in the target scene;
and the target digital twin model determining module is used for carrying out association binding on the identification information and corresponding articles in the primary digital twin model so as to determine the target digital twin model.
9. A computer-readable storage medium, characterized in that the computer-readable storage medium comprises a stored program, wherein the program when run performs the method of any one of claims 1 to 7.
10. An electronic device, the electronic device comprising:
A processor;
A memory for storing the processor-executable instructions;
the processor is configured to read the executable instructions from the memory and execute the instructions to implement the method of any one of the preceding claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410153135.5A CN118015190A (en) | 2024-02-02 | 2024-02-02 | Autonomous construction method and device of digital twin model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410153135.5A CN118015190A (en) | 2024-02-02 | 2024-02-02 | Autonomous construction method and device of digital twin model |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN118015190A true CN118015190A (en) | 2024-05-10 |
Family
ID=90949458
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410153135.5A Pending CN118015190A (en) | 2024-02-02 | 2024-02-02 | Autonomous construction method and device of digital twin model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118015190A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118211437A (en) * | 2024-05-22 | 2024-06-18 | 杭州义益钛迪信息技术有限公司 | Digital twin model construction method, device, equipment and storage medium |
-
2024
- 2024-02-02 CN CN202410153135.5A patent/CN118015190A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118211437A (en) * | 2024-05-22 | 2024-06-18 | 杭州义益钛迪信息技术有限公司 | Digital twin model construction method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111063021B (en) | Method and device for establishing three-dimensional reconstruction model of space moving target | |
| US8503760B2 (en) | System and method for real-time object recognition and pose estimation using in-situ monitoring | |
| Choi et al. | Depth analogy: Data-driven approach for single image depth estimation using gradient samples | |
| Ali et al. | Guided image filtering in shape-from-focus: A comparative analysis | |
| US20070185946A1 (en) | Method and apparatus for matching portions of input images | |
| JP5538868B2 (en) | Image processing apparatus, image processing method and program | |
| Nousias et al. | Large-scale, metric structure from motion for unordered light fields | |
| CN118015190A (en) | Autonomous construction method and device of digital twin model | |
| CN112435223B (en) | Target detection method, device and storage medium | |
| GB2566443A (en) | Cross-source point cloud registration | |
| JP4427052B2 (en) | Image processing apparatus and area tracking program | |
| CN115222884A (en) | Space object analysis and modeling optimization method based on artificial intelligence | |
| Park et al. | Neural object learning for 6d pose estimation using a few cluttered images | |
| Lundin et al. | Automatic registration of 2D histological sections to 3D microCT volumes: Trabecular bone | |
| Adam et al. | Objects can move: 3d change detection by geometric transformation consistency | |
| Budianti et al. | Background blurring and removal for 3d modelling of cultural heritage objects | |
| JPH04130587A (en) | Three-dimensional picture evaluation device | |
| CN117218713A (en) | Action resolving method, device, equipment and storage medium | |
| Lai et al. | Computer Vision–ACCV 2016: 13th Asian Conference on Computer Vision, Taipei, Taiwan, November 20-24, 2016, Revised Selected Papers, Part III | |
| CN112288817B (en) | Three-dimensional reconstruction processing method and device based on image | |
| Arevalo et al. | Improving piecewise linear registration of high-resolution satellite images through mesh optimization | |
| Lyra et al. | Development of an efficient 3D reconstruction solution from permissive open-source code | |
| Peng et al. | Deep-Learning-Based Precision Visual Tracking | |
| CN116188497B (en) | Method, device, equipment and storage medium for optimizing generation of DSM (digital image model) of stereo remote sensing image pair | |
| Azawi et al. | Motion Compensated Restoration of Colonoscopy Images |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |